<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Taylan Lakerta on Medium]]></title>
        <description><![CDATA[Stories by Taylan Lakerta on Medium]]></description>
        <link>https://medium.com/@taylan.lakerta?source=rss-e5eac3ccf55------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*iyXFdIcw2eB16XHc</url>
            <title>Stories by Taylan Lakerta on Medium</title>
            <link>https://medium.com/@taylan.lakerta?source=rss-e5eac3ccf55------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:57:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@taylan.lakerta/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[☁️Object Storage Lifecycle Management on Huawei Cloud]]></title>
            <link>https://medium.com/huawei-developers/%EF%B8%8Fobject-storage-lifecycle-management-on-huawei-cloud-0d0694769e88?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/0d0694769e88</guid>
            <category><![CDATA[storage]]></category>
            <category><![CDATA[object-storage-services]]></category>
            <category><![CDATA[huawei-cloud]]></category>
            <category><![CDATA[data-lifecycle]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Tue, 10 Sep 2024 06:01:56 GMT</pubDate>
            <atom:updated>2024-09-10T06:01:56.964Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><p>Hello to everyone! In this article, we`ll introduce you to configuring and managing lifecycle rules for data management purposes on Huawei Cloud with Object Storage Service(OBS). Object Storage Service (OBS) is a scalable service that provides secure, reliable, and cost-effective cloud storage for massive amounts of data. There are tiers within the OBS that provide cost-efficiency and reliability to its users. You can find details about these tiers in the upcoming sections of this article. With OBS Lifecycle management rules a user can easily implement policies to curate data and move it to the most appropriate tier over time. We`ll first introduce to you the storage classes that are available in Huawei Cloud OBS, then create a bucket for the demo and finally create a lifecycle rule via Huawei Cloud Console over a bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*uB39_GIKqvJ4Bcakp5zxNA.png" /><figcaption>Data Storage Lifecycle-General</figcaption></figure><h3>Object Storage Service-Storage Tiers</h3><p>OBS provides unlimited storage capacity for objects of any format, catering to the needs of common users, websites, enterprises, and developers. There is no limitation on the storage capacity of the entire OBS system or of a single bucket, and any number of objects can be stored. As a web service, OBS supports APIs over Hypertext Transfer Protocol (HTTP) and Hypertext Transfer Protocol Secure (HTTPS). You can use OBS Console or OBS tools to access and manage data stored in OBS anytime, anywhere. With OBS SDKs and APIs, you can easily manage data stored in OBS and develop upper-layer applications. You can find further details about OBS in the following <a href="https://support.huaweicloud.com/intl/en-us/obs/index.html">link</a>.</p><p>OBS provides the following storage classes: Standard, Infrequent Access, Archieve, and Deep Archive. These storage classes can meet different needs for storage performance and costs.</p><p>· <strong>Standard Storage Class</strong> features low latency and high throughput. It is therefore good for storing frequently (multiple times per month) accessed files or small files (less than 1 MB). Its application scenarios include big data analytics, mobile apps, hot videos, and social apps.</p><p>· <strong>The Infrequent Access Storage class</strong> is ideal for storing data that is accessed infrequently (less than 12 times a year) yet has quick response requirements. Its application scenarios include file synchronization, file sharing, and enterprise backup.</p><p>· <strong>The Archive Storage class is </strong>ideal for archiving data that is rarely accessed (once a year on average). Its application scenarios include data archiving and long-term data backups. The Archive storage class is secure, durable, and inexpensive, and can be used to replace tape libraries. To keep costs low, it may take hours to restore data from the Archive storage class.</p><p>· <strong>The Deep Archive </strong>storage class is suitable for archiving data that is barely-accessed (once a few years). Compared with the Archive storage class, it costs less but takes longer time (usually several hours) to restore data.</p><p>You can find a comparison of storage tiers in the table below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*GyXqLGrMKq_imihvhOsBQA.png" /><figcaption>Storage Tiers in Huawei Cloud OBS</figcaption></figure><p>You must restore an object in the Archive or Deep Archive storage class before you can download it, configure its metadata, or access it with a URL.</p><p>Objects can be restored at an expedited or a standard speed. With the expedited speed, it takes 1 to 5 minutes to restore Archive objects and 3 to 5 hours to restore Deep Archive objects. With the standard speed, it takes 3 to 5 hours and 5 to 12 hours to restore Archive and Deep Archive objects, respectively. Now, Let`s configure lifecycle rules for a bucket.</p><h3>Configuring Lifecycle Rules</h3><p>You can define a lifecycle rule to automatically delete a set of objects or transition them between storage classes after a certain number of days since the objects were last updated. You can find below a diagram for this process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*EzwMJEwbb-s2F0UmhmUukg.png" /><figcaption>Lifecycle Management Diagram</figcaption></figure><p>You may configure lifecycle rules to:</p><p>· Periodically delete logs that are only meant to be retained for a specific period of time (a week or a month).</p><p>· Transition documents that are seldom accessed to the Infrequent Access or Archive storage class or delete them.</p><p>· Store some types of data in OBS for archive purposes, such as digital media, financial and medical records, raw genome sequence data, long-term database backups, and data that must be retained for regulatory compliance.</p><p>· Schedule the deletion of a large number of files from a bucket. Manually deleting objects is time-consuming, and only a limited number of objects can be deleted at a time.</p><p>You can define lifecycle rules to identify objects in the scenarios above and further manage their lifecycles.</p><p>Objects that are no longer frequently accessed can be transitioned to Infrequent Access or Archive or Deep Archive to reduce your storage costs. In short, transition basically means that the object storage class is altered without copying the object. You can also manually change the storage class of an object on the Objects page.</p><h4>Creating a Lifecycle Rule</h4><p>1. In the navigation pane of the OBS Console, choose Object Storage.</p><p>2. In the bucket list, click the bucket you want to operate to go to the Objects page.</p><p>3. In the navigation pane, choose Overview.</p><p>4. In the Basic Configurations area, click Lifecycle Rules. The Lifecycle Rules page is displayed.</p><p>5. Alternatively, you can choose Basic Configurations &gt; Lifecycle Rules in the navigation pane.</p><p>Click Create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*aNQNZsQaEurMQ7Ew9yi8PA.png" /><figcaption>Creating a Rule</figcaption></figure><p>6. Configure a lifecycle rule.</p><p><strong>Basic Information:</strong></p><p>· Status:</p><p>Select Enable to enable the lifecycle rule.</p><p>· Rule Name:</p><p>It identifies a lifecycle rule. A rule name can contain a maximum of 255 characters.</p><p>· Prefix: It is optional.</p><p><strong>Transition to Infrequent Access After (Days):</strong> After this number of days since the last update, objects meeting specified conditions will be transitioned to Infrequent Access. This number must be at least 30.</p><p><strong>Transition to Archive After (Days):</strong> After this number of days since the last update, objects meeting specified conditions will be transitioned to Archive. If you configure to transition objects first to Infrequent Access and then Archive, the objects must stay in Infrequent Access for at least 30 days before they can be transitioned to Archive. If the transition to Archive is used, but the transition to Infrequent Access is not, there is no limit on the number of days for the transition.</p><p><strong>Delete Objects After (Days):</strong> After this number of days since the last update, objects meeting certain conditions will expire and then be deleted. This number must be an integer larger than that specified for any of the transition operations.</p><h3>Conclusion</h3><p>Your data can be automatically managed with the lifecycle management features of Huawei Cloud OBS and can fulfill both regulatory compliances and cost-efficiency. It allows its users to automatically move their data between storage tiers without manual intervention under conditions that the data needs to be kept hot, warm, and cold. With this feature, data management is easy and time efficient, making it one of the most useful features Huawei Cloud offers.</p><h3>References</h3><p><a href="https://support.huaweicloud.com/intl/en-us/ugobs-obs/obs_41_0033.html">Lifecycle Management_Object Storage Service_Huawei Cloud</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d0694769e88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/%EF%B8%8Fobject-storage-lifecycle-management-on-huawei-cloud-0d0694769e88">☁️Object Storage Lifecycle Management on Huawei Cloud</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️Huawei Cloud DMS for Kafka]]></title>
            <link>https://medium.com/huawei-developers/%EF%B8%8Fhuawei-cloud-dms-for-kafka-3168b8ff9877?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/3168b8ff9877</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[messaging]]></category>
            <category><![CDATA[huawei-cloud]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Wed, 10 Jul 2024 11:47:15 GMT</pubDate>
            <atom:updated>2024-07-10T11:47:15.049Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/737/1*y5m4IPnUuxT3woL8tT-0NQ.jpeg" /><figcaption>DMS for Kafka</figcaption></figure><h3><strong>Introduction</strong></h3><p>Hello to everyone! 👋 In this article, we will talk about Huawei Cloud Distributed Message Service (DMS) for Kafka. Apache Kafka is distributed message middleware that features high throughput, data persistence, horizontal scalability, and stream data processing. It adopts the publish-subscribe pattern and is widely used for log collection, data streaming, online/offline system analytics, and real-time monitoring. Distributed Message Service (DMS) for Kafka is a fully-managed service that delivers high reliability, concurrency, and scalability in real-time data transmission, stream data processing, system decoupling, and traffic balancing use cases.</p><p>First of all, we will briefly discuss the features of the DMS for Kafka service. We will then create a DMS for Kafka cluster and complete our demo with producers and consumers for an example topic in the demo. If you are ready, let`s move.</p><h3><strong>Huawei Cloud DMS for Kafka</strong></h3><p>Distributed Message Service (DMS) for Kafka is a message queuing service based on the open-source Apache Kafka. It provides Kafka premium instances with isolated computing, storage, and bandwidth resources. The computing, storage, and bandwidth resources used by an instance are exclusively occupied by the user. You can apply for instances as required. The instances can be used right out of the box, taking off the deployment and O&amp;M pressure for you so that you can focus on developing your services.</p><p>Huawei Cloud DMS for Kafka provides easy-to-use message queuing based on Apache Kafka. Services can be quickly migrated to the cloud without any change, reducing maintenance and usage costs.</p><p>· Rapid Deployment: Simply set instance information on the DMS for Kafka console, submit your order, and a complete Kafka instance will be automatically created and deployed.</p><p>· Service migration without modifications: DMS for Kafka is compatible with open-source Kafka APIs and supports all message processing functions of open-source Kafka.</p><p>· Security: Operations on Kafka instances are recorded and can be audited. Messages can be encrypted before storage. In addition to Simple Authentication and Security Layer (SASL) authentication, Virtual Private Clouds (VPCs) and security groups also provide security controls on network access.</p><p>· Data reliability: Kafka instances support data persistence and replication. Messages can be synchronously or asynchronously replicated between replicas and flushed to disk.</p><p>· High availability: Kafka runs in clusters, enabling failover and fault tolerance so that services can run smoothly. Kafka instance brokers can be deployed across AZs to enhance service availability. Data is synchronized between different AZs based on Kafka’s in-sync replica (ISR) mechanism. A topic must have multiple data copies and distribute them across ISRs. When ISR replication is normal, the recovery point objective (RPO) is close to 0.</p><p>· Simple O&amp;M: Huawei Cloud provides a whole set of monitoring and alarm services, eliminating the need for 24/7 attendance. Kafka instance metrics are monitored and reported, including the number of partitions, topics, and accumulated messages. You can configure alarm rules and receive SMS or email notifications on how your services are running in real time.</p><p>· Massive accumulation and scaling: Kafka features high scalability because it runs in a distributed system, or cluster. Users can configure up to 200 partitions for a topic. The storage space, broker quantity and flavor can be also expanded. This means that billions of messages can be accumulated, suitable for scenarios requiring high concurrency, high performance, and large-scale access.</p><p>· Flexible specifications: You can customize the bandwidth and storage space for the instance and the number of partitions and replicas for topics in the instance.</p><p>The final note for DMS for Kafka Service is Kafka instances are compatible with Apache Kafka v1.1.0, v2.7, and v3.x., for competency purposes, users need to keep the client and server versions the same. Next, we`ll go through the step-by-step instructions for installing DMS for Kafka service, creating a sample topic, and utilizing producer and consumer for the topic.</p><h3><strong>Produce and Consume Messages with DMS for Kafka</strong></h3><p><strong>Prepare the Environment</strong></p><ol><li>First, we need to create the DMS for Kafka cluster and the VPC for Kafka client services. For the demo, we can create the VPC with the following parameters.</li></ol><p><em>Region: Singapore</em></p><p><em>Name: vpc-tl-kafkacluster (You can give a name whatever you want)</em></p><p><em>Ipv4 CIDR Block: Keep it as is. (1912.168.0.0/16)</em></p><p><em>Subnet Name: subnet-tl-kafkacluster (You can give a name whatever you want)</em></p><p><em>Subnet Ipv4 CIDR Block: Keep it as is. (192.168.0.0/24)</em></p><p><em>AZ: Keep it as is.</em></p><p>Keep the other parameters as default and create the VPC. As a result, the VPC we created for the demo was as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGBxrGexdPQ7DOPoYaUdDA.png" /><figcaption>VPC for the Demo</figcaption></figure><p>2. Let’s build our Kafka cluster now. Select DMS for Kafka from the Huawei Cloud Console, click buy instance, and enter the following parameters.</p><p><em>Billing Mode: Pay-per-use</em></p><p><em>Region: AP-Singapore</em></p><p><em>Project: Keep it as is</em></p><p><em>AZ: AZ1, AZ2, AZ3</em></p><p><em>Instance Name: kafka-cluster</em></p><p><em>Enterprise Project: Give your project name.</em></p><p><em>Specifications: Cluster</em></p><p><em>Version: 3x</em></p><p><em>Broker Flavor: kafka.2u4g.cluster</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SmOo_hKB_rEj5XLvOfw5bg.png" /><figcaption>Buying DMS for Kafka Cluster — 1</figcaption></figure><p><em>Brokers: 3</em></p><p><em>Stora space per broker: Ultra-high I/O 100 GB</em></p><p><em>Capacity Threshold Policy: Automatically delete</em></p><p><em>VPC: Use the VPC and subnet that we`ve provisioned in the first step.</em></p><p><em>Security Group: Create an SG with the below inbound rules. I`ve created it and named it sg-kafkacluster-tl.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*QdFaqxAZh-yCT8h3e9LvXw.png" /><figcaption>DMS for Kafka Security Group Inbound Rules</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_GHjpIjyA-5zA5dBi9KvHA.png" /><figcaption>Buying DMS for Kafka Cluster — 2</figcaption></figure><p><em>Plaintext Access: Disable</em></p><p><em>Ciphertext Access: Enable</em></p><p><em>Security Protocol: SASL_SSL</em></p><p><em>Private IP Adress: Auto</em></p><p><em>Public Network Access: Disable</em></p><p><em>SASL/PLAIN: Enable</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*93ILTVCkVaHrpcclqew-4w.png" /><figcaption>Buying DMS for Kafka Cluster — 3</figcaption></figure><p>Finally, just click to buy and wait for 3–5 minutes for the provisioning.</p><p><strong>Prepare the Kafka Client</strong></p><ol><li>For the demo I need a Kafka Client and I used ECS for it. Let`s provision it. Go to ECS page and click Buy ECS and use the below values for the given parameters.</li></ol><p><em>Region: Singapore</em></p><p><em>Billing Mode: Pay-per-use</em></p><p><em>AZ: Random</em></p><p><em>CPU: x86</em></p><p><em>Specifications: General computing-plus c6.large.2 (2vCPUs, 4GB RAM)</em></p><p><em>Public Image: Ubuntu 20.04</em></p><p><em>System Disk: Keep it as is (General Purpose 40 GB)</em></p><p><em>VPC: Choose the VPC I`ve created</em></p><p><em>Subnet: Choose the subnet I`ve created</em></p><p><em>Security Group: Create a new one with the below inbound rules and named it as sg-ecskafkaclient-tl.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JMkz4jtnmANm74hCeOLJ3A.png" /><figcaption>Kafka Client Security Group Inbound Rules</figcaption></figure><p><em>EIP: Auto Assign</em></p><p><em>Billed By: Traffic</em></p><p><em>Bandwidth Size: 50</em></p><p><em>ECS name: Give a name</em></p><p><em>Login Mode: Password and provide a strong password with best practices as the pop-up says.</em></p><p><em>CBR: Not required</em></p><p>Finally, buy it. It will be available in a few seconds. The next step is preparing the Kafka Client.</p><p>2. Kafka Client Installation. Connect to your ECS with a Remote Login or a remote terminal like MobaXterm. I`ve used Moba for the demo. After connecting the instance, use the below commands to update the OS of the ECS and install Java.</p><pre>apt update -y<br>apt install openjdk-8-jdk</pre><p>To view the Java installation path, use the below command</p><pre>update-alternatives --config java</pre><p>And mine is as follows;</p><pre>/usr/lib/jvm/java-8-openjdk-amd64</pre><p>Now, we need to add these paths as a variable to the .bashrc profile.</p><pre>vi ~/.bashrc<br>export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64<br>export PATH=$JAVA_HOME/bin:$PATH</pre><p>And reload it.</p><pre>source ~/.bashrc</pre><p>3. Download the Kafka client from the below link to your ECS and decompress it.</p><pre>wget https://archive.apache.org/dist/kafka/2.8.0/kafka_2.12-2.8.0.tgz<br>tar -xzvf kafka_2.12-2.8.0.tgz</pre><p>4. The next step is getting connection info from Kafka cluster. In the kafka cluster connection info section get the broker IP addresses. Then, put them into /etc/hosts file in your ECS Kafka client. To do this;</p><pre>vi /etc/hosts<br>192.168.0.226   server01<br>192.168.0.110   server02<br>192.168.0.14    server03</pre><p>5. Download the certificate from the Kafka Cluster. To do this Cluster&gt;Basic Information&gt;SSL Certificate in the Connection area.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/1*0BQtf1lN7osIf_5sOyjd1g.png" /><figcaption>Downloading SSL Certificate</figcaption></figure><p>Decompress it in your local machine.</p><p>6. Create a directory in your /root directory at the ECS Kafka client.</p><pre>mkdir -p /root/kafka_certificates</pre><p>7. Upload the certificate files into the directory that you`ve created via the MobaXterm file browser.</p><p>8. Add the following commands in both the consumer.properties and producer.properties</p><pre>sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \<br>username=&quot;kafka-master&quot; \<br>password=&quot;XXXXXXXXX&quot;;   //   (Use your kafka cluster pw)  <br>sasl.mechanism=PLAIN<br>security.protocol=SASL_SSL<br>ssl.truststore.location=/root/kafka-certificates/client.jks<br>ssl.truststore.password=dms@kafka<br>ssl.endpoint.identification.algorithm=<br></pre><p><strong>Produce and Consume Messages</strong></p><ol><li>Now, We can produce messages from the Kafka client. Before producing messages, create a topic in the Kafka cluster from the DMS for Kafka console. I`ve named it topic-01 with the below parameters.</li></ol><p><em>Topic Name: topic-01</em></p><p><em>Partitions: 3</em></p><p><em>Replicas: 3</em></p><p><em>Aging Time: 72</em></p><p><em>Synchronous Replication: Disable</em></p><p><em>Synchronous Flushing: Disable</em></p><p><em>Message Timestamp: Select CreateTime</em></p><p><em>Max. Message Size (bytes): Retain the default value.</em></p><p>Jump into your Kafka client.</p><pre>cd ../bin<br>./kafka-console-producer.sh --broker-list 192.168.0.99:9093,192.168.0.191:9093,192.168.0.62:9093 --topic topic-01 --producer.config ../config/producer.properties<br>&gt;Hello<br>&gt;DMS<br>&gt;Kafka!<br>ctrl+c</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JOr3loL0umfFjtG3I9UT6g.png" /><figcaption>Producing Kafka Messages</figcaption></figure><p>2. Let`s consume these messages.</p><pre>./kafka-console-consumer.sh --bootstrap-server 192.168.0.99:9093,192.168.0.191:9093,192.168.0.62:9093 --topic topic-01 --group test-consumer-group --from-beginning  --consumer.config ../config/consumer.properties<br>&gt;Hello<br>&gt;DMS<br>&gt;Kafka!<br>ctrl+c</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bv3VaL9g4xuKR0-nx7GGPw.png" /><figcaption>Consuming Kafka Messages</figcaption></figure><p>As you can see in the above output our cluster is up and running. It also gets and delivers the producer and the consumer messages.</p><h3><strong>Conclusion</strong></h3><p>Kafka is a popular message-oriented middleware that features highly reliable, asynchronous message delivery. It is widely used for transmitting data between different systems in many industries, including enterprise applications, payment, telecommunications, e-commerce, social networking, instant messaging, video, Internet of Things, and Internet of Vehicle. In this demo we`ve simulated the above definition and in a way, we have also provided service decoupling and we have also provided the micro-service paradigm with the Huawei Cloud Managed Kafka service.</p><h3><strong>References</strong></h3><p><a href="https://support.huaweicloud.com/intl/en-us/wtsnew-kafka/index.html">What&#39;s New_Distributed Message Service for Kafka</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3168b8ff9877" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/%EF%B8%8Fhuawei-cloud-dms-for-kafka-3168b8ff9877">☁️Huawei Cloud DMS for Kafka</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️Multi Node Hadoop Installation on Ubuntu with Huawei Cloud ECS]]></title>
            <link>https://medium.com/huawei-developers/multi-node-hadoop-installation-on-ubuntu-with-huawei-cloud-ecs-2a11d22fe6ba?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a11d22fe6ba</guid>
            <category><![CDATA[hadoop]]></category>
            <category><![CDATA[huawei-cloud]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[ubuntu]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Wed, 05 Jun 2024 05:36:37 GMT</pubDate>
            <atom:updated>2024-06-12T11:08:01.290Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><p>Hello to everyone! In this article, we explain the installation of Hadoop Distributed File System(HDFS)on virtual machines with Ubuntu OS (22.04). Although many Cloud Service Providers (CSPs), such as Huawei Cloud, have Hadoop as a managed service (Huawei Cloud MRS), in this article, we especially focus on the installation of Hadoop in the Cloud IaaS services. First of all, after briefly discussing what is Hadoop, give a brief description of Huawei Cloud and especially ECS service, finally explain the Hadoop installation for 2 node cluster step by step. Let’s start!</p><h3>Hadoop</h3><p>Apache Hadoop is an open-source integrated application framework used to develop data processing applications that are executed in a distributed server environment.</p><p>Applications built using the Hadoop ecosystem are run on large data segments distributed across computer clusters. Low-capacity computers are inexpensive and widely available. These are mainly useful for getting more computing power at low cost.</p><p>Because Hadoop can process and store such a wide assortment of data, it enables organizations to set up data lakes as expansive reservoirs for incoming streams of information. In a Hadoop data lake, raw data is often stored as is so data scientists and other analysts can access the full data sets, if need be; the data is then filtered and prepared by analytics or data management teams to support different applications.</p><p>Similar to data found in a personal computer system’s local file system, in Hadoop data resides in a distributed file system called the Hadoop Distributed File System (HDFS). The processing model is based on the concept of ‘Data Locality’, where computational logic is sent to cluster nodes (servers) containing data.</p><p>Hadoop includes the following four modules as its primary components:</p><p>i. Hadoop Distributed File System: HDFS is a java-based distributed file system designed to store very large volumes of data, which is at the very core of the Hadoop project. It has fault-tolerance capacity and highly scalable and an ideal for low-cost storing and processing big data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CoEYOKIsmb9dL-HpunHTvg.jpeg" /><figcaption>HDFS Structure</figcaption></figure><p>*NameNode: The master node does not store physical data on it but knows where it all is stored.</p><p>*DataNode: It stores the data, it is the slave, it is the performer of the work itself. It regularly sends block lists to Namenode.</p><p>*Without namenode, the file system will not work. That’s why it’s important to protect NameNode.</p><p>ii. Hadoop YARN: Short for Yet Another Resource Negotiator. YARN is Hadoop’s cluster resource management and job scheduling technology In Hadoop clusters, YARN sits between HDFS and the processing engines deployed by users</p><p>iii. Hadoop MapReduce: This is a built-in programming framework and processing engine for running batch applications that group together various transactions or processes.</p><p>iiii. Hadoop Common: A set of shared utilities and libraries, Hadoop Common supports the other modules by providing various cluster configuration, management and security features.</p><p>The ecosystem that has been built up around Hadoop includes a range of other open source technologies that can complement and extend its basic capabilities. The following are some notable Apache big data tools that are related to Hadoop:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*w5QnlFpY9m9Au8Fp3O0_Hw.png" /><figcaption>Apache Big Data Ecosystem</figcaption></figure><p>· Ambari</p><p>· Flume</p><p>· HBase</p><p>· Hive</p><p>· Mahout</p><p>· Oozie</p><p>· Spark</p><p>· Pig</p><p>· Ranger</p><p>· Tez</p><p>· Zookeeper</p><p>Since we are focusing on only the HDFS installation, you can find details from Apache website (<a href="https://hadoop.apache.org/">https://hadoop.apache.org/</a> )about these tools.</p><h3>Huawei Cloud</h3><p>Huawei Cloud is a Huawei brand for cloud services.</p><p>Huawei Cloud provides more than 240 services to its customers with more than 30 Region 80 AZ and 2800 CDN points. With the Everything-as-a-Service approach, Huawei offering cloud solutions for almost every industry including automative, healthcare, finance etc.</p><p>Elastic Cloud Server(ECS), one of Huawei’s Compute services, is one of the most important services of Huawei Cloud. With ECS, which can offer flexible and cost-effective virtual machines with templates for almost every business requirements you can quickly migrate your workloads to the cloud. You can find further information at official ECS <a href="https://support.huaweicloud.com/intl/en-us/ecs/index.html">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/1*S_vdhBugMsQeApXOCpHNmg.jpeg" /><figcaption>ECS Architecture</figcaption></figure><h3>Installing Multi Node Cluster on Huawei Cloud ECS</h3><p>1. Buying Huawei Cloud ECS Instances</p><p>We are going to buy 2 ECS instances for this demo. One of them will be served as a Master Node(NameNode) and the other as a DataNode. For this purpose, we are getting 2 machines on Huawei Cloud with the following metrics.</p><p>· Pay/Use Model</p><p>· Region: tr-west-1 (You can use whatever region you want.)</p><p>· 2 vCPU, 4 GB RAM, 40 GB Disk</p><p>· Ubuntu 22.04 OS</p><p>· VPC: 192.168.0.0/16</p><p>· Subnet: 192.168.0.0/24</p><p>· EIP: Make Auto Assign for the Master Node and choose Traffic 20Mbit/s bandwidth size, not required for the DataNode.</p><p>· Give your instances names. I`ve given ecs-nn to master and ecs-dn to worker node. You need to make these names rather easier, since we`ll use them in the Hadoop installation process. Choose `Password` for login mode and type a password and note it.</p><p>· Project: You can use default or your own project.</p><p>· Keep other parameters as is.</p><p>Both of the machines were provisioned in same VPC and subnet for low latency and performance issues. Besides, it is a best-practice to run big data clusters in local networks and make them reachable from the internet if necessary.</p><p>Then, we`ve configured Security Group(SG) and its rules as below and attached the SG to both 2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*IKoZ6tMgrg6Hdfnu4YZyDA.png" /><figcaption>ECS Security Groups</figcaption></figure><p>For the demo purposes we have to open 9870 and 50070 ports to show WEB UI of the cluster that meant to be installed for this demo. We also need to give Public IP to Master Node for fulfilling the demo via Huawei Cloud Elastic IP service. Finally, you can find below our running instances from the Huawei Cloud.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*Kn_erMvC4zdmtrwQ0YDHcQ.png" /><figcaption>Running ECS Instances</figcaption></figure><p>2. OS Packet Updates</p><p>After buying our ECS instances, we need to login them one by one and make the necessary installation. The first step is updateing the OS packets. Login via the Cloud Console from the ECS Dashboard, click the Remote Login and type your username and the password as you`ve configured during the 1st step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*yTTmzR_NWWm5kzVGXWAz8A.png" /><figcaption>Login to Your ECS Instances</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*V49JAxBqoq-buh2_TFG7kw.png" /><figcaption>SSH into Your NameNode and DataNode via Huawei Cloud ECS Remote Login</figcaption></figure><p>After logging in your instances. You need to update OS repositories with using below command;</p><pre>apt update</pre><p>3. SSH Installation&amp;Key-Based Authentication</p><p>The next step is configuring SSH keys for both machines. We need password-less SSH between the name node and the data node(s). Let us create a public-private key pair for this purpose on the namenode. Use below commands one by one in both ECS instances.</p><pre>apt install openssh-server<br>ssh-keygen -t rsa -P -f /root/.ssh/id_rsa<br>cat .ssh/id_rsa.pub &gt;&gt; ~/.ssh/authorized_keys</pre><p>Now, you`ve authorized_keys files in both MasterNode and WorkerNode. Copy each of them and paste each other (append them, not overwrite the file.). At last they are two public keys in both authorized_files of your instances.</p><pre>ls -ltr /root/.ssh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*LqLI8L6CC07GVwY-9Ey_Dw.png" /><figcaption>SSH Files</figcaption></figure><p>After passwordless login is done test the connection between the nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*TAMw6Ai-7OWVB6wDWrdo9A.png" /><figcaption>Passwordless Login</figcaption></figure><p>4. Updating /etc/hosts file</p><p>Next step is configuring hosts files in both machines. You need to add private IP and hostname parameters as below for the both machines.</p><pre>vi /etc/hosts</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*_1r1n5AAgFcNZKK3UVMn6w.png" /><figcaption>Updated hosts file</figcaption></figure><p>5. Installing JDK</p><p>The next step is installing JDK for both machines.</p><pre>apt install openjdk-8-jdk-headless </pre><p>6. Download&amp;Install Hadoop</p><p>Now, we`re almost ready to install Hadoop. We`ve used latest version of Hadoop for this demo. If you want to use earlier versions, just update the download link. We need to download and install Hadoop in both machines.</p><pre>wget https://dlcdn.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6-src.tar.gz<br>tar –xzvf hadoop-3.3.6.tar.gz <br>mv hadoop-3.3.6 hadoop</pre><p>7. Setting Environment Variables</p><p>We need to modify the .bashrc file for the environment variables for both NameNode and DataNodes.</p><pre>vi ~/.bashrc<br><br>export HADOOP_HOME=/root/hadoop<br>export PATH=$PATH:$HADOOP_HOME/bin<br>export PATH=$PATH:$HADOOP_HOME/sbin<br>export HADOOP_MAPRED_HOME=${HADOOP_HOME}<br>export HADOOP_COMMON_HOME=${HADOOP_HOME}<br>export HADOOP_HDFS_HOME=${HADOOP_HOME}<br>export YARN_HOME=${HADOOP_HOME}<br>export HDFS_NAMENODE_USER=&quot;root&quot;<br>export HDFS_DATANODE_USER=&quot;root&quot;<br>export HDFS_SECONDARYNAMENODE_USER=&quot;root&quot;<br>export YARN_RESOURCEMANAGER_USER=&quot;root&quot;<br>export YARN_NODEMANAGER_USER=&quot;root&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*Ug2l9ne53xr8FUDiws7wxg.jpeg" /><figcaption>.bashrc file</figcaption></figure><p>The .bashrc file must be like that. You can configure it for your environments files path. After saving the changes and exiting the text editor, we run the following command to refresh the settings:</p><pre>source ~/.bashrc</pre><p>8. Updating Hadoop-env.sh File</p><p>We also need to update hadoop-env.sh for the JAVA Home for both machines.</p><pre>vi ~/hadoop/etc/hadoop/hadoop-env.sh<br><br>export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/1*CKnUfemD-jhSKTWvQXVCnw.jpeg" /><figcaption>Modifying the JAVA Home</figcaption></figure><p>9. Updating core-site.xml</p><p>Run the following command to open the core-site.xml file. IP address is the private IP address of the master node in both instances files.</p><pre>vi ~/hadoop/etc/hadoop/core-site.xml<br><br>&lt;configuration&gt;<br> &lt;property&gt;<br>        &lt;name&gt;fs.defaultFS&lt;/name&gt;<br>        &lt;value&gt;hdfs://192.168.0.252:9000&lt;/value&gt;<br>    &lt;/property&gt;<br><br>    &lt;property&gt;<br>  &lt;name&gt;dfs.namenode.datanode.registration.ip-hostname-check&lt;/name&gt;<br>  &lt;value&gt;false&lt;/value&gt;<br>&lt;/property&gt;<br><br>&lt;/configuration&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/533/1*STjptOZBSMzOdBrQB2MavA.jpeg" /><figcaption>core-site.xml</figcaption></figure><p>10. Updating hdfs-site.xml</p><p>Add the following lines into hdfs-site.xml files of both machines.</p><pre>vi ~/hadoop/etc/hadoop/hdfs-site.xml<br><br>&lt;configuration&gt;<br><br>  &lt;property&gt;<br>        &lt;name&gt;dfs.replication&lt;/name&gt;<br>        &lt;value&gt;2&lt;/value&gt;<br>    &lt;/property&gt;<br>    &lt;property&gt;<br>        &lt;name&gt;dfs.namenode.name.dir&lt;/name&gt;<br>        &lt;value&gt;file:///usr/local/hadoop/hdfs/data&lt;/value&gt;<br>    &lt;/property&gt;<br>    &lt;property&gt;<br>        &lt;name&gt;dfs.datanode.data.dir&lt;/name&gt;<br>        &lt;value&gt;file:///usr/local/hadoop/hdfs/data&lt;/value&gt;<br>    &lt;/property&gt;<br><br>&lt;/configuration&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*6r1JuKm-kZcP0qqIcPAAww.jpeg" /><figcaption>hdfs-site.xml</figcaption></figure><p>We`ve configured replication as 2, since we are going to have 2 machines in the cluster, if you have more machines you scale it up as you want. And for the data directories, we`re going to make available the above file paths for this demo. You can change them use as you are going to make it.</p><p>11. Updating mapred-site.xml</p><p>We`re almost done. Update the mapred-site XML files in both machine.</p><pre>vi ~/hadoop/etc/hadoop/mapred-site.xml<br><br>&lt;configuration&gt;<br> &lt;property&gt;<br>        &lt;name&gt;mapreduce.jobtracker.address&lt;/name&gt;<br>        &lt;value&gt;192.168.0.252:54311&lt;/value&gt;<br>    &lt;/property&gt;<br>    &lt;property&gt;<br>        &lt;name&gt;mapreduce.framework.name&lt;/name&gt;<br>        &lt;value&gt;yarn&lt;/value&gt;<br>    &lt;/property&gt;<br>&lt;/configuration&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*me_B04EU1k6LvK1_5UjQ9A.png" /><figcaption>mapred-site.xml</figcaption></figure><p>12. Updating yarn-site.xml</p><p>Finally, update the yarn-site.xml as below.</p><pre>vi ~/hadoop/etc/hadoop/yarn-site.xml<br><br>&lt;configuration&gt;<br><br>  &lt;property&gt;<br>        &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;<br>        &lt;value&gt;mapreduce_shuffle&lt;/value&gt;<br>    &lt;/property&gt;<br>    &lt;property&gt;<br>        &lt;name&gt;yarn.nodemanager.aux-services.mapreduce.shuffle.class&lt;/name&gt;<br>        &lt;value&gt;org.apache.hadoop.mapred.ShuffleHandler&lt;/value&gt;<br>    &lt;/property&gt;<br>    &lt;property&gt;<br>       &lt;name&gt;yarn.resourcemanager.hostname&lt;/name&gt;<br>       &lt;value&gt;192.168.0.252&lt;/value&gt;<br>    &lt;/property&gt;<br><br>&lt;/configuration&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*D845M0kcveC9g21fW1MeGw.png" /><figcaption>yarn-site.xml</figcaption></figure><p>13. Initiating HDFS File System</p><p>Creating the files on the OS file directory for the HDFS in both machines.</p><pre>mkdir –p /usr/local/hadoop/hdfs/data<br>chown root:root –R /usr/local/hadoop/hdfs/data<br>chmod 700 /usr/local/hadoop/hdfs/data</pre><p>14. Setting Up Masters&amp;Workers File</p><p>We should list all worker hostnames or IP addresses in the etc/hadoop/workers file, one per line. Therefore, the last step is to edit the file and add the private IP addresses of the nodes to the file.</p><pre>vi ~/hadoop/etc/hadoop/workers  </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*TVhfez7g3RpS3zeTgewVFA.png" /><figcaption>workers file</figcaption></figure><pre>vi ~/hadoop/etc/hadoop/masters</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*Ybq9g33r5sg6JmlGffGhzQ.png" /><figcaption>masters file</figcaption></figure><p>15. Formatting HDFS and the Cluster</p><p>Now, we need to use below commands to format hdfs and initiate it from the master machine. Just keep in mind after the all config works done the further activities must be done via master node!</p><pre>hdfs namenode –format<br>start-dfs.sh</pre><p>To check whether hdfs is working or not, type below commands on the both machines.</p><pre>jps</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*BvO-qmWBTtK7WmtsQtlVZg.png" /><figcaption>Controlling the Cluster via jps</figcaption></figure><p>If the above outputs can be viewed, everything is fine. Now we must check the WEB UI.</p><p>16. Access&amp;View Web UI</p><p>Open your browser and use the public IP of your Namenode and 9870 port. Bingo, there it is! Your HDFS is ready to go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*otJypbZS3xF3Frp7lmfNMA.png" /><figcaption>Overview of the HDFS Cluster via WEB UI</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*GFy-M5fwvvV9FkA0svNDjA.png" /><figcaption>Viewing the DataNode</figcaption></figure><h3>Conclusion</h3><p>In this article, we explained how to configure HDFS in a multi node cluster with Ubuntu 22.04 OS using Huawei Cloud ECS instances. Although Huawei Cloud and other CSPs provide this service as a Managed service, business requirements require we may need to install such a structure directly on virtual or physical machines. In our next article, we will maximize resource utilization by configuring Yarn and MapReduce from benchmark values on our existing cluster.</p><h3>References</h3><ul><li><a href="https://sparkbyexamples.com/hadoop/apache-hadoop-installation/">Apache Hadoop Installation on Ubuntu (multi-node cluster).</a></li><li><a href="https://www.apache.org/">Welcome to The Apache Software Foundation</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a11d22fe6ba" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/multi-node-hadoop-installation-on-ubuntu-with-huawei-cloud-ecs-2a11d22fe6ba">☁️Multi Node Hadoop Installation on Ubuntu with Huawei Cloud ECS</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️Comparison of Database as a Services (DbaaS) AZURE Database Service vs Huawei Cloud RDS]]></title>
            <link>https://medium.com/huawei-developers/comparison-of-database-as-a-services-dbaas-azure-database-service-vs-huawei-cloud-rds-e9e7e653396e?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9e7e653396e</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[rds]]></category>
            <category><![CDATA[huawei]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Wed, 29 May 2024 07:27:32 GMT</pubDate>
            <atom:updated>2024-07-10T07:24:02.172Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Hello to everyone! In the previous article we`ve discussed key features of the two leading DbaaS providers AWS and Huawei Cloud RDS services. You can refer to this <a href="https://medium.com/huawei-developers/%EF%B8%8Fcomparison-of-database-as-a-services-dbaas-aws-rds-vs-huawei-cloud-rds-3b5e223b7d5d">link</a> for further information.</p><p>In this article, we will provide a detailed comparison between two leading DbaaS providers solutions Azure Database Services and Huawei Cloud RDS. After giving a brief description about DbaaS, we’ll introduce Azure Database Services and Huawei Cloud RDS solutions, which are particularly based on relational database management systems. Lastly, we will compare the overlay virtual machines(VM), the storage attributes, the supported database engines, the number of regions that RDS accessible, pricing and API support.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*yr7lcqQtHquf9TOw3_IAsg.png" /><figcaption>Huawei Cloud-Azure</figcaption></figure><h3>DbaaS</h3><p>In an on-premises computing environment, the database server is part of the IT infrastructure in an organization’s data center and is installed, managed by the organization’s own IT staff. A database administrator (DBA) is responsible for configuring, managing, monitoring and troubleshooting the databases that run on the server.</p><p>In contrast, under the DBaaS model, the provider maintains the system infrastructure and database and delivers it as a fully managed cloud service. DbaaS can be classifed under the Platform as a Service (PaaS) in Cloud Service Models. The service covers high-level administrative functions, such as database installation, configuration, maintenance and upgrades. Additional tasks, such as backups, patching and performance management, are also typically handled by the provider.</p><h3>Azure Database Service</h3><p>Azure Database Services is a managed database service provided by Microsoft Azure. Azure offers relational databases and non-relational databases for application needs. Since’ this article only focused on managed relational databases, Azure`s other managed databases is ignored. With Azure Database Services`s built-in machine intelligence helps automate administrative tasks like high availability, scaling, and tuning query performance so the applications are always up and ready. It also supports mission-critical applications with unlimited database scale and up to 99.999% availability.</p><h3>Huawei Cloud RDS</h3><p>Huawei Cloud RDS is a relational database service that is reliable, scalable, secure, economical, easy to manage, and immediatly ready to for use. With RDS, users can create single or primary/standby DB instances in ease and timely manner. Huawei Cloud RDS also includes a comprehensive performance monitoring system, multi-level security measures, and a professional database management platform, allowing its users to easily set up and scale up a relational database.</p><h3>Features Comparison</h3><h4>1. Instances</h4><p><strong><em>Azure</em></strong></p><p>The Azure Database Service is available for a total of <strong>112 </strong>different virtual machine options. These machines are selectable in a wide range from <strong>2vCPU to 128 vCPU</strong>. Azure is able to deliver workload-specific optimized machines like memory optimized, general purpose and so on. However, not all of these machines can support all of the DB engines that Azure Database Service supports.</p><p><strong><em>Huawei</em></strong></p><p>The Huawei RDS service is available for a total of <strong>159</strong> different virtual machine options. These machines are selectable in a wide range from <strong>2vCPU to 512 vCPU</strong>. Huawei is able to deliver workload-specific optimized machines like memory optimized, compute optimized and so on. However, not all of these machines can support all of the DB engines that Huawei Cloud RDS supports. Frankly, we can say, Huawei offers significantly more options in instance diversity than Azure.</p><p><strong>Winner: Huawei Cloud RDS!</strong></p><p><strong>2. Storage</strong></p><p><strong><em>Azure</em></strong></p><p>Azure Database Service uses its own block storage service. There are two storage types: Premium SSD and Premium SSD v2. The following table lists some of the features of the single storage unit of Azure Database Service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*NV8MvnOZt-nDXrTvnBLjJQ.png" /><figcaption>Basic Storage Features of Azure Database Service</figcaption></figure><p><strong><em>Huawei</em></strong></p><p>Huawei Cloud also uses its own block storage service (EVS) for data storage. RDS supports Cloud SSD (also called Ultra-high I/O) and Extreme SSD to suit different performance requirements of your workloads. The following table lists some of the features of the single storage unit of Huawei Cloud RDS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/392/1*qf83D1fTeiL4JbqQ5uFM5w.png" /><figcaption>Basic Storage Features of Huawei Cloud RDS</figcaption></figure><p><strong>Winner: Azure Database Service!</strong></p><p><strong>3. DB Engines</strong></p><p><strong><em>Azure</em></strong></p><p>Regardless of the supported versions, Azure supports the following db engines.</p><p>· MariaDB</p><p>· MySQL</p><p>· Oracle*(It`s managed by Oracle, not Azure)</p><p>· PostgreSQL</p><p>· Microsoft SQL Server</p><p><strong><em>Huawei</em></strong></p><p>Regardless of the supported versions, Huawei Cloud supports the following db engines.</p><p>· MySQL</p><p>· PostgreSQL</p><p>· Mirosoft SQL Server</p><p>· MariaDB</p><p>Both Azure and Huawei supports same count of DB Engines.</p><p><strong>Winner: Tie!</strong></p><p><strong>4. Regions</strong></p><p>There is a major difference difference between the counts of DBaaS available regions at Azure and Huawei Cloud. Azure has more available DbaaS available region than Huawei Cloud RDS available regions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/708/1*n_FeAxghJxN5xjnYktV2uQ.png" /><figcaption>Service Availability in terms of Region Count</figcaption></figure><p><strong>Winner: Azure Database Service!</strong></p><p><strong>5. Billing</strong></p><p><strong><em>Azure</em></strong></p><p>Azure Database Service instances are billed based on the following components:</p><p>· DB instance per hours</p><p>· Storage (per GİB per month)</p><p>· IOPS requests (per 1 million requests)</p><p>· Backup storage</p><p>· Data Transfer (Data transfer in and out of your DB instance from or to the internet and other Azure Regions.)</p><p><strong><em>Huawei Cloud</em></strong></p><p>Huawei Cloud RDS allows its customers to pay only for what they use. RDS DB instances created in a Dedicated Computing Cluster (DCC) support pay-per-use billing and yearly/monthly billing. Customers will be billed for Huawei Cloud RDS DB instances, database storage, public network traffic and additionally used backup storage. RDS DB instances are accessible from both private and public networks. Only the traffic from public networks is billed.</p><p><strong>In terms of billing items, Azure is billing more items than Huawei and have more expensive billing units as can be seen in the below figures. With the exact same vCPUs, RAM Capacity, Storage and Service Region (2 vCPUS, 4 GB RAM, 50 GB Storage and UAE Regions were configured for both of Azure and Huawei Cloud) Azure Database Service is almost 1.5X time more expensive then Huawei Cloud RDS.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*_HQFkt63vcgH8_DjgKx-RA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*84M08vZsLr4De6zh99klpg.png" /><figcaption>Azure Database Service Costs</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/793/1*YCkw4hygDFLT3uRllp7qTA.png" /><figcaption>Huawei Cloud RDS Costs</figcaption></figure><p><strong>Winner: Huawei Cloud RDS!</strong></p><p><strong>6. APIs</strong></p><p><strong><em>Azure</em></strong></p><p>In addition to the Azure Management Console Azure also provides a wide variety of APIs for database operations. Users can use the API to automate tasks for managing their DB instances and other objects in Azure Database Services. There are more then <strong>300</strong> APIs available for Azure Database Services.</p><p><strong><em>Huawei Cloud</em></strong></p><p>Huawei Cloud APIs perform operations on RDS DB instances, such as DB instance creation, backup and restoration, query, parameter modifications, and deletions. There are <strong>200</strong> APIs available for Huawei Cloud RDS.</p><p><strong>Winner: Azure Database Service!</strong></p><h3>Conclusion</h3><p>In this article, the leading 2 Cloud Service Provider (CSP)s DbaaS (Microsoft Azure&amp;Huawei Cloud) is reviewed. They were compared in terms of instance options, storage features, supported DB engines, number of regions accessible, pricing and API support. Azure is one step ahead in terms of storage features, supported number of accessible regions and available APIs for database ops. Huawei Cloud, on the other hand, takes the lead in terms of instance diversity and pricing. In general, both services are on a par with each other, and although Huawei Cloud RDS is relatively new, it is a DbaaS that can easily compete with Azure and AWS Managed Database Services.</p><h3>References</h3><ul><li><a href="https://support.huaweicloud.com/intl/en-us/rds/index.html">Progressive Knowledge_Relational Database Service</a></li><li><a href="https://learn.microsoft.com/en-us/azure/?product=popular">Azure documentation</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9e7e653396e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/comparison-of-database-as-a-services-dbaas-azure-database-service-vs-huawei-cloud-rds-e9e7e653396e">☁️Comparison of Database as a Services (DbaaS) AZURE Database Service vs Huawei Cloud RDS</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️Comparison of Database as a Services (DbaaS) AWS RDS vs Huawei Cloud RDS]]></title>
            <link>https://medium.com/huawei-developers/%EF%B8%8Fcomparison-of-database-as-a-services-dbaas-aws-rds-vs-huawei-cloud-rds-3b5e223b7d5d?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b5e223b7d5d</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[rds]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[huawei-cloud]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Wed, 24 Apr 2024 11:10:32 GMT</pubDate>
            <atom:updated>2024-04-24T11:10:32.946Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*U_Su1GU33Q1LVOFugNxWUQ.png" /><figcaption>Databases</figcaption></figure><h3>Introduction</h3><p>Hello to everyone! 👋 Database as a service (DBaaS) is a cloud computing managed service offering that provides access to a database without requiring the setup of physical hardware, the installation of software or the need to configure the database. Most database administration and maintenance tasks are handled by the service provider, enabling users to quickly benefit from the database service. The use of DBaaS is growing as more organizations shift from on-premises systems to cloud databases. DBaaS vendors include cloud platform providers that sell database software and other database makers that host their software on one or more of the cloud platforms. Most DBaaS environments run on public cloud infrastructure.</p><p>In this article, we will provide a comprehensive comparison between two leading DbaaS providers solutions AWS RDS and Huawei Cloud RDS. Firstly, after giving a brief description about DbaaS, then we’ll introduce their respective solutions, which are particularly focused on relational data models in brief. At last, we will compare the overlay virtual machines(VM), the storage attributes the supported database engines, the number of regions to which it is accessible, security, pricing and API support.</p><h3><strong>Database-as-a-Service (DBaaS)</strong></h3><p>In an on-premises computing environment, the database server is part of the IT infrastructure in an organization’s data center and is installed, managed by the organization’s own IT staff. A database administrator (DBA) is responsible for configuring, managing, monitoring and troubleshooting the databases that run on the server.</p><p>In contrast, under the DBaaS model, the provider maintains the system infrastructure and database and delivers it as a fully managed cloud service. DbaaS can be classifed under the Platform as a Service (PaaS) in Cloud Service Models. The service covers high-level administrative functions, such as database installation, configuration, maintenance and upgrades. Additional tasks, such as backups, patching and performance management, are also typically handled by the provider.</p><h3><strong>AWS RDS</strong></h3><p>Amazon Relational Database Service (RDS) is a managed database service provided by Amazon Web Services (AWS). It makes it easy to set up and operate a scalable relational database in the AWS cloud. Amazon RDS supports an array of database engines to store and organize data. It also takes care of many database management and administration tasks, such as data migration, backup, recovery and patching.</p><h3>Huawei Cloud RDS</h3><p>Huawei Cloud RDS is a relational database service that is reliable, scalable, secure, economical, easy to manage, and immediatly ready to for use. With RDS, users can create single or primary/standby DB instances in ease and timely manner. Huawei Cloud RDS also includes a comprehensive performance monitoring system, multi-level security measures, and a professional database management platform, allowing its users to easily set up and scale up a relational database.</p><h3>Features Comparison</h3><h4>1. Instances</h4><p><strong><em>AWS</em></strong></p><p>The AWS RDS service is available for a total of <strong>101 </strong>different virtual machine options. These machines are selectable in a wide range from <strong>2vCPU to 128 vCPU</strong>. AWS is able to deliver workload-specific optimized machines like memory optimized, compute optimized and so on. However, not all of these machines can support all of the DB engines that AWS RDS supports.</p><p><strong><em>Huawei Cloud</em></strong></p><p>The Huawei RDS service is available for a total of <strong>159</strong> different virtual machine options. These machines are selectable in a wide range from <strong>2vCPU to 512 vCPU</strong>. Huawei is able to deliver workload-specific optimized machines like memory optimized, compute optimized and so on. However, not all of these machines can support all of the DB engines that Huawei Cloud RDS supports. Frankly, we can say, Huawei offers significantly more options in instance diversity than AWS.</p><p>Winner: Huawei Cloud RDS!</p><p><strong>2. Storage</strong></p><p><strong><em>AWS</em></strong></p><p>AWS RDS uses EBS, AWS’s block storage service. Amazon RDS provides three storage types: General Purpose SSD (also known as gp2 and gp3), Provisioned IOPS SSD (also known as io1 and io2 Block Express), and magnetic (also known as standard). The following table lists some of the features of the single storage unit of AWS RDS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/489/1*vaXSZPGHYmTE5QOYpMVvag.png" /><figcaption>Key Parameters for Storage — AWS</figcaption></figure><p><strong><em>Huawei Cloud</em></strong></p><p>Huawei Cloud also uses its own block storage service (EVS) for data storage. RDS supports Cloud SSD (also called Ultra-high I/O) and Extreme SSD to suit different performance requirements of your workloads. The following table lists some of the features of the single storage unit of Huawei Cloud RDS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*GG2wp3UejIB7fYaUEuH6sQ.png" /><figcaption>Key Parameters for Storage — Huawei Cloud</figcaption></figure><p>Winner: AWS RDS!</p><p><strong>3. DB Engines</strong></p><p><strong><em>AWS</em></strong></p><p>Regardless of the supported versions, AWS RDS supports the following db engines.</p><p>· Db2</p><p>· MariaDB</p><p>· MySQL</p><p>· Oracle</p><p>· PostgreSQL</p><p>· Microsoft SQL Server</p><p><strong><em>Huawei Cloud</em></strong></p><p>Regardless of the supported versions, Huawei Cloud supports the following db engines.</p><p>· MySQL</p><p>· PostgreSQL</p><p>· Mirosoft SQL Server</p><p>· MariaDB</p><p>AWS RDS slightly supports more DB engine then the Huawei Cloud.</p><p>Winner: AWS RDS!</p><p><strong>4. Regions</strong></p><p>There are no epic difference between the counts of RDS available regions at AWS and Huawei Cloud. AWS RDS has slightly more than Huawei Cloud RDS available regions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/603/1*kPWZ6cWLYgd3xYo-crpfGQ.png" /><figcaption>Available Number of Regions for RDS Comparison</figcaption></figure><p>Winner: AWS RDS!</p><p><strong>5. Security</strong></p><p><strong><em>AWS</em></strong></p><p>The Security services included in AWS RDS are;</p><p>·Authentication (IAM database authentication, Password authentication, Kerberos authentication)</p><p>·Data Protection (SSL, Encryption …)</p><p>·Logging&amp;Monitoring (CloudWatch Alarms, CloudTrail Logs, Performance Insights, Database Logs)</p><p>·Resillience (Multi AZ Deployment, RDS Backups)</p><p>·Compliance (SOC, PCI, FedRAMP, HIPAA etc.)</p><p><strong><em>Huawei Cloud</em></strong></p><p>The security services included in Huawei Cloud RDS are;</p><p>·Identity Authentication (Password verification, IAM verification)</p><p>·Access Control (Permissions Control, VPC and subnets, Security Groups)</p><p>·Data Protection (SSL, Encryption, Cross-AZ Deployment, Cross-region backup, Deletion protection)</p><p>·Audit and Logs (Cloud Trace Service, Database Security Service)</p><p>·Resillience</p><p>·Risk Monitoring (Monitoring Metrics, Protection for Critical Operations)</p><p>·Fault Recovery</p><p>·Certificates (Compliance certificates (Bridge Letter SOC 202204–202211, ISO 22301, ISO 10012, ISO 27001 etc).</p><p>Both AWS and Huawei Cloud offers wide variety of security services for their RDS solutions. They are similar in number of services and capabilities.</p><p>Winner: Tie!</p><p><strong>6. Billing</strong></p><p><strong><em>AWS</em></strong></p><p>AWS RDS instances are billed based on the following components:</p><p>· DB instance per hours</p><p>· Storage (per GİB per month)</p><p>· IOPS requests (per 1 million requests)</p><p>· Backup storage</p><p>· Data Transfer (Data transfer in and out of your DB instance from or to the internet and other AWS Regions.)</p><p><strong><em>Huawei Cloud</em></strong></p><p>Huawei Cloud RDS allows its customers to pay only for what they use. RDS DB instances created in a Dedicated Computing Cluster (DCC) support pay-per-use billing and yearly/monthly billing.Customers will be billed for Huawei Cloud RDS DB instances, database storage, public network traffic and additionally used backup storage. RDS DB instances are accessible from both private and public networks. Only the traffic from public networks is billed.</p><p>In terms of billing items, AWS is billing more items than Huawei. Hence, AWS RDS is more expensive then Huawei Cloud RDS.</p><p>Winner: Huawei Cloud RDS!</p><p><strong>7. APIs</strong></p><p><strong><em>AWS</em></strong></p><p>In addition to the AWS Management Console and the AWS Command Line Interface (AWS CLI), Amazon RDS also provides an API. Users can use the API to automate tasks for managing their DB instances and other objects in Amazon RDS. There are <strong>162 </strong>APIs available for AWS RDS<strong>.</strong></p><p><strong><em>Huawei Cloud</em></strong></p><p>Huawei Cloud APIs perform operations on RDS DB instances, such as DB instance creation, backup and restoration, query, parameter modifications, and deletions. There are <strong>200</strong> APIs available for Huawei Cloud RDS. We can clearly say, there are more APIs for Huawei RDS then the AWS RDS.</p><p>Winner: Huawei Cloud RDS!</p><h3>Conclusion</h3><p>In this article, the leading 2 Cloud Service Provider (CSP)s DbaaS (AWS&amp;Huawei Cloud) is reviewed. They were compared in terms of instance options, storage features, supported DB engines, number of regions accessible, security, pricing and API support. AWS is one step ahead in terms of storage features, supported db engines, number of accessible regions. Huawei Cloud, on the other hand, takes the lead in terms of instance diversity, pricing, and API support. In the security parameter, there is a tie. In general, both services are on a par with each other, and although Huawei Cloud RDS is relatively new, it is a DbaaS that can easily compete with AWS RDS.</p><h3>References</h3><ul><li><a href="https://support.huaweicloud.com/intl/en-us/rds/index.html">Progressive Knowledge_Relational Database Service</a></li><li><a href="https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Welcome.html">What is Amazon Relational Database Service (Amazon RDS)?</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b5e223b7d5d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/%EF%B8%8Fcomparison-of-database-as-a-services-dbaas-aws-rds-vs-huawei-cloud-rds-3b5e223b7d5d">☁️Comparison of Database as a Services (DbaaS) AWS RDS vs Huawei Cloud RDS</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁Create a Private Debian System Disk Image from an External Debian ISO Image]]></title>
            <link>https://medium.com/huawei-developers/create-a-private-debian-system-disk-image-from-an-external-debian-iso-image-adb3c964d1e9?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/adb3c964d1e9</guid>
            <category><![CDATA[debian]]></category>
            <category><![CDATA[image-management]]></category>
            <category><![CDATA[huawei-cloud]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Tue, 26 Mar 2024 13:50:55 GMT</pubDate>
            <atom:updated>2024-03-26T13:55:27.770Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Hello everyone! In this article, we will explain how to create a disk image from the Debian OS ISO image in Huawei Cloud’s Image Management Service (IMS) service and how to create a virtual machine with the image we create. IMS allows you to manage the entire lifecycle of your images. You can create ECSs or BMSs from public, private, or shared images. You can also create a private image from a cloud server or an external image file to make it easier to migrate workloads to the cloud or on the cloud. First, we will briefly talk about IMS, then we will explain how to register the ISO image file in Huawei Cloud. We will then provision a temporary ECS virtual machine from the ISO image and install Debian OS. Next, we will make the necessary operating system configurations to prepare the Debian System Disk Image in our temporary machine. Finally, with the image we create, we will provision and activate a new ECS. It’s going to be a long article, so don’t forget to grab your coffee! 😊</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/0*TDLxAHvI-u85L0V-.jpeg" /><figcaption>Debian 10 GUI</figcaption></figure><h3><strong>Image Management Service</strong></h3><p>An image is a cloud server or disk template that contains an operating system (OS), service data, or necessary software. IMS provides public, private, Marketplace, and shared images. Public images are provided by the cloud platform, private images are created by users, and shared images are private images that other users share with you. Finally, a marketplace image is a 3rd party image published in the Marketplace. It has an OS, various applications, and custom software preinstalled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*2X52XU-xZSzcygAje0reKA.png" /><figcaption>IMS Types</figcaption></figure><p>IMS provides:</p><p>· Public images that contain common OSs,</p><p>· Creation of a private image from an ECS or external image file,</p><p>· Public image management, such as searching for images by OS type, name, or ID, and viewing the image ID, system disk size, and image features such as user data injection and disk hot swap,</p><p>· Private image management, such as modifying image attributes, sharing images, and replicating images,</p><p>· Creation of ECSs using an image.</p><p>After briefly explaining IMS, we will give the answer to the question of how to create private images for OSs without public images using the IMS service in Huwaei Cloud by testing the 10.12 version of Debian OS.</p><h3>Creating a Debian System Disk Image from an ISO File</h3><p>An ISO file is a disk image of an optical disc. A large number of data files can be compressed into a single ISO file. Likewise, to access the files stored in an ISO, the ISO file needs to be decompressed. The below diagram briefly shows the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*1YB7CS5rYo-P7jf4duGiYw.png" /><figcaption>Creating a Debian system Disk Image Process</figcaption></figure><p><strong><em>Registering an ISO File as an ISO Image</em></strong></p><p>The first step is registering the ISO file as an ISO image. Register an external ISO file on the cloud platform as a private image (ISO image). Before registering an image, upload the ISO file to the OBS bucket. You can download the Debian ISO image to your local machine from this (<a href="https://laotzu.ftp.acc.umu.se/cdimage/archive/10.12.0/amd64/iso-cd/debian-10.12.0-amd64-netinst.iso">https://laotzu.ftp.acc.umu.se/cdimage/archive/10.12.0/amd64/iso-cd/debian-10.12.0-amd64-netinst.iso</a>) We’re using 10.12 version for this demo. After downloading it to your machine, you can upload the file into your Huawei Cloud OBS bucket. For uploading and managing objects into OBS, please refer to this link (<a href="http://We’re using 10.12 version for this demo. After downloading it into your machine, you can upload iso file into your Huawei Cloud OBS bucket. For uploading and managing objects into OBS, please refer to this link (https://support.huaweicloud.com/intl/en-us/qs-obs/obs_qs_0003.html ).">https://support.huaweicloud.com/intl/en-us/qs-obs/obs_qs_0003.html</a> ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*ZmJKnpTRYRiANj6cvOz0hA.png" /><figcaption>Uploading the ISO Image</figcaption></figure><p>After uploading is completed, access the IMS console. Click “Create Image” in the upper right corner. In the “Image Type and Source” area, select Import Image for Type select “ISO Image” for “Image Type”. In the image list, select the bucket and then the image file. In the “Image Information” area set the below parameters.</p><p>Architecture: x86</p><p>Boot Mode: BIOS</p><p>OS: Debian GNU/Linux 10.12.0 64bit</p><p>System Disk: 50 GB</p><p>Name: “Just as you want”</p><p>Enterprise Project: “Select your project”</p><p>Click “Next”. Confirm the settings, and click the “Submit”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*wVW4jwKNXRhRO_A_SkRoYA.png" /><figcaption>Uploading the ISO Image 2</figcaption></figure><p>Switch back to the IMS page to monitor the image status. When the image status changes to Normal, the image is registered successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*UbgtTLwDVeG1EVmX9NnSiw.png" /><figcaption>IMS Console</figcaption></figure><p><strong><em>Creating a Linux ECS from an ISO Image</em></strong></p><p>An ISO image created from an ISO file is used only for creating a temporary ECS. It will not be available on the ECS console. You cannot use it to create ECSs or change ECS OSs. You need to install an OS on the temporary ECS and use that ECS to create a system disk image which can be used to create ECSs or change ECS OSs.</p><p>1. Access the IMS console. Click the Private Images tab. Locate the row that contains the ISO image and click Create ECS in the Operation column.</p><p>2. Configure the ECS as prompted and click OK.</p><p>3. After the ECS is created, you can log in remotely to continue with OS and drivers installation. Don’t forget to bind an EIP for package downloading. You can release the EIP after the whole process is finished.</p><p><strong><em>Installing Debian OS</em></strong></p><p>1. Login into your newly created ECS.</p><p>2. Choose Graphical Debian Installer.</p><p>3. Select language and your location, for this demo I’ve selected English and UK.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*hZ6D_bkv0Q2EodrYr_4Ttg.png" /><figcaption>Debian Installation on GUI 1</figcaption></figure><p>4. Configure your keyboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*gQXeudRjdTIY7G4APeRNLg.png" /><figcaption>Debian Installation on GUI 2</figcaption></figure><p>5. Give your ECS a hostname and a domain name(You can give any text input for the domain name, we will not use it for internal or external communication via a DNS).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*KCPFjPwWGsxfwQA9DbGCsg.png" /><figcaption>Debian Installation on GUI 3</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*jQrFamVF6NcNEwObHPudEQ.png" /><figcaption>Debian Installation on GUI 4</figcaption></figure><p>6. Set your root password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*2Gx9CcLAuMyS7DjgNXn_Iw.png" /><figcaption>Debian Installation on GUI 5</figcaption></figure><p>7. Set your default username and password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*R37Q-ae8qBwnwajJBFvtlA.png" /><figcaption>Debian Installation on GUI 6</figcaption></figure><p>8. Set Partition Disks for Debian 10 to Guided-use entire disk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*wHdE0-TF2zFt8oXWpQ3TQQ.png" /><figcaption>Debian Installation on GUI 7</figcaption></figure><p>9. Select the disk that appears on the screen.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*0RrglVzwM3La-ad8OuWH8g.png" /><figcaption>Debian Installation on GUI 8</figcaption></figure><p>10. Select all files in one partition (recommended for new users)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*cVR62EEf6b35JPQSNeHnCw.png" /><figcaption>Debian Installation on GUI 9</figcaption></figure><p>11. Select finish partitioning and write changes to disk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*9xVnA-g1jJXCNG1DMrbdiw.png" /><figcaption>Debian Installation on GUI 10</figcaption></figure><p>12. Select Yes for writing changes to disk and wait for a while for the installation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*V6hSuN_Jr66Hmt18I3KxrA.png" /><figcaption>Debian Installation on GUI 11</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*mRIaKMx7BU009NHOUMugZQ.png" /><figcaption>Debian Installation on GUI 12</figcaption></figure><p>13. Configure the package manager and select No.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*a2Mvs5LVzQljziRkeGeyRg.png" /><figcaption>Debian Installation on GUI 13</figcaption></figure><p>14. Select a proper repository URL from the list. Let’s select TR. Then select 3rd URL for the repository mirror.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*Hk6-MRsoI_CNO7b1-KmsLQ.png" /><figcaption>Debian Installation on GUI 14</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*T6iApiYwF_QhfO2MKz5uRQ.png" /><figcaption>Debian Installation on GUI 15</figcaption></figure><p>15. Configure the package manager proxy blank for none.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*YRvkQ6ndsK6Cr7JUW1YbMQ.png" /><figcaption>Debian Installation on GUI 16</figcaption></figure><p>16. Configure the popularity-contest, select no.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*LBbTYXut3sqtSdyWDAdGCw.png" /><figcaption>Debian Installation on GUI 17</figcaption></figure><p>17. Just a little remaining to do so far. Click Debian desktop environment, SSH server, and standard system utilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*e6tJbOpNj_2GlDneDljO6g.png" /><figcaption>Debian Installation on GUI 18</figcaption></figure><p>18. Install the GRUB boot loader on a hard disk, and click Yes. Then, click the device name under the Enter device manually.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*aIiOElIepkRfExoD2G0Ghw.png" /><figcaption>Debian Installation on GUI 19</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*fuDLXGy4f6kaNLdw1tVUnQ.png" /><figcaption>Debian Installation on GUI 20</figcaption></figure><p>19. The installation is complete now, you can reboot the ECS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*cflIg9vOO0K_HvJtbzmKPw.png" /><figcaption>Debian Installation on GUI 21</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*NmcA4uDTiNrxK2Lz-NpEew.png" /><figcaption>Debian Installation Complete</figcaption></figure><p><strong><em>Configuring the ECS and Creating a Debian System Disk Image</em></strong></p><p>After installing an OS for the temporary ECS, configure the ECS and install the native Xen and KVM drivers to ensure that ECSs created subsequently are available.</p><p>1. Login into your temporary ECS with your password.</p><p>2. Open up your terminal and type the “ip addr” command, note the private IP and the NIC interface (e.g. 192.168.1.15 and ens3)</p><p>3. Add your default user to the sudoers file. Type the below commands to do this.</p><pre>su –<br>sudo visudo<br>username ALL=(ALL:ALL) ALL (You must type your username into username, in this demo this would be taylan)<br>Save the file and type below command to switch the default user.<br>exit</pre><p>Save the file and type the below command to switch the default user<em>.</em></p><p>4. Continue with the default user. Type these commands for DHCP configuration.</p><pre>sudo vi /etc/network/interface</pre><p>If DHCP has been configured for all NICs, enter :q to exit the vi editor. If the ECS has multiple NICs, you must configure DHCP for all the NICs. In our demo, the final output is shown below. You must configure it with your parameters.</p><pre>auto lo<br>iface lo inet loopback<br><br>auto ens3<br>iface ens3 inet dhcp</pre><p>5. Check the ssh service</p><pre>sudo systemctl status sshd.service </pre><p>If it is not active, type the below commands respectively.</p><pre>sudo apt-get update<br>sudo apt-get install openssh-server<br>sudo systemctl start ssh<br>sudo systemctl enable ssh</pre><p>6. Uninstall the PV driver. When optimizing a Linux private image with Xen virtualization, you need to install native Xen and KVM drivers on the source ECS of the image. To ensure that you can successfully install the native Xen and KVM drivers, you must uninstall PV drivers from the ECS first.</p><pre>ps -aux | grep uvp-monitor</pre><p>If there is no output provided. This step is ok. Otherwise;</p><pre>/etc/.uvp-monitor/uninstall</pre><p>The PV driver is uninstalled successfully. Reboot the system for the uninstallation to take effect.</p><p>7. Installing Xen and KVM Drivers</p><pre>sudo vi /etc/initramfs-tools/modules<br># Examples: <br># <br># raid1 <br># sd_mOd <br>xen-blkfront<br>xen-netfront<br>virtio_blk<br>virtio_scsi<br>virtio_net<br>virtio_pci<br>virtio_ring<br>virtio<br><br>:wq!<br>update-initramfs -u</pre><p>8. Run the following commands to check whether native Xen and KVM drivers have been installed:</p><pre>lsinitramfs /boot/initrd.img-`uname -r` |grep xen<br>lsinitramfs /boot/initrd.img-`uname -r` |grep virtio</pre><p>If the below outputs appear, Xen and KVM drivers have been installed.</p><pre>[root@ CTU10000xxxxx home]# lsinitramfs /boot/initrd.img-`uname -r` |grep xen <br>lib/modules/3.5.0-23-generic/kernel/drivers/net/ethernet/qlogic/netxen <br>lib/modules/3.5.0-23-generic/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko <br>lib/modules/3.5.0-23-generic/kernel/drivers/net/xen-netback <br>lib/modules/3.5.0-23-generic/kernel/drivers/net/xen-netback/xen-netback.ko <br>lib/modules/3.5.0-23-generic/kernel/drivers/block/xen-blkback <br>lib/modules/3.5.0-23-generic/kernel/drivers/block/xen-blkback/xen-blkback.ko <br> <br>[root@ CTU10000xxxxx home]# lsinitramfs /boot/initrd.img-`uname -r` |grep virtio <br>lib/modules/3.5.0-23-generic/kernel/drivers/scsi/virtio_scsi.ko</pre><p>9. Clear the system logs. Do it as a root user first and do it with your default user.</p><pre>echo &gt; /root/.ssh/authorized_keys<br>echo &gt; /home/username/.ssh/authorized_keys<br>sudo rm -rf /var/log/*</pre><p>10. Clear the history logs.</p><pre>echo &gt; /root/.bash_history<br>history -c</pre><p>11. When optimizing a Linux private image, you need to change the disk identifier to UUID in the GRUB configuration file of the ECS. As a root user;</p><pre>blkid</pre><p>Note the uuid.</p><pre>cat /boot/grub/grub.cfg | grep set-root</pre><p>Be sure that its values match with prior steps values for uudi. If there is mismatching or something wrong set grub.cfg files set-root values with the noted uuid. You can also refer to this link <a href="https://support.huaweicloud.com/intl/en-us/usermanual-ims/ims_01_0324.html">https://support.huaweicloud.com/intl/en-us/usermanual-ims/ims_01_0324.html</a> steps 5–9.</p><p>12. As a root user again type the below command:</p><pre>cat /etc/fstab</pre><p>If the disk is represented by UUID, no further operation is required.</p><p>If not refer to <a href="http://If the disk is represented by UUID, no further operation is required. If not refer to https://support.huaweicloud.com/intl/en-us/usermanual-ims/ims_01_0325.html">https://support.huaweicloud.com/intl/en-us/usermanual-ims/ims_01_0325.html</a></p><p>13. Check whether the ECS has data disks. Run the following command to check the number of disks attached to the ECS:</p><pre>fdisk -l</pre><p>If the number is greater than 1, the ECS has data disks. Run the following command to check the data disks attached to the ECS:</p><pre>mount</pre><p>If the command output does not contain any EVS disk information, no EVS data disks need to be detached.</p><p>If information similar to the following is displayed, refer to the next step.</p><p><em>/dev/vda1 on / type ext4 (rw,relatime,data=ordered)</em></p><p><em>/dev/vdb1 on /mnt/test type ext4 (rw,relatime,data=ordered)</em></p><p>If the number is equal to 1, no data disk is attached to the ECS. Delete the configuration information in the fstab file.</p><pre>vi /etc/fstab</pre><p>Run the following command to detach data disks from the ECS.</p><pre>umount /dev/vdb1</pre><p>Finally, run the following command to check the data disks attached to the ECS:</p><pre>mount</pre><p>If the command output contains no information about the data disks, they have been detached from the ECS.</p><p>14. We’re about to finalize the process now. Go back to the Huawei Cloud console. Go to the ECS dashboard. Click the ECS that you have just configured. More-&gt;Manage Image/Disk/Backup-&gt;Create Image</p><p>15. Click Create Image, and Set the Region you’re working on, in this case, it is EU-Dublin. Choose System Disk Image image type. Select your ECS. Give IMS name. Select your enterprise project.</p><p>16. It’s OK now. You can choose this private image for your ECS machines.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*LvD9SqSqjOlc_k8GngDkjA.png" /><figcaption>New ECS /w Debian System Disk Image</figcaption></figure><h3><strong>Conclusion</strong></h3><p>In this article, we have detailed how to create a Debian OS system disk image, a customized Linux distribution, from external ISO images. Thus, we have shown step by step how to prepare the image file for OS images that are not publicly available in various Huawei Cloud regions.</p><h3>References</h3><p><a href="https://support.huaweicloud.com/intl/en-us/ims/index.html">Progressive Knowledge_Image Management Service_Huawei Cloud</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=adb3c964d1e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/create-a-private-debian-system-disk-image-from-an-external-debian-iso-image-adb3c964d1e9">☁Create a Private Debian System Disk Image from an External Debian ISO Image</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️Huawei Cloud Disaster Recovery Services]]></title>
            <link>https://medium.com/huawei-developers/%EF%B8%8Fhuawei-cloud-disaster-recovery-services-3b17ac9e7e3c?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b17ac9e7e3c</guid>
            <category><![CDATA[huawei]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[business-continuity]]></category>
            <category><![CDATA[disaster-recovery]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Tue, 20 Feb 2024 10:55:12 GMT</pubDate>
            <atom:updated>2024-02-20T12:46:51.145Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><p>Hello to everyone! 👋 In this article, we will talk about Huawei Cloud’s disaster recovery (DR) services. With DR services, Huawei Cloud helps organizations of all sizes, from small businesses to large enterprises to fullfill their business continuity requirements. Huawei Cloud can provide reliable, highly available and secure IT infrastructures for the customers workloads with its services, particularly Storage Disaster Recovery Service and the other supporting services. First of all, we will talk about what is Disaster Recovery and how it should be, and then we will talk about the services offered by Huawei Cloud in DR. Let’s begin if you are ready.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/0*UxvakJM3GuAUwXNx.jpeg" /><figcaption>Huawei Cloud Disaster Recovery Services</figcaption></figure><h3>Disaster Recovery</h3><p>Disaster recovery (DR) is an organization’s ability to respond to and recover from an event that negatively affects business operations. The goal of DR is to reduce downtime, data loss and operational disruptions while maintaining business continuity. To prepare for this, organizations often perform an in-depth analysis of their systems and IT infrastructure and create a formal document to follow in times of crisis. This document is known as a disaster recovery plan. Now let’s look at the some key definitions related with DR.</p><p><strong>What is a Disaster?</strong></p><p>The practice of DR revolves around events that are serious in nature. These events are often thought of in terms of natural disasters, but they can also be caused by systems or technical failure, human errors, or intentional attacks. These events are significant enough to disrupt or completely stop critical business operations for a period of time.</p><p><strong>Elements of a DR Strategy</strong></p><p>Organizations should consider several factors while developing a disaster recovery strategy. Common elements of a DR strategy include the following:</p><p>1. Risk Analysis: Risk analysis or risk assessment is an evaluation of all the potential risks the business could face, as well as their outcomes.</p><p>2. Business Impact Analysis: A business impact analysis (BIA) evaluates the effects of the identified risks on business operations. Understanding the difference between risk analysis and BIA and conducting the assessments can also help an organization define its goals when it comes to data protection and the need for backup. Organizations generally quantify these using measurements called recovery point objective (RPO) and recovery time objective (RTO). RPO is the maximum age of files that an organization must recover from backup storage for normal operations to resume after a disaster. RTO refers to the amount of time an organization estimates its systems can be down without causing significant or irreparable damage to the business. RPO and RTO are both important elements in disaster recovery, but the metrics have different uses. RPO is acted on before a disruptive event takes place to ensure data is backed up, while RTO comes into play after an event occurs.</p><p><strong>DR and Backup</strong></p><p>Differences between disaster recovery (DR) and backup are as follows:</p><p>· DR is used to prevent impacts on the systems caused by natural disasters, such as fires and earthquakes. A production site and its DR site must be located with a certain secure distance. Backup is to prevent impacts on the systems caused by inappropriate manual operations, virus infection, and logic errors. It is used to restore the service system data. Usually, a system and its backup are deployed in the same data center.</p><p>· A DR system protects data but more focuses on protecting service continuity. A data backup system only ensures that data generated at different time points can be restored. Generally, the system performs the full backup for the first time, which takes a long period of time. The subsequent backup is incremental and can be completed within a short period of time.</p><p>· The highest DR standard is to implement zero RPO. You can set a maximum of 24 automatic backup policies at different time points in one day to restore data to different backup points.</p><p>· If a disaster occurs, such as earthquakes or fires, a DR system takes only several minutes to perform a failover, but a backup system takes several hours or even dozens of hours to restore the data.</p><p><strong>DR Sites</strong></p><p>An organization uses a DR site to recover and restore its data, technology infrastructure and operations when its primary data center is unavailable. DR sites can be internal, external or cloud-based. Since the main scope of this article about cloud DR services, we do not go further about external and internal DR sites, but with cloud based DR capabilities. The widespread acceptance of cloud services lets organizations, typically reliant on alternate or on-premises DR locations, host their disaster recovery in the cloud. Cloud DR goes beyond simple backup to the cloud. It requires an IT team to set up automatic failover of workloads to a public cloud platform in the event of a disruption.</p><p>In short, considering the above mentioned DR and its requirements, we can look at the DR services offered by Huawei Cloud to ensure business continuity.</p><h3>Huawei Cloud Disaster Recovery Services</h3><p><strong>1.</strong> <strong>Cloud Backup and Recovery</strong></p><p>Cloud Backup and Recovery (CBR) enables you to easily back up Elastic Cloud Servers (ECSs), Bare Metal Servers (BMSs), Elastic Volume Service (EVS) disks, SFS Turbo file systems, local files and directories, and on-premises VMware virtual environments. In case of a virus attack, accidental deletion, or software or hardware fault, you can use the backup to restore data to any point when the data was backed up.</p><p>CBR offers crash-consistent backup for multiple disks on a server and application-consistent backup for database servers. The backups protect against human errors, virus attacks, and natural disasters, and ensure your data security and reliability.</p><p>Incremental forever backups shorten the time required for backup by 95%. With Instant Restore, CBR offers an RPO of as low as 1 hour and an RTO of only several minutes.</p><p>You can also replicate backups across regions to implement remote disaster recovery.</p><p>You can use CBR to quickly restore data to the latest backup point if any of the following incident occur:</p><p>· Hacker or virus attacks</p><p>· Accidental deletion</p><p>· Application update errors</p><p>· System breakdown</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*geK4-JcimDuAG1va.jpeg" /><figcaption>Data Backup and Restoration /w CBR</figcaption></figure><p>CBR uses a multi-level reliability architecture and provides technical solutions, including cross-region replication, cross-AZ DR of backup data in the same region, and intra-AZ device and data redundancy to guarantee data durability and reliability.</p><p>Last but not least, there are other features in CBR to provide a better reliability and instant counter measures for DR such as instant restore and enhanced backups.</p><p>Instant Restore restores data and creates images from backups, much faster than a normal restore. Instant Restore is an enhanced function of CBR and requires no additional configuration. After Instant Restore is provided, you take less time to restore server data or create images.</p><p>Enhanced backups are backups generated after Instant Restore is provided. Enhanced backups make it faster to restore server data or create images. Before providing Instant Restore, CBR generates common backups. After providing Instant Restore, CBR first performs a full backup for each associated resource and then generates enhanced backups. CBR only generates enhanced backups currently. For the same resource, an enhanced backup and a common backup have the same backup content and size. They only differ in the restoration speed.</p><p><strong>2.</strong> <strong>Storage Disaster Recovery Service</strong></p><p>Storage Disaster Recovery Service (SDRS) provides cloud disaster recovery (DR) for your data centers. If your on-premises or cloud data center fails, you can fail over services to the DR center on Huawei Cloud, and then fail back the services after the production center recovers. This helps you improve service continuity and ensure the security and reliability of critical data.</p><p><strong>2.1.</strong> <strong>Asynchronous Replication</strong></p><p>Asynchronous replication continuously replicates servers at local IDCs to Huawei Cloud with RPO in a few seconds. By leveraging asynchronous replication techniques at the host layer, it allows for cross-AZ disaster recovery and keeps crash consistency for your data. If production site services fail to recover within a short period of time due to force majeure (fire and earthquake) or device faults (software and hardware damage), you can quickly recover services at the disaster recovery site with some simple configurations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*-EjiLAbe6l4fJck4.png" /><figcaption>Asynchronous replication</figcaption></figure><p><strong>2.2.</strong> <strong>Synchronous Replication</strong></p><p>Synchronous replication replicates servers from one AZ to another in real time with zero RPO. By leveraging synchronous replication techniques at the storage layer, it allows for cross-AZ disaster recovery and keeps crash consistency for your data. If production site services fail to recover within a short period of time due to force majeure (fire and earthquake) or device faults (software and hardware damage), you can quickly recover services at the disaster recovery site with some simple configurations.</p><p>SDRS is suitable for stateful applications, such as Microsoft Office 365, that requires the storage of user data on the particular server running this application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*0b2mFoRG9MROvTBg.png" /><figcaption>Synchronous replication</figcaption></figure><p>SDRS has the following advantages:</p><p>· Site server replication: You can set up disaster recovery for site servers from the production site to the disaster recovery site.</p><p>· Zero impact on applications: You can replicate all applications on the servers. The replication has no impact on the applications.</p><p>· Crash consistency: Storage-based, real-time data synchronization keeps crash consistency for your data across two AZs. Specifically, application data might not be consistent during a failover, but disk data is always consistent.</p><p>· Disaster recovery drill: By running disaster recovery drills, you can simulate recovery scenarios and formulate recovery plans. When a fault occurs, you can use the plans to recover services as quickly as possible.</p><p>· Synchronous replication: Replicate the servers to the disaster recovery site in real time to ensure zero recovery point objectives (RPO).</p><p>· Asynchronous replication: Continuously replicate the servers in the user’s data center and ensures RPO within seconds.</p><p><strong>3.</strong> <strong>Cloud Server Backup Service</strong></p><p>Cloud Server Backup Service(CSBS) enables you to back up ECSs. It works based on the consistency snapshot technology for disks. With CSBS, backup data can be used for restoring ECS data and provide business continunity and DR capabilities.</p><p>CSBS offers backup protection for ECSs. It supports crash-consistent backup, which enhances data security.</p><p><strong>4.</strong> <strong>Volume Backup Service</strong></p><p>Volume Backup Service (VBS) provides snapshot-based data protection for Elastic Volume Service (EVS) disks.</p><p>VBS secures your data, even if an EVS disk is faulty or encounters a logical error (for example, mis-deletion, hacker attacks, and virus infection). It allows you to effortlessly back up your data, and these data backups can be used to restore data quickly.</p><p>VBS supports both full and incremental backup modes. By default, the system performs a full backup initially, and then performs incremental backups. You can use a data backup generated in either backup mode to restore the source EVS disk to the state the EVS disk was in when the backup was created.</p><p>Both CSBS and VBS are backup services and provide backup protection for tenant data. The backup generated by CSBS is also displayed on the VBS page. Although they have similar purposes, there are some service differences between CSBS and VBS as shown in the below table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*MrTaI4tNsUthfTS-.png" /><figcaption>Main differences between CSBS and VBS</figcaption></figure><p><strong>5.</strong> <strong>Other Services for DR</strong></p><p><strong>5.1.</strong> <strong>Regions&amp;Availability Zones</strong></p><p>In Huawei Cloud there are availability zones (AZ) and regions. A region and availability zone (AZ) identify the location of a data center. You can create resources in a specific region and AZ. Regions are divided based on geographical location and network latency. An AZ contains one or more physical data centers. Each AZ has independent cooling, fire extinguishing, moisture-proof, and electricity facilities. Within an AZ, computing, network, storage, and other resources are logically divided into multiple clusters. AZs within a region are interconnected using high-speed optical fibers, to support cross-AZ high-availability systems. The below figure shows their relationship;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*7VO1ri6JVbcr_oZ7.png" /><figcaption>AZ-Region Relationship</figcaption></figure><p>Huawei Cloud provides services in many regions around the world. You can select a region and an AZ based on requirements. When deploying resources, consider your applications’ requirements on disaster recovery (DR) and network latency. For high DR capability, deploy reources in different AZs within the same region.</p><p><strong>5.2.</strong> <strong>Best Practice for DR using CBR&amp;SDRS</strong></p><p>Two-site three-center is a disaster recovery (DR) solution used for service continuity. With three data center coexisted, the continuity of mission-critical services can be guaranteed even if any two centers fail. This significantly improves the availability of the solution.</p><p>The two-site three-center DR solution includes a production center, an intra-city DR center, and a remote DR center.</p><p>· Production center: It provides services for external applications.</p><p>· Intra-city DR center: It is usually dozens of kilometers away from the production center. Applications can be switched to the intra-city DR center without data loss. This is the level-1 DR protection of this solution.</p><p>· Remote DR center: It is usually built hundreds or thousands of kilometers away from the production center to protect against regional disasters. Data is periodically, asynchronously replicated to the remote DR center. This is the level-2 DR protection of this solution.</p><p>You can use Huawei Cloud SDRS and CBR to deploy two-site three-center DR. This solution responds quickly to small-scale failures and large-scale natural disasters, protecting service data and guaranteeing service continuity.</p><p>In this solution, SDRS synchronizes data from the production site to a DR site in a different AZ of the same region for cross-AZ DR. CBR periodically backs up the production data to the other DR site in a different region for cross-region DR. The production site, cross-AZ DR site, and cross-region DR site all together form the geo-redundant architecture to meet the two-site three-center DR requirements.</p><p>The cross-AZ DR site corresponds to the intra-city DR center, and the cross-region DR site corresponds to the remote DR center.</p><p>In this document, the two-site three-center DR solution (SDRS+CBR) is called the SDRS+CBR DR solution for short.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*dFaZqgkh96ujALR1.png" /><figcaption>SDRS+CBR 2 Site 3 Center DR Solution Overview</figcaption></figure><p>In the SDRS+CBR DR solution, the solution is deployed in two regions and involves three sites: a production site, a cross-AZ DR site in the same region (cross-AZ DR center for short in the following document), and a cross-region DR site.</p><p>In this solution, SDRS synchronizes data from the production site to the cross-AZ DR site. CBR periodically backs up the production data to the cross-region DR site.</p><p>Compared with two-site solutions (the cross-AZ DR solution and cross-region DR solution), the SDRS+CBR DR solution combines the advantages of both two-site solutions and can protect against both AZ-level failures and regional disasters. If a regional disaster occurs, this solution can restore the service data to the latest backup time point to ensure minimal data loss.</p><h3>Conclusion</h3><p>The main objective of a DR and DR plans is to minimize negative effects of an incident on business operations to provide business continuity. A disaster recovery plan can range in scope from basic to comprehensive. Evolving disaster recovery plans and strategy based on the business continuity needs can be easily implemented with Huawei Cloud’s services mentioned in this article. Huawei Cloud services such backup/restore to synchronous replication capabilities help organizations almost any size to robust their workloads disaster recovery strategy and plans, both on-site and in the cloud.</p><h3>References</h3><ul><li><a href="https://www.techtarget.com/searchdisasterrecovery/definition/disaster-recovery">What is disaster recovery (DR)? | Definition from TechTarget</a></li><li><a href="https://support.huaweicloud.com/intl/en-us/index.html">Help center index_HUAWEI CLOUD</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b17ac9e7e3c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/%EF%B8%8Fhuawei-cloud-disaster-recovery-services-3b17ac9e7e3c">☁️Huawei Cloud Disaster Recovery Services🚑</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[☁️ Provisioning Tiered Application Infrastructure with Terraform on Huawei Cloud using Modules]]></title>
            <link>https://medium.com/huawei-developers/%EF%B8%8F-provisioning-tiered-application-infrastructure-with-terraform-on-huawei-cloud-using-modules-c30ebecfc365?source=rss-e5eac3ccf55------2</link>
            <guid isPermaLink="false">https://medium.com/p/c30ebecfc365</guid>
            <category><![CDATA[iac]]></category>
            <category><![CDATA[modules]]></category>
            <category><![CDATA[segmentation]]></category>
            <category><![CDATA[huawei]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Taylan Lakerta]]></dc:creator>
            <pubDate>Tue, 30 Jan 2024 05:11:07 GMT</pubDate>
            <atom:updated>2024-01-30T05:11:07.010Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><p>Hello to everyone! 👋 In this article, we will create a secure, layered system infrastructure for applications on Huawei Cloud by using Terraform’s module structure. Terraform uses APIs to automatically create and manage infrastructure resources. All major cloud providers provide accessible APIs that work with Terraform and enable the tool to create declarative configuration files. These providers are listed in the Terraform Registry. The Registry also lists the modules, policy libraries and tasks that teams can use to quickly deploy common infrastructure configurations and automatically manage them via code. IT teams can use modules as self-contained configuration package (prebuilt or created by the user) to streamline the use of Terraform. Let’s get ready to rumble!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/1*CE86oXjHwzGfdPLbeZuisg.png" /><figcaption>Provisioning Tiered Application Infrastructure with Terraform on Huawei Cloud using Modules</figcaption></figure><h3><strong>What is Terraform?</strong></h3><p>HashiCorp Terraform is an infrastructure as code (<a href="https://www.techtarget.com/searchitoperations/definition/Infrastructure-as-Code-IAC">IaC</a>) software tool that allows <a href="https://www.techtarget.com/searchitoperations/definition/DevOps">DevOps</a> teams to automate infrastructure provisioning using reusable, shareable, human-readable <a href="https://www.techtarget.com/searchitoperations/definition/configuration-file">configuration files</a>. The tool can automate infrastructure <a href="https://www.techtarget.com/whatis/definition/provisioning">provisioning</a> in both on-premises and <a href="https://www.techtarget.com/searchcloudcomputing/definition/cloud-computing">cloud</a> environments.</p><p>Using Terraform, teams can describe and provision all the components of the infrastructure in the form of <a href="https://www.techtarget.com/whatis/definition/code">code</a>. The code resides in configuration (config) files that can be easily shared, reused and <a href="https://www.techtarget.com/searchsoftwarequality/definition/versioning">versioned</a>. The files help create a consistent <a href="https://www.techtarget.com/searchcio/definition/workflow">workflow</a> to manage the entire cloud or <a href="https://www.techtarget.com/searchdatacenter/definition/data-center">data center</a> infrastructure and its resources throughout their lifecycle.</p><p>Terraform’s config files are declarative, that is, they describe the infrastructure’s end state. The tool itself handles the underlying logic eliminating the need to write step-by-step instructions — a cumbersome and time-consuming process — to create the required infrastructure resources.</p><h3><strong>Modules, Why do we use modules?</strong></h3><p>In <a href="https://www.techtarget.com/searchitoperations/definition/Terraform">Terraform</a>, modules can group resources into logical units, letting you break down a complex configuration into smaller, reusable components. Doing so will make a Terraform configuration easier to maintain and can reduce redundancy between <a href="https://www.techtarget.com/searchitoperations/tutorial/Manage-multiple-environments-with-Terraform-workspaces">multiple Terraform projects</a>. By writing and leveraging modules, you can simplify Terraform configurations, improve readability and increase code reusability.</p><p>When you’re deploying a resource or group of resources multiple times and possibly using a similar set of configurations, it might be time to consider combining them into a module. For example, you need to deploy a set of virtual machines, each with an image, a type and maybe a static elastic IP address. Combining these resources creates a logical unit called a virtual machine for an ECS for Huawei Cloud.</p><p>With that single unit, a module, you can create parameters to specify things such as VM size, type or flavor. Then when you want to create a virtual machine and all the supporting resources, you can call the module once and it creates everything for you.</p><h3><strong>Provisioning Resources with Terraform on Huawei Cloud</strong></h3><p>Now, Let’s provision resources through Visual Studio Code, in which Terraform add-ons installed and configured previously.</p><p>1.When dealing with cloud providers such as Huawei Cloud, we often need to provide authentication credentials, such as access keys and secret keys to access your cloud account credentials. I’ve used my own account for this purpose. I’ve added also the region to environment variables. I’ve used Huawei Cloud’s Istanbul region for this demo. Just put your envrionment variables in your VS Code project’s terminal. After you can initiate and start to react with your cloud account with “terraform init” command on the terminal.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*2U0azmjJYvfl3jKqvl2o7g.png" /><figcaption>Environment Variables</figcaption></figure><p>By using environment variables, you can maintain a clean seperation between your infrastructure code and sensitive credetials, improving security, maintainability, and flexibility in managing your Terraform configurations.</p><p>2. The main.tf file for our demo can be built as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*tKmZmfcf_mcdcW1yZNPrGQ.png" /></figure><p>As you can see from the above screenshot, I’ve frequently used modules and variables for the demo with the purpose of building isolated, reliable and secure Huawei Cloud resources.</p><p>I’ve also used outputs.tf file for printing out some content about the demo on the VS Code terminal. In this demo, I’ve added private ip, public ip and security group id for the compute instance in the outputs file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*E-4FfoG1eMpDHgAWi5iEbQ.png" /></figure><p>3. I’ve build 3 modules for this demo. First one is vpc. Before proceeding, you must define provider file in each module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*4kafSIHS3O2qF7BJH7CAWg.png" /></figure><p>4. The second module is subnet. You can find below subnet module’s main.tf file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*pjlIptaDBMzgHv3Ksli2Ew.png" /></figure><p>5. Last but not least, instance module can be viewed below. For this demo, I’ve only provisioned one compute instance in the public subnet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*AbERQyI6E8GhWy5cTXOAPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*yCRFEomJhqsa5JHYu3Z4LA.png" /></figure><p>6. After these steps, save all the files at the VSCode and just write down “terraform apply — auto-approve” command into terminal and drink one more coffee when waiting for the resources are being provisioned.</p><p>If you can see these green outputs, this means your code is running well and everything was provisioned on the Huawei Cloud. You can also see the some outputs that I’ve mentioned on this article earlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*h4LmoLavcbj9i5b00CcQPA.png" /><figcaption>After Terraform Apply Command and The Desired Outputs on the Terminal</figcaption></figure><p>7. You can now also see your resources in the Huawei Cloud console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*hcAoeNgCC_57_OGPM0dcXg.png" /><figcaption>Provisioned VPC</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*j0mZ8fvHeULkhyGFyg0Wqg.png" /><figcaption>Provisioned ECS</figcaption></figure><h3><strong>Conclusion</strong></h3><p>Terraform modules allows you to create logical abstraction on the top of some resource set. It lets you break down a complex configuration into smaller, reusable components. Doing so will make a Terraform configuration easier to maintain and can reduce redundancy between <a href="https://www.techtarget.com/searchitoperations/tutorial/Manage-multiple-environments-with-Terraform-workspaces">multiple Terraform projects</a>.</p><p>In this article, by using Terraform modules we can easily provision and isolate resources in Huawei Cloud. If you want to get to know Huawei Cloud services better, please check <a href="https://support.huaweicloud.com/intl/en-us/index.html">https://support.huaweicloud.com/intl/en-us/index.html</a> and you can also provision resources with Terraform on Huawei Cloud with some free services like VPC, some ECS capabilities as well.</p><h3><strong>References</strong></h3><ul><li><a href="https://registry.terraform.io/providers/huaweicloud/huaweicloud/latest/docs">Terraform Registry</a></li><li><a href="https://www.techtarget.com/searchitoperations/definition/Terraform#:~:text=Terraform%20Open%20Source%20is%20a,safely%20manage%20and%20track%20changes">What is Terraform? | Definition from TechTarget</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c30ebecfc365" width="1" height="1" alt=""><hr><p><a href="https://medium.com/huawei-developers/%EF%B8%8F-provisioning-tiered-application-infrastructure-with-terraform-on-huawei-cloud-using-modules-c30ebecfc365">☁️ Provisioning Tiered Application Infrastructure with Terraform on Huawei Cloud using Modules</a> was originally published in <a href="https://medium.com/huawei-developers">Huawei Developers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>