<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aaron Bachman on Medium]]></title>
        <description><![CDATA[Stories by Aaron Bachman on Medium]]></description>
        <link>https://medium.com/@abach06?source=rss-59e8fc4e60f2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5p-ozdvckvuMUgs-HOmAIg.gif</url>
            <title>Stories by Aaron Bachman on Medium</title>
            <link>https://medium.com/@abach06?source=rss-59e8fc4e60f2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:30:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abach06/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Creating a Highly Available 3-Tier Architecture for Web Applications in AWS]]></title>
            <link>https://medium.com/@abach06/creating-a-highly-available-3-tier-architecture-for-web-applications-in-aws-23f37d49bd25?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/23f37d49bd25</guid>
            <category><![CDATA[auto-scaling-groups]]></category>
            <category><![CDATA[aws-ec2]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Sun, 23 Jul 2023 12:08:40 GMT</pubDate>
            <atom:updated>2023-07-23T12:08:40.498Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How to create and test a web tier, application tier, and database tier for a highly available and scalable architecture.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TCwcsibobVZsXEZEH6C-6w.png" /></figure><p>AWS provides a wide range of resources for developing and managing cloud applications, which can be customized to construct highly dependable and resilient cloud infrastructures. Suppose you are tasked with developing a three-tier architecture that is readily available for your organization’s new web application. This tutorial is extensive but comprehensive. You may want to bookmark this guide for future reference on creating web, application, and data tiers.</p><h3><strong>What is a 3-Tier Architecture?</strong></h3><p>A three-tier architecture comprises three layers, namely the presentation tier, the application tier, and the data tier. The presentation tier serves as the front-end, hosting the user interface, such as the website that users or clients interact with. The application tier, commonly referred to as the back-end, processes the data. Finally, the data tier is responsible for data storage and management.</p><h3>Benefits of a 3 Tier Architecture</h3><ol><li>Scalability: Each tier can scale independently, allowing organizations to optimize their resources and minimize costs.</li><li>Reliability: Each tier can be replicated across multiple servers, improving application availability and reliability.</li><li>Performance: By dividing the application into separate layers, 3-tier architecture reduces network traffic and enhances application performance.</li><li>Security: Each tier can have its own security group, allowing different organizations to implement customized security measures for each layer.</li><li>Reduced development time: Different teams can work on different tiers simultaneously, resulting in faster deployments.</li><li>Flexibility: Each tier can be developed using different technologies and programming languages, enabling organizations to leverage the best tools for each layer.</li></ol><h3>Part 1: Creating a VPC and Subnets</h3><p>Using the architecture diagram as a reference, we will need to start by creating a new VPC with 2 public subnets and 4 private subnets.</p><p>Log into the AWS management console and click the Create VPC button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Mt5GC9mOkZNPUS7JBG92sw.jpeg" /></figure><p>We are going to create a VPC with multiple public and private subnets, availability zones, and more, so let’s choose “VPC and more.”</p><p>Name your VPC. I am using the auto-assigned IPV4-CIDR block of “10.0.0.0/16.” Choose these settings:</p><ul><li>no IPV6</li><li>default Tenancy</li><li>2 Availability Zones</li><li>2 public subnets</li><li>4 private subnets</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QnI-NuMTNNMjGALGVYKb3w.png" /></figure><p>Next, for Nat gateway choose “in 1 AZ,” none for VPC endpoints, and leave the Enable DNS hostnames and Enable DNS resolutions boxes checked.</p><p>Before we create the VPC expand the customize AZ’s and customize subnets CIDR blocks tabs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*Akln-zCf6vcFhBYUBN3mxg.png" /></figure><p>Click the “Create VPC” button.</p><p>The diagram below highlights the route that your new VPC will take.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lbALCx7FrrTQC28w7RIp8A.png" /></figure><p>You will then be shown a workflow chart that shows your resources being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/837/1*cNKg1hYHpv2sywNZaV3ltg.png" /></figure><p>View your new VPC after everything is created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LMxWqJmpFGFClJcPOXGGUQ.png" /></figure><p>Next click on the Subnets tab in the VPC console. Select one of the new subnets that was created, then under the “Actions” tab, expand the down arrow and select “Edit subnet settings.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rZa4XHjWhLAIN55kPxiLIg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*h54KWTjqwl50wrx3em-oCg.png" /></figure><p>Check “Enable auto-assign IPv4 address” and “Save.” We need to do this for all 6 new subnets that were created.</p><h3>Update Web Tier Public Route Table</h3><p>We need to navigate to the route table tag under the VPC dashboard to make sure that the Route table that was automatically created is associated with the correct subnets. Below I highlighted in green the correct public subnets that already associated. If you have none or you only created a stand-alone VPC, you will have to click on “edit subnet associations” and select the ones needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bdY47UKnouSe4y33-Enm7Q.png" /></figure><h3>Part 2: Creating a Web Server Tier</h3><p>Next we will create our first tier that represents our front end user interface (web interface). We will create an auto scaling group of EC2 instances that will host a custom webpage for us. Start by heading to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WkCxz1mdFBA33GtYDDFn2Q.png" /></figure><p>Next we are going to launch an EC2 instance.</p><p>Name your instance and select an AMI. Please use Amazon linux2; I used the updated version and was not able to ssh into it</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6NM3lDGwpMhUAZ_0mYUksw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V4tyKBNt_S4ubTNIPx4dKA.png" /></figure><p>Select the key pair that you will use, and make sure to select your new VPC and the correct subnet. Auto-assign IP should be enabled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*4b_sTiZx1W1w2FMPJVdn1A.png" /></figure><p>Create a new security group. For inbound security group rules, add rules for ssh, HTTP, and HTTPS from anywhere. This is not standard or safe, practice, but for this demonstration it is fine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*m3URXl__XZn4R1EtBWQFwA.png" /></figure><p>Leave the configuration storage settings alone. In the Advanced details, head all the way to the bottom. We are going to use a script to launch an Apache web server when the instance starts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*qvNwx-TniTytrZnpaq6ftA.png" /></figure><p>Launch your new instance!</p><p>Once your instance is up and running, copy the public IP address and paste it into a web server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tfxiaA5uWQNCMdQwIbzUUw.png" /></figure><p>I am having an issue getting mine to say “My Company Website”. I will solve this at a later date. All that really matters is that the instance has Apache installed, and we can reach it. Next I will SSH into the instance to make sure that works as well.</p><p>For this project to work we need to create an auto scaling group that we attach to our EC2 instance. This will increase our reliability and availability. Next we are going to create a launch template. Before we do this we will need to define a launch template; this template will outline what resources are going to be allocated when an auto scaling group launches on-demand instances. Under the EC2 dashboard, select Launch Templates, and click the “Create launch template” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8OiePC-j4G2WicwHWWcIoA.png" /></figure><p>Name your template, and check the box to “provide guidance.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*CXb70dzlzua-NG1KBDaEKQ.png" /></figure><p>Use our recently launched AMI t2.micro instance type and select your key pair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*k-fHJK8fc0H13WcMW0T72A.png" /></figure><p>For the firewall, use “Select existing security group,” and make sure the security group (SG) that we created for the web tier is selected. Under Advanced network configuration, enable Auto-assign public IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*fELZYhvq63O9EbY8MePZhg.png" /></figure><p>We are going to leave the storage options alone for now. Click on the Advanced details tab, scroll down, and enter the same script as we did earlier for our EC2 instance.</p><p>Click the “Create launch template” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q4qMgyeWUALtJFqsvRjFsA.png" /></figure><p>Navigate to the autoscaling tab at the bottom of the EC2 dashboard. Click “Create auto scaling group.” The launch template that we just finished creating is the template that our auto scaling group will use to launch new EC2 instances when scaling up.</p><p>Name your auto scaling group (ASG), choose the launch template that you created, then click the Next button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cKwcw_EYoLGBw7YdYNF20w.png" /></figure><p>Under Network, make sure to select the VPC that you created earlier, then also under availability zones and subnets select the public subnets that were created; yours may differ.</p><p>Click the Next button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*vhGnVnkcWgguxCfbD2C9lg.png" /></figure><p>Now we are given to option to allocate a load balancer for our ASG. A load balancer will distribute the load from incoming traffic across multiple servers. This helps with availability and performance.</p><p>Select “Attach to a new load balancer” and “Application load balancer,” name your load balancer, then select “Internet facing” as this is for our web tier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*gXoUcjmxA00NxNHDjtN4RQ.png" /></figure><p>Your VPC and the two public subnets should already be selected.</p><p>Under the “Listeners and routing” section, select “Create a target group,” which should be on port 80 for HTTP traffic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*MDstEI98RLOgwk3uYDkvgQ.png" /></figure><p>Leave No VPC Lattice service checked.</p><p>Click to turn on Elastic Load Balancing health checks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*xPBniUBNMcK6CNrTVhMQJw.png" /></figure><p>Check “enable group metrics collection within CloudWatch.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*b3NMX5L87NtrNZyU3HV9GA.png" /></figure><p>Next we configure the group size and scaling policy for our ASG. For reliability and performance, enter 2 for desired capacity and minimum capacity. For maximum capacity, enter 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*fC2ETFtS_8lWbG6zupQbCA.png" /></figure><p>In scaling policies, select “Target tracking scaling policy.” The metric type should be set to “average CPU utilization” with a target value of 50.</p><p>Click the Next button.</p><p>On the next screen we could add notifications through SNS topics, but I skipped this for now. Click the Next button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*qkihP8zqJeWQOJHod8pq9g.png" /></figure><p>The next screen allows you to add tags which can help to search, filter, and track your ASG’s across AWS. For now just click the Next button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*2_0RcjddLKRDis-KKAgRiA.png" /></figure><p>Review your setting on the next page, and at the bottom click the “Create auto scaling group” button.</p><p>You should see a lovely green banner declaring your success here. After the ASG is finished updating capacity, navigate to your EC2 dashboard to confirm that your new instance has been created.</p><p>Note: In my previous examples my names were not accepted for the auto scaling group, make sure you follow the naming conventions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A_nFGgNJ1IVz7cyS5PJBtA.png" /></figure><p>As you can see, the ASG is doing its job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dTi5oC1MfbabkEHPR2UdZA.png" /></figure><p>Before we move on it is a good idea to take a moment and connect to the instances that were created; as you can see: success!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xz1PEn6-zHVpSDULxLt0mQ.png" /></figure><h3><strong>Part 3: Creating an Application Tier</strong></h3><p>Next up we are going to create the back-end of our 3-tier architecture. We could create an EC2 instance first, but for this portion I will start by heading to the Launch Templates tab under the EC2 dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v862F5BPkMkR9WBfwoWQiA.jpeg" /></figure><p>Name your new template, and select the Guidance tab again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*LZzyrpbUa1DNiMtwfkStng.png" /></figure><p>Select browse more AMI’s, then Amazon Linux 2 for your AMI. Select t2.micro for instance type, and also select you keypair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*g47TszkTJ1lMvwt85cVBfA.png" /></figure><p>Under the network setting, we want to limit access to the application tier for security purposes. You don’t want any public access to the the application layer or the data tier after it. We will create a new security group. Select our VPC; I realize now that for this part I could have chosen a better name!</p><p>Name your new SG and select the VPC that we created when we started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*ioY3DA8xwhw_1Lq9vcMTJg.png" /></figure><p>We will create 3 security group rule starting with ssh; use My IP as the source. For Security Group 2, use Custom TCP; the source here is the security group from our web tier (tier-1). For the third group, select All ICMP-IPv4, and set the Source type as anywhere. This will allow us to ping our application tier from the public internet to test if traffic is routed properly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*4boxu5GdkIbwEHcxA295rw.png" /><figcaption>This is my updated screenshot</figcaption></figure><p>We will again leave the storage volumes alone, and head to the bottom of Advanced details to enter our script. And then click next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ro2us7-p4G5P2Jq_ZXigHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wDnuKbzwRATBoYdwpTrhlA.png" /></figure><p>After I had created the launch template I realized I made a couple of mistakes. Instead of modifying the template I decided to start from scratch, but again upon further reflection, in the future I would just modify the template as this would save time, and we all know time is money. The whole reason for these exercises is to learn, right!</p><p>I went back to the security group and updated my inbound rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rE_nQ9wfUk0ZhqzyDmojJQ.png" /></figure><p>Once this was fixed I went back and recreated the application layer template using the ApplicationTierSG1 that I just altered. I just double checked the security group rules and used the same settings for everything else above.</p><h3><strong>Application Tier Auto Scaling Group</strong></h3><p>Ok, now we are ready to create our auto scaling group for the application layer. Under the EC2 dashboard go to create an auto scaling group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HPaMIPZbZaHS4ZylD3A_cw.png" /></figure><p>Name your new ASG and select the proper launch template, then click the Next button.</p><p>Choose the correct VPC and 2 private subnets, then click the Next button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*4ZYk353MhBgXLt-s-ognbg.png" /></figure><p>We are again given the option to attach a load balancer, and we want to do this. Select an application load balancer, name it, and set it as an internal load balancer. Double check that the VPC and subnets are correct. Mine are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*e-xPhC83g6VqE4tmrN2uTg.png" /></figure><p>Under “Listeners and routing” create a new target group, and use port 80 once again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/1*pHlhjOL7y6hvVf_-D-iU2g.png" /></figure><p>Below I have again chosen to turn on health checks and enable group metrics within CloudWatch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*RyyP-CZ7E9R-eSTG-j8xPw.png" /></figure><p>On the next screen set your desired capacity, minimum capacity, and maximum capacity again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U0rTHhU0NYGIm0dyINueuQ.png" /></figure><p>Then I have chosen to select target tracking with a CPU utilization of 50%.</p><p>Click the Next button, add notifications if you want or need and then tags. Review your new ASG settings and create it.</p><p>As you can see below, my new application layer ASG is updating the capacity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3CdBqO9sfEnCSqpuhPMaYQ.png" /></figure><p>Once the new EC2 instances are created and running, we will try to ssh into them. If we set it up correctly, we should not be able to.</p><p>When I tried to SSH into the application tier EC2 instance running, the connection timed out, this is exactly what we want here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aEN9tq16ETyMPMFmOQBUGQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wFcl5OwUh33XwlKJqALmsg.png" /><figcaption>I also tried to connect using EC2 connect. This failed as well.</figcaption></figure><p>We still need to check if our tier-1 servers interact with our tier-2 servers. To test this, you will need to log into your tier-1 EC2 instances via SSH and run a ping command to a private IP address of our tier-2 servers. Below you can see a successful ping.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OXwuCEbw07oL4NNlLEyXjA.png" /></figure><p>After waiting for my new ASG to update its capacity, it only started 1 instance. I must have clicked back and reset my capacity selections to the default 1 instance. Below I updated the capacity that I wanted.</p><p>I apologize for the changes in settings here. I completed this project over a long period of time and multiple computers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9n7sH7bTpMT2n8z7OiMM7g.png" /></figure><h3><strong>Update the application (Tier-2) Route table</strong></h3><p>Head back to the VPC dashboard, select Route tables, and select one of the route tables that was automatically created when we created our VPC. I only have 1 subnet associated with this table so click on Edit subnet associations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pABcRZ_UWghXDS2OMKYWqA.jpeg" /></figure><p>Add another subnet that is private.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D01jGJkJzR4f7bbd_5WKwg.png" /></figure><h3><strong>Part 4: Created a Database Tier</strong></h3><p>Almost there! We have created 2 out of the 3 tiers and tested both successfully.</p><p>We are now going to build our database; AWS offers several types of databases but for this exercise we are going to use a MySQL RDS database.</p><h3>Create a DB Subnet Group</h3><p>We will begin by creating a subnet groups. Navigate to the RDS console and on the left side menu, click “Subnet Groups” and then the orange “Create DB subnet group.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pClXtgX3NGJKF0sRZyxYqA.jpeg" /></figure><p>For the next part we need to know the availability zones for the last two subnets that were automatically created. Head back to the VPC console. Under Subnets, find the last 2 subnets that you have; make sure <strong><em>not</em> </strong>to select the private subnets that you already used in tier 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*Txap86F9b10ahU0pkDakjA.png" /></figure><p>Back at the RDS console, select the availability zones that you are going to use.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*uXFULIS6bFK0Md5PZG7oyw.jpeg" /></figure><p>Next up we need to select the proper subnet; the drop down menu only lists the subnet ID. Below is another screenshot of my subnets; the second column is the ID.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*CFXzRDMOo77a3pWpPBx2JA.png" /></figure><p>Back in the RDS console click Create database.</p><p>Select the MySQL DB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vKXJqzrxFz-Ny14kKOB3Zg.png" /></figure><p>Next you can choose a muli-AZ deployment with 3 database instances. One is a primary instance with two read-only standby instances. This makes for a very reliable system, but we do not need this at this time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*fh6le5m93fqNTsC-CfXSdg.jpeg" /></figure><p>There are also availability and durability options; however, with the free-tier, none are available. We do not need them either.</p><p>Under Settings, name your DB and create a master username and password. This username and password these should be different than your AWS account user login, as it is specific to the database you are creating.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*xgMBlZgwliVLsNtXHEI2Jg.png" /></figure><p>You will need your username and password. Make sure to store them in a secure place!</p><p>Under Instance configuration, the burstable classes option is is pre-selected because it’s the only one available for the free tier. I left my instance type as adb.t2.micro. You can add storage as needed; I left mine on default settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/790/1*NMc5uurfwxj514LJtdqktg.jpeg" /></figure><p>We are going to set up our network manually so choose not to connect to an EC2 resource. Select the proper VPC; the subnet group that you created earlier should be listed as default. Select Create new VPC security group (firewall).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*lwLz4_3guDqXgGyvOw5zFA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/833/1*suxVr_FFgwdexTgzlt1sfQ.png" /></figure><p>In Database authentication I left the default checked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/773/1*1MYfPCq289g5R7EErBCHaA.png" /></figure><p>Click the “Create database” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*WGvZMEiHyQgmZn4v8XP4dQ.png" /></figure><h3>Update the Database Tier Security Group</h3><p>Navigate over to the VPC console, select Security groups on the left side menu, and then find the database tier security group you just created.</p><p>Select the security group that you just created. You need to edit inbound rules; by default the database SG has an inbound rule to allow MySQL/Aurora traffic on port 3306 from your IP address. Delete this rule.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kslD9XYwcqv2C_QDp2jIPw.jpeg" /></figure><p>Create a new rule for MYSQL/Aurora on port 3306: for the Source, select Custom to add your security group for your application layer (tier-2 SG).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RfgCwnc50lU3cYdSdvix-Q.png" /></figure><h3>Update Tier 3 Private Route Tables</h3><p>It’s the home stretch, people — I hope you’re still here!</p><p>In the last step for our database tier, we need to make sure that the route table we associate with our databases private subnets has both subnets listed in subnet associations. If not, add the other subnet, and save.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xQVySttmgHDhfpP8Ai9uVA.jpeg" /></figure><p>Our three tier architecture is done! We have already tested our web and application layers, but we are going to go a step futher here.</p><h3>Part 5: Testing</h3><p>We can’t directly SSH to the database, but we can use an SSH forwarding agent to achieve this.</p><p>You need to add your access key pair file to your keychain. To do this, first make sure you are in your local host (use the command <em>exit</em> to get out of any EC2 instance you’re connected to). Then use the following command:</p><pre>ssh-add -K &lt;keypair.pem&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yGjGQS92lfJ71eKv7sWwVg.png" /></figure><p>Now that your key pair file is added to your keychain, the SSH agent will scan through all of the keys associated with the keychain and find your matching key.</p><p>Now reconnect to the web tier EC2; however, this time use -A to specify you want to use the SSH agent.</p><pre>ssh -A ec2-user@&lt;public-ip-address&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EQZkqo9yYNLOEK7FkNsgIw.png" /></figure><p>Once you are logged back into your tier-1 EC2, use the following command to check if the SSH agent forwarded the private key.</p><pre>ssh-add -l</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vls7c0qkoPj3nzYvh16CSQ.jpeg" /></figure><p>Our key pair has been forwarded to our public instance. Go copy your tier-2 application layer private IP address and copy it into the next command.</p><pre>ssh -A ec2-user@&lt;private-ip-address&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kiMFerRBfYRakKFbMYmzkg.png" /></figure><p>We have now SSH’ed from your public tier 1 web instance into your private tier 2 application instance!</p><p><strong>Testing Connectivity to the Database Tier</strong></p><p>There are a few ways you can connect to your RDS database from your application tier. One way is to install MySQL on your private tier 2 instance to access your database. We are going to utilize this method. While logged into your application tier instance, use this command:</p><pre>sudo dnf install mariadb105-server</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tKQetTol7KdF3xHapWyuuA.png" /></figure><p>This command installs the MariDB package, which is used to read MySQL. Once installed, you should be able to use the following command to log into your RDS MySQL database. You will need your RDS endpoint, user name, and password. To find your RDS database endpoint, navigate to the database you created and find the endpoint under Connectivity &amp; Security.</p><pre>mysql -h &lt;rds-database-endpoint&gt; -P 3306 -u &lt;username&gt; -p</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4xLU7ShtSLIlvwxrVN-RuQ.png" /></figure><p>We have now successfully connected to our MySQL database from the application tier. We have connectivity with all of our tiers!</p><p>I had a lot of trouble with this project, especially with connectivity. I did not have any issues with the web tier and the application layer; however, I was not able to download MySQL into my secondary application layer. I spent a lot of time reviewing everything I did and looking for solutions. I even rebuilt my entire project, but doing all this work I consistently overlooked one single point: the AMI I was using was not Amazon Linux2. It was Amazon Linux 2023. This is a very valuable lesson I will not forget!</p><p>I hope that this tutorial was helpful to some of you. Please remember to go back and delete the resources that cost money. Stop your <strong><em>Ec2 instances </em></strong>and delete your <strong><em>NAT gateway, elastic IP, and ALBs. </em></strong>This is a good practice to get into to prevent running up unnecessary<strong><em> </em></strong>costs. Good luck!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23f37d49bd25" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Magic: Spinning Up Dual Deployments and Shared Services with Custom ConfigMaps and…]]></title>
            <link>https://medium.com/cloud-native-daily/kubernetes-magic-spinning-up-dual-deployments-and-shared-services-with-custom-configmaps-and-92a25560fc11?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/92a25560fc11</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubectl]]></category>
            <category><![CDATA[k8s]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Sun, 23 Jul 2023 05:52:50 GMT</pubDate>
            <atom:updated>2023-07-25T04:29:18.921Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Kubernetes Magic: Spinning Up Dual Deployments and Shared Services with Custom ConfigMaps and Persistent Storage</strong></h3><h4>A comprehensive guide to spinning up deployments and shared services using custom ConfigMaps and persistent storage in Kubernetes.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ZhTXciBfIlAR20uNeum7A.png" /></figure><p>On to a new magical adventure, this time with Kubernetes! Well maybe not magical but definitely cool. Today we will embark on a journey to spin up two deployments. The first deployment, will consisting of two pods, will run the NGINX image while displaying a custom index.html page that declares “This is Deployment One,” thanks to our ConfigMap’s mystical guiding hand. The second deployment will also hold two pods with the NGINX image, enchanting visitors with its own custom index.html page bearing the inscription “This is Deployment Two,” courtesy of its configmap.yaml file.</p><p>But the real magic happens when we create a service that unites both deployments, offering access to both using a single IP address and port number. No hocus-pocus is needed; it’s Kubernetes in action, bringing these deployments together.</p><p>To verify, we’ll use the curl command to witness the index.html pages from both Deployment One and Deployment Two.</p><p>For those looking to take their Kubernetes skills to the next level, we’ll also delve into creating a Kubernetes persistent storage manifest, providing 250MB of disk space on the host computer in the /temp/k8s directory. This ensures our data stays safe and available, without any magical tricks required.</p><p>If you are joining me, I really appreciate it, so on with this adventure where we explore dual deployments, shared services, custom ConfigMaps, and the practical magic of Kubernetes to make it all work seamlessly!</p><p>In this project, I will try to be short and to the point. Go here for information on <a href="https://kubernetes.io/">Kubernetes!</a> This is a great resource and will cover all aspects of Kubernetes.</p><p>For my project, I downloaded microK8s, and kubectl onto an EC2 instance in AWS. In this project, I will not be covering installing microK8s or kubectl, but there are a lot of resources on the web describing just how to do this.</p><p><strong><em>Step 1: Create ConfigMaps</em></strong></p><p>I have a ConfigMap for both deployments named “deployment-one-configmap.yaml” and “deployment-two-configmap.yaml”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HbNEq0v4UK_XmkPmZObeHg.png" /></figure><p>In the screenshot above, you can see that I used vim to add the code needed for each deployment. Below you see what I added for both files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*Pr-OM6xazKEVWcKqLrN08w.png" /><figcaption>deployment-one-configmap.yaml</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gZXfdC3-FY-D4nT9HcWKwg.png" /><figcaption>deployment-two-configmap.yaml</figcaption></figure><p>I now need to apply these new files to my Kubernetes cluster. Using the commands below.</p><pre>kubectl apply -f deployment-one-configmap.yaml<br>kubectl apply -f deployment-two-configmap.yaml</pre><p>The following command get kubectl configmap will show you if those files have been applied.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9D9D0lKBT1wfw7vffgLa2w.png" /></figure><p><strong><em>Step 2: Create Deployments for your pods</em></strong></p><p>Next, I created two deployment files, each using the nginx image and referencing the respective ConfigMaps that I created earlier for the custom index.html pages.</p><p>Deployment One: named (deployment-one.yaml) is below.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: deployment-one<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: nginx<br>  template:<br>    metadata:<br>      labels:<br>        app: nginx<br>    spec:<br>      containers:<br>        - name: aaron-nginx<br>          image: nginx<br>          ports:<br>            - containerPort: 80<br>          volumeMounts:<br>            - name: config-volume<br>              mountPath: /usr/share/nginx/html<br>      volumes:<br>        - name: config-volume<br>          configMap:<br>            name: deployment-one-configmap</pre><p>Save this and then create another file for deployment two and add the code below to the file named “deployment-two.yaml”.</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: deployment-two<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: nginx<br>  template:<br>    metadata:<br>      labels:<br>        app: nginx<br>    spec:<br>      containers:<br>        - name: aaron-nginx<br>          image: nginx<br>          ports:<br>            - containerPort: 80<br>          volumeMounts:<br>            - name: config-volume<br>              mountPath: /usr/share/nginx/html<br>      volumes:<br>        - name: config-volume<br>          configMap:<br>            name: deployment-two-configmap</pre><p>We now need to apply these deployments to the cluster.</p><pre>kubectl apply -f deployment-one.yaml<br>kubectl apply -f deployment-two.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*94mBWc9tMr8fWr_LRQePpg.png" /></figure><p>After looking around a little more I realized that instead of the command above, you could run a simpler command kubectl apply -f deployments.yaml. This will accomplish the same thing.</p><p>Now if we run the command kubectl get pods:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UpmY-FXCef1BW8vhkOSRig.png" /></figure><p><strong><em>Step 3: Create a service</em></strong></p><p>We will now create a service that points to both deployments.</p><p>The file is named “(service.yaml)”, and is below.</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: nginx-service<br>spec:<br>  selector:<br>    app: nginx<br>  ports:<br>    - protocol: TCP<br>      name: http<br>      port: 80<br>      targetPort: 80<br>      nodePort: 30080   # Add the desired NodePort number here<br>  type: LoadBalancer   </pre><p>Note: I had several versions of each of these files. I hope these are all pretty much copy and paste, but you may have issues with syntax or other issues. After you save this file: apply it with:</p><pre>kubectl apply -f service.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*_diFr_PRiNz837EIdI1Zjw.png" /></figure><p>If you run kubectl get service you should see something similar to this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mrEQ_D4VWbkZQ29jBJLvwg.png" /></figure><p><strong><em>Step 4: Validate the Deployments</em></strong></p><p>To do this I first had to obtain the Ip of the service that was created. The following command will do this kubectl get svc nginx service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T_2_zwsLDAc1fEooOgdtlg.png" /></figure><p>In the next command input, I input the Cluster-IP address, and service port that is listed above.</p><pre>curl Service_IP_Address:Service_Port</pre><p>You should see an output like this, and the “Deployment One” and “Deployment Two” alternating in the curl response.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/784/1*JcfGvKsKUhybcQt8B2Hh7Q.png" /></figure><p>I had an issue here, in my original service.yaml file I had used nginx in the app under selector. I kept getting only the first deployment, and then I tried using both like this but specifying multiple labels in a selector like this is not allowed.</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: nginx-service<br>spec:<br>  selector:<br>    matchLabels:<br>      app: nginx-deployment-one<br>      app: nginx-deployment-two<br>  ports:<br>    - protocol: TCP<br>      port: 80<br>      targetPort: 80<br>  type: ClusterIP</pre><p>I ended up using the following which was not a problem</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: nginx-service<br>spec:<br>  selector:<br>    app: nginx<br>  ports:<br>    - protocol: TCP<br>      name: http<br>      port: 80<br>      targetPort: 80<br>      nodePort: 30080<br>  type: LoadBalancer</pre><p>Verify your deployments with kubectl get deployments.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7R-wCiOCFLay9NwHxfX_4g.png" /></figure><p>Next, we can use the command kubectl gt svc to get the IP address and port for our service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2p5xIvEk7vUzwx8k4vJxEw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*_ksaTfO-AkGxBvwXhVpceA.png" /></figure><p>Below you can see the successful curl command points to both</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rwEii34ngFw6-Dm6cSi0OA.png" /></figure><p>I had to go back to edit my second config file; unfortunately, I forgot to get a new screenshot of the curl command with the proper title for the second deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*jtHzl0vM1lSRqIo8Nq7HYQ.png" /></figure><p>I forgot to get a screenshot of the second one, but if you keep giving this command you should see “This is Deployment Two”.</p><p><em>Step 5: Advanced portion</em></p><p>In this portion, I will create a Kubernetes persistent storage manifest that gets 250MB of disk space on the host computer in the /temp/k8s directory. Create a corresponding persistent storage claim manifest file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5FXoSYIMU4kXl-PB1wogA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vEfM4z2WWF0zg5-Fod6oQQ.png" /></figure><p>After this, I had to make sure that I added the volume mount for the persistent volume claim for both of the deployments.</p><p>I added volume mounts for the Persistent Volume Claim (PVC) to both “deployment-one-configmap” and “deployment-two-configmap” files. This will allow the pods to use persistent storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nUKSHY2jDi_BDKZ3gP_tqA.png" /><figcaption>This screenshot only shows one file</figcaption></figure><p><strong><em>Complex portion:</em></strong></p><p>For the complex portion of this project, we are going to create a Kubernetes CronJob that spins up both deployments and the service at a specific time, and then test it.</p><p>The first thing we are going to do is create a new file called “cronjob.yaml”.</p><p>Once created I used VIM to add the new code that is needed for this step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zIXGt_FWnFofoEJ-Ymp_YQ.png" /></figure><p>Run the command kubectl apply -f cronjob.yaml to apply cronjob to your Kubernetes cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*r4_sy10PqFUCAi3ekkBdQg.png" /></figure><p><strong><em>Test the cronjob:</em></strong></p><p>Next up we want to test the cronjob to make sure that it is functioning properly. Before we do that we will use the following command to make sure that the cronjob has been created and is active.</p><p>kubectl get cronjobs</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KRMfFqpiGy7inkibqG5Ylw.png" /></figure><p>Next up, to test we will trigger the cronjob manually. Use the following code to trigger yours. Note: Make sure to have the correct name for your cronjob, and insert it in here if it is not my-cronjob.</p><pre>kubectl create job --from=cronjob/my-cronjob my-cronjob-manual-run</pre><p>This command creates a new Job named my-cronjob-manual-run based on the named your cronjob name CronJob. This Job will run once and initiate the deployments and service based on the schedule defined in the CronJob.</p><p>Next, we will see about the status of the job to make sure that it was completed with the following command. When running this command you will want to make sure that my-cronjob-manual-run Job appears in the list, and its status shows &quot;Completed&quot;.</p><pre>kubectl get jobs</pre><pre>kubectl get deployments<br>kubectl get services</pre><p>I ended up creating a second job, with a different name to see if the outcome would be different, but neither cron job completed successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yHDbFc3S88Fh3TWlCBMGnQ.png" /></figure><p>To troubleshoot further I ran kubectl describe job my-cronjob-manual-run. At the bottom you can see I had deleted one, and that the current Job has reached the specified backoff limit, so it failed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d-lU-tvwutluqadHgiIT9w.png" /></figure><p>To continue troubleshooting I ran kubectl get pods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MqRs4MNe0ttU_NKdJYfvVw.png" /></figure><p>Next, I ran kubectl describe pod &lt;pod-name&gt;, and received the following output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XeU5v8tkOaboE5Nt2XeMrA.png" /></figure><p>I ran multiple commands trying to find information about the pod for my cron job, only to realize after a significant amount of time that because the job failed, the pod was actually deleted and not available. Why my pod named (my-cronjob-manual-run-9rkvt) still says 1/2 ready does not make sense. When I ran kubectl logs my-cronob-manual-run-9rkvt, the output still says that the pod is not available, which makes sense if the pod has in fact been deleted. I tried to run the command to describe the pod again with the same results.</p><p>When a Pod enters a “CrashLoopBackOff” state, it means that the Pod keeps crashing immediately after starting due to some issue. Kubernetes automatically restarts the Pod, but if the issue persists, it will keep failing in a loop.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZqrhZMhD_1rV6Z4siI2b6g.png" /></figure><p>At this point, I have not been able to get any further in troubleshooting, but I wanted to include it here because even though I have not solved this problem yet, a lot of learning was happening and that is what this journey is all about!</p><h4><strong><em>Conclusion:</em></strong></h4><p>During this project, I had the opportunity to work with various Kubernetes components, such as Pods, Deployments, Services, ConfigMaps, and more. I gained insights into how these building blocks come together to create a highly resilient, scalable, and efficient infrastructure for applications.</p><p>One of the most significant advantages I see with Kubernetes is its ability to handle complex workloads. Whether you’re running a small web application or a large-scale microservices architecture, Kubernetes ensures that your application runs smoothly, can scale seamlessly, and recovers from failures automatically.</p><p>Thank you again for joining me on this journey. I hope you’ve found this exploration of Kubernetes as rewarding and enlightening as I have.</p><h4>Further Reading:</h4><ul><li><a href="https://medium.com/cloud-native-daily/kubernetes-monitoring-tools-584d82c94185">Kubernetes Monitoring Tools</a></li><li><a href="https://gethelios.dev/opentelemetry-a-full-guide/?utm_source=medium&amp;utm_medium=cloud+native+daily">OpenTelemetry: A full guide</a></li><li><a href="https://gethelios.dev/blog/kubernetes-monitoring-opentelemetry/?utm_source=medium&amp;utm_medium=cloud+native+daily">Kubernetes Monitoring with OpenTelemetry</a></li><li><a href="https://gethelios.dev/blog/cut-engineering-costs-save-time-helios?utm_source=medium&amp;utm_medium=referral&amp;utm_campaign=cloud+native+daily">Microservices Monitoring: Cutting Engineering Costs and Saving Time</a></li><li><a href="https://gethelios.dev/blog/testing-microservices-with-helios/?utm_source=medium&amp;utm_medium=referral&amp;utm_campaign=cloud+native+daily">Testing Microservices - Trace Based Integration Testing Example</a></li><li><a href="https://gethelios.dev/blog/opentelemetry-tracing/?utm_source=medium&amp;utm_medium=referral&amp;utm_campaign=cloud+native+daily">OpenTelemetry Tracing: Everything you need to know</a></li><li><a href="https://gethelios.dev/blog/serverless-observability/?utm_source=medium&amp;utm_medium=cloud+native+daily">Serverless observability, monitoring, and debugging explained</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=92a25560fc11" width="1" height="1" alt=""><hr><p><a href="https://medium.com/cloud-native-daily/kubernetes-magic-spinning-up-dual-deployments-and-shared-services-with-custom-configmaps-and-92a25560fc11">Kubernetes Magic: Spinning Up Dual Deployments and Shared Services with Custom ConfigMaps and…</a> was originally published in <a href="https://medium.com/cloud-native-daily">Cloud Native Daily</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create a Docker Swarm using AWS]]></title>
            <link>https://medium.com/@abach06/create-a-docker-swarm-using-aws-4ad1988366e4?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ad1988366e4</guid>
            <category><![CDATA[docker-swarm]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Thu, 20 Jul 2023 03:44:32 GMT</pubDate>
            <atom:updated>2023-08-06T06:08:36.703Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z8JHDF6mzcJstNEQknKEQQ.png" /></figure><p>Today, by utilizing AWS EC2 instances we are going to create and launch a Docker swarm with one manager node and three worker nodes. We will then verify the cluster is working by deploying the following tiered architecture.</p><ul><li>A service based on the Redis docker image with 4 replicas.</li><li>A service based on the Apache docker image with 10 replicas.</li><li>A service based on the Postgres docker image with 1 replica.</li></ul><p><strong>Prerequisites:</strong></p><ul><li>AWS account</li><li>Basic AWS EC2 knowledge</li><li>Basic knowledge of Docker and its commands</li><li>Access to CLI</li><li>Knowledge of basic Linux commands</li></ul><p><strong>Set up a swarm manager security group</strong></p><p>See the <a href="https://www.bretfisher.com/docker-swarm-firewall-ports/">references</a> from <a href="https://www.bretfisher.com/">Bret Fisher</a> on ports.</p><p>First, in order to use <strong>Docker swarm</strong> we need to open several ports specific to it. We will also need to open an ssh port. Two separate security groups will also need to be created as the ports differ for <strong>Swarm managers</strong> vs <strong>Swarm workers</strong>.</p><p>In the <strong>AWS console, </strong>navigate to the <strong>VPC console, </strong>click on <strong>security groups, </strong>then “create security group.” Name your group and give it a description. You can specify a VPC or use a default.</p><p>You will need to add the following ports under your inbound rules:</p><ul><li>ssh port 22</li><li>Custom TCP 2377</li><li>Custom TCP 7946</li><li>Custom UDP 7946</li><li>Custom UDP 4789</li><li>Custom Protocol 50 all</li></ul><p>After you add all of the required ports, create your security group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wEgAVsvUSY6HEC8k7ZmmLA.png" /><figcaption>I had to go back and add Anywhere for the source for the rest. This made for an easier screenshot.</figcaption></figure><p><strong>Swarm worker security group</strong></p><p>Now let’s create another security group for our Swarm workers. Use the same process above to create another security these inbound ports:</p><ul><li>ssh 22</li><li>Custom TCP 7946</li><li>Custom UDP 7946</li><li>Custom UDP 478</li><li>Custom Protocol ESP (50) all</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-lxTdhKqx21xbrAiMoFH7w.png" /></figure><p><strong>Create and start 4 EC2 instances</strong></p><p>If you need more in-depth information to help create EC2 instances, check out my guide <a href="https://medium.com/@abach06/a-journey-into-the-cloud-setting-up-a-web-server-in-the-aws-cloud-cc6c8964c6a2">here</a>.</p><p>For this project we need to create four t2.micro Amazon Linux 2 AMI EC2 instances. One instance is provisioned for the manager node and one each for the worker nodes.</p><p>In the AWS console, navigate to the EC2 console and click <strong>Launch Instances. </strong>Name your new instance and<strong> </strong>select the <strong>Amazon Linux 2 AMI</strong>, and <strong>t2.micro I</strong>nstance type.</p><p>The settings for all of the EC2 Instances that we are creating are almost identical except for the security group.</p><p>For the first EC2, we are creating the Swarm manager node instance. Name your instance, select Amazon Linux 2 AMI from the AMI selection. t2.micro, and select your keypair. If you don’t have one, create one. The default VPC is selected but you can change it if you specified a VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_ixC3pvjXrte29Hnx6DWg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GRiQHHKIM-qE1vkqeiXnmg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gJO1jNNtP6wP0EZwF2uJeg.png" /></figure><p>Leave the storage and other options as default. Once you have created the first instance for your Swarm Manager, go ahead and create three more instances for your swarm Workers; label them swarm_worker_node_1, 2 and 3. Make sure to select the worker security group for them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fFWNnXmWgZyQhYOlcp4sHQ.png" /></figure><p><strong>Connecting and installing Docker:</strong></p><p>Next up we will connect to each of the instances that we created and install Docker.</p><p>Click on the the <strong>Swarm_Mgr </strong>box,<strong> </strong>and click on the connect button. In your CLI, navigate to the folder that contains your keypair. Under the ssh tab in the connect menu, copy the example of the connect command, paste that into your CLI, and hit enter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v7nsGmrE2xlD8331y7__1Q.png" /></figure><p>You will be asked if you want to continue connecting. Type yes, and return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*guAalk_wVn_kQSrpW8UsOg.png" /></figure><p>You can next open 3 more terminal windows and connect to each of the worker node instances. This will be more efficient.</p><p>Next let’s update all of the packages. This needs to be done for all of the instances.</p><pre>sudo yum update -y</pre><p>Our next task is to install Docker on our instances.</p><pre>sudo yum install docker -y<br>sudo systemctl enable docker<br>sudo systemctl start docker<br>sudo docker version</pre><p>This would have been much easier to do had I just created a script when I had created and launched my EC2 instances. Doing is learning, and this will be a lesson well remembered!</p><p>Once you have installed and connected to Docker, you should see something similar to this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YuIsXgO1l7QCWQJzjnZtHQ.png" /></figure><p>We are also going to install Docker Compose on our instances. Run the commands below in each of your instances.</p><pre>sudo yum install curl<br>sudo curl -L &quot;https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)&quot; -o /usr/local/bin/docker-compose<br>sudo chmod +x /usr/local/bin/docker-compose<br>docker-compose --version</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lw0eT-viI-V-e9w48gQm9g.png" /></figure><p>At this point you should have created and ssh’d into all 4 of your EC2 instances, updated the packages, and installed Docker and Docker compose.</p><p>Continuing on, we will need to add a user to the Docker group. Enter the following command into each instance, then exit and reconnect to each instance.</p><pre>getent group</pre><pre>docker swarm init</pre><p>The command above should be run on your manager node. The output below shows that this node is now a manager.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N9efmoJD3dHc5XmFRDLJyw.png" /></figure><p>Copy the command to add a worker.</p><p><strong>Add Worker Nodes</strong></p><p>If you aren’t already connected via ssh to your other EC2 instances, connect to them now. Then paste the code that we copied above into each of the worker nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xvvd3Wjdj71TlCQqiK7hOw.png" /></figure><p>Go back to the manager node CLI and enter the following command.</p><pre>docker node ls</pre><p>This will verify your Docker Swarm setup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W7sUouJnZgD5prLPZ1v85Q.png" /></figure><h3>Create a Redis Service with Four Replicas</h3><p>From now on we will be working from the Swarm manager, so you no longer need to stay connected to the three worker nodes.</p><p>Now that we have successfully created our swarm, we are going to run a Redis service based on the official Docker Redis image. A <a href="https://docs.docker.com/engine/swarm/key-concepts/#:~:text=A%20swarm%20consists%20of%20multiple,worker%2C%20or%20perform%20both%20roles.">Swarm Service </a>is a definition of tasks to be executed on the manager or worker nodes. Go <a href="https://hub.docker.com/_/redis">here</a> to see and inspect the image. You will be able to see the version of the image.</p><p>Below you can see the version in bottom left. In the top right, you can click a link to pull redis right from here; however, we are not going to use this right now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YXNeOGyIXLMEsFeNE4wAmw.png" /></figure><p>Use the following command to create the Redis service.</p><pre>docker service create --name my-redis --replicas 4 redis</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zC9z80m88mWAOTvvU4MRVA.png" /></figure><p>To inspect and confirm the 4 redis services have been created, use the following command.</p><pre>docker service ps &lt;service_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2OpGY_lE4Org-K4tU5VRVw.png" /></figure><h3>Create an Apache Service with Ten Replicas</h3><p>Next up is Apache. You can search on hub.docker for it or click <a href="http://Create an Apache Service with Ten Replicas">here</a>. I am including this so that you understand that it is very easy to navigate and find the images that you need. At this point however we do not need any other info from the image on the website.</p><pre>docker service create --name my_apache --replicas 10 httpd:latest</pre><p>After the service has been created, inspect it as we did for the Redis service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oa7vsMqr9wq6h166c9o8Sg.png" /></figure><h3>Create a Postgres Service with One Replica</h3><p>Postgres will be the last service that we are going to create today using a single replica. Because we are only creating one replica, we do not need --replica in the command. Go <a href="https://hub.docker.com/_/postgres">here</a> for the official documentation on the Postgres image.</p><p>Postgres has a special environment variable requirement of POSTGRES_PASSWORD=&lt;password&gt; The password can be anything that you choose. To practice docker-compose we are going to create a YAML file.</p><p>In the terminal of your Swarm manager, use mkdir to create a new directory. Create a new file by using touch labeled <em>docker_compose.yml .</em></p><p>Enter vim usingvim docker-compose.yml . Then hit i for insert and enter the following. Pay special attention to the spacing.</p><pre>version: &#39;3.1&#39;<br>services:<br><br>  db:<br>      image: postgres<br>      restart: always<br>      environment:<br>        POSTGRES_PASSWORD: my_password<br>~                                        </pre><p>Next we are going to deploy the service across the swarm with the following command.</p><pre>docker-compose up -d </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*stZ7oX2U9W9DQ5awr7WbIQ.png" /></figure><p>After this step is complete use the next command</p><pre>docker stack deploy -c docker-compose.yml postgres</pre><p>You should see creating service postgres_db. If you do, we are in business!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FpOVh2kTeXJxR0yVaMG5LQ.png" /></figure><p>Use the command docker service ls to inspect your services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GjLusgca88mq2gsnI_1sDQ.png" /></figure><p>As you can see from mine, I had a few issues getting everything to work at first. I believe that my initial name was too long or had a something that was not acceptable. I changed the name, then shortened it, and after that it worked. If you have issues or something is does not get created, use the `docker service rm &lt;service-id&gt;*** command to delete them.</p><p>Use docker service ps &lt;service name&gt;to list the replicas (tasks) that are associated with a service in Docker swarm mode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KraC_br17h7t5KH-j6v-qQ.png" /></figure><p>Run the same command for your apache and Postgres services.</p><p>Before you go, use the docker swarm leave command to leave the swarm, then stop or delete the EC2 instances that we started for this project.</p><p><strong>Review/conclusion</strong></p><p>In this tutorial, using AWS we started a Docker Swarm with a single manager and three worker nodes. We then created a highly available container service with three separate services and different numbers of replicas for each service.</p><p>Thank you for continuing to follow me on my DevOps journey. I hope to see you again soon!</p><p>Connect with me on LinkedIn here. <a href="http://www.linkedin.com/in/">www.linkedin.com/in/abachmandevops</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ad1988366e4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Jenkins server on AWS EC2 with Terraform]]></title>
            <link>https://medium.com/@abach06/diving-head-first-into-terraform-c365dcde9a59?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c365dcde9a59</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[ec2-instance]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Thu, 20 Jul 2023 02:37:12 GMT</pubDate>
            <atom:updated>2023-07-20T08:44:20.206Z</atom:updated>
            <content:encoded><![CDATA[<p>Using Terraform to deploy an EC2 instance with a bootstrapped Jenkins server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_OVK_Y29Ryif37iJNdrfjg.png" /><figcaption>We will not be using Terraform Cloud in this tutorial</figcaption></figure><p>Continuing my journey with Level Up in Tech, today we will be learning and practicing with Terraform and Jenkins. I had a great time just starting Terraform, and I am excited to continue learning with this great technology!</p><p>What is Terraform? Terraform is an extremely useful and fun open-source infrastructure as code (IaC) tool developed by HashiCorp. It allows you to define and provision infrastructure resources across various cloud providers, such as Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP), and more, as well as on-premises infrastructure.</p><p>Click <a href="https://developer.hashicorp.com/terraform/intro">here</a> to read more about Terraform and its benefits.</p><p>Jenkins is an open-source automation server that facilitates continuous integration (CI) and continuous delivery (CD) practices in software development. It provides a platform for automating various stages of the software delivery pipeline, such as building, testing, and deploying applications.</p><p><strong>Todays objective:</strong></p><p>Your team would like to start using Jenkins as their CI/CD tool to create pipelines for DevOps projects. They need you to create the Jenkins server using Terraform so that it can be used in other environments and so that changes to the environment are better tracked.</p><ul><li>Use Terraform to deploy an EC2 instance in your default VPC</li><li>Bootstrap the EC2 instance with a script that installs and starts Jenkins upon instance creation</li><li>Create a security group that allows traffic on port 22 from your IP and all traffic from port 8080</li><li>Create a private S3 bucket to be used for Jenkins artifacts</li><li>Verify that you can reach Jenkins via port 8080 in your web browser</li></ul><p><strong>Prerequisites:</strong></p><ul><li>AWS access and CLI access</li><li>IDE of your choosing (I am using Cloud9)</li><li>Terraform installed on your local machine (it comes pre-installed on Cloud9)</li><li>Basic Terraform knowledge</li></ul><p><strong>Setting up</strong></p><p>In your IDE, make sure that you have <a href="https://developer.hashicorp.com/terraform/downloads?ajs_aid=75d6ce5b-d6c9-43a2-8006-daa0c49efa6a&amp;product_intent=terraform">Terraform</a> installed and working correctly.</p><p>Check out my code sources on <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs">AWS provider docs</a> and spend some time looking around — this is a fantastic resource!</p><p><strong><em>Configure AWS Provider with Access Key Credentials:</em></strong></p><p>In order for Terraform to interact with your AWS account, you will first need to supply your account credentials. Find your access key and secret access key, and then configure it using the following commands.</p><p>For information on how to create new keys click <a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html">here</a>.</p><pre>export AWS_ACCESS_KEY_ID=&quot;&lt;access_key&gt;&quot;<br>export AWS_SECRET_ACCESS_KEY=&quot;&lt;secret_key&gt;&quot;</pre><p><strong><em>Create a new directory and a main.tf file:</em></strong></p><p>Create a new directory using mkdir Terraform_project, then change into your new directory. Create your file using touch main.tf.</p><p>The ‘main.tf’ file is the configuration file where you define the desired infrastructure state by declaring resources, their properties, and the relationships between them. Terraform builds all the desired infrastructure from what is declared in this file. Multiple configuration files can be used, but today we are only using a single configuration file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2dzWXGRsX6SJdzoKqBGyBA.png" /></figure><p>Once you have created your main.tf file, open it.</p><p>You need to already have a default VPC with an associated default subnet in your AWS account. If you do not have these, go create them now.</p><p><strong><em>Use Terraform to launch an EC2 instance:</em></strong></p><p>In the main.tf file we created and opened, we are going to copy and paste code from the <a href="https://registry.terraform.io/">Official Terraform Registry</a> and the <a href="https://developer.hashicorp.com/terraform/docs">Official Terraform Documentation</a> pages. These pages contain everything that is needed for this project and will include the provider and resources that we need to put into our configuration file.</p><p><strong><em>Assign a provider:</em></strong></p><p>Go to the <a href="https://registry.terraform.io/">Official Terraform Registry</a> for a list of the available providers. You can copy and paste the code directly from the provider page with a specific version, or as you see in the code below where I declare my provider, specify the latest version. My region is us-east-1; use whatever region that is closest to you.</p><pre>#Configure the AWS Provider in Terraform <br>provider &quot;aws&quot; {<br>  region = &quot;us-east-1&quot;<br>}</pre><p>If you want to specify a specific version or a different provider, you could enter it like this. The code below has the latest version.</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source = &quot;hashicorp/aws&quot;<br>      version = &quot;5.1.0&quot;<br>    }<br>  }<br>}</pre><p><strong><em>Define a resource block for your default VPC:</em></strong></p><pre># Reference the default VPC for Jenkins server<br>resource &quot;aws_subnet&quot; &quot;jenkins_subnet&quot; {<br>  vpc_id     = &quot;vpc-0161ee3c07b7719d5&quot; # Replace with your default VPC ID<br>  cidr_block = &quot;10.0.0.0/16&quot;           # Adjust the CIDR block if needed<br>}</pre><p><strong><em>Define a resource for an EC2 instance:</em></strong></p><p>In another resource block, we will define the EC2 instance with the desired AMI, name, size, and type that is required.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RktHZFXTcdFbUc-xY9PmxQ.png" /></figure><p>The code above has the vpc_security_goup_ids already in it. I have not gone over this yet. If you get your resource block from the Terraform website it won’t have this in it either. This assigns the security group that we are going to create down the road to the EC2 instance that we are creating. I have left it in because I had some trouble with previous iterations of this code, and I forgot to get a screenshot of it before.</p><p>To find a specific AMI, navigate to EC2 in the AWS console, and then to AMI catalogue. Find the AMI that you prefer. I will be using the Amazon Linux 2 AMI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nxjrCf2NHbt8cTwimwsRoA.png" /></figure><p><strong>Bootstrap the EC2 Instance to Automatically Install and Launch Jenkins during Startup:</strong></p><p>Our team wants to uses Jenkins as their CI/CD tool to create pipelines for DevOps projects, so we will need to have a cloud server that is running Jenkins. We will create a script that will install and start Jenkins on our EC2 instance automatically. Look at the official Jenkins documentation <a href="https://www.jenkins.io/doc/tutorials/tutorial-for-installing-jenkins-on-AWS/">here</a>. The documentation shows how to install and configure Jenkins.</p><p>In the script below I have described each step, but in practice I was not able to get this multi-line string to work.</p><pre>  # Specify a custom script that Terraform executes when launching the EC2 instance<br>  user_data = &lt;&lt;-EOF<br>#!/bin/bash<br><br># I&#39;m customizing this EC2 instance for my Jenkins server<br><br># Update the system packages<br>sudo yum update -y<br><br># Add Jenkins repository key<br>sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo<br>sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key<br><br># Upgrade all installed packages<br>sudo yum upgrade -y<br><br># Install Java Development Kit (JDK)<br>sudo amazon-linux-extras install java-openjdk11 -y<br><br># Install Jenkins<br>sudo yum install -y jenkins<br><br># Enable Jenkins service to start on boot<br>sudo systemctl enable jenkins<br><br># Start Jenkins as a service<br>sudo systemctl start jenkins<br><br># Retrieve the initial Jenkins admin password (needed to access the Jenkins web interface)<br>sudo cat /var/lib/jenkins/secrets/initialAdminPassword<br><br>EOF<br>  # Ensure that the user_data script is replaced on every change<br>  user_data_replace_on_change = true<br>}<br><br></pre><p>It should look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tNx_d9k6E7f-QaYY1aOkDg.png" /></figure><p>In Terraform, &lt;&lt;-EOF at the beginning and EOF at the end are used to define a heredoc, which stands for &quot;here document.&quot; A heredoc is a way to define multi-line strings in a more readable and convenient manner. The user_data_replace_on_change attribute, when set to true, indicates that the user_data property of a resource (in this case an EC2 instance) should be replaced whenever changes are detected.</p><p>After I inserted the script in my main.tf file, I ran the command terraform init to initialize Terrraform, and then I ran the command terrraform fmt to format my code. I missed the EOF at the end the first time around and also missed the closing marker and received an error. Terraform is pretty good at showing you which lines of code need attention. As you progress with it, you will experience this a lot! After this I also rearranged some of my code and it now works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ki_L2qRi-f_hVTyFfEh0Lg.png" /></figure><p>Note: the screenshot above is also in a slightly different order. I had an issue with this version and it would not work. Use the original script for your project.</p><p><strong>Create a security group:</strong></p><p>AWS by default does not allow any traffic in or out of an newly created EC2 instance. Here we will need to create or modify an existing security group that will allow traffic on port 22 from your IP and allows all traffic from port 8080. To allow traffic on these ports from any IP address, I will also be including port 443 with a CIDR block of 0.0.0.0/0. We will include this in our code, and Terraform will create it for us! Before you push your code to github or other source control, make sure that you delete your private IP address.</p><pre># My AWS security group to control access to my Jenkins server<br>resource &quot;aws_security_group&quot; &quot;my_jenkins_sg&quot; {<br>  name        = &quot;my-jenkins-security-group&quot;<br>  description = &quot;Access to ports 22, 443, and 8080 for Jenkins&quot;<br><br>  # Allow incoming SSH (port 22) from anywhere<br>  ingress {<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    description = &quot;Incoming SSH (port 22) from anywhere&quot;<br>  }<br><br>  # Allow incoming HTTP (port 8080) from anywhere<br>  ingress {<br>    from_port   = 8080<br>    to_port     = 8080<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    description = &quot;Incoming HTTP (port 8080) from anywhere&quot;<br>  }<br><br>  # Allow incoming HTTPS (port 443) from anywhere<br>  ingress {<br>    from_port   = 443<br>    to_port     = 443<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    description = &quot;Incoming HTTPS (port 443) from anywhere&quot;<br>  }<br><br>  # Allow all outgoing traffic to anywhere<br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    description = &quot;Allow all outgoing traffic to anywhere&quot;<br>  }<br><br>  # Adding more tags to make the security group easily identifiable<br>  tags = {<br>    Name        = &quot;My-Jenkins-Security-Group&quot;<br>    Environment = &quot;Development&quot;<br>    Owner       = &quot;Aaron Bachman&quot;<br>    Project     = &quot;My Jenkins Project&quot;<br>  }<br>}</pre><p>Add all of this to your main.tf file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z5_ULk4HMhNwBn5re9BhYw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3NmyRiBaI3cDpxRa9ECDzQ.png" /></figure><p>If your syntax does not look like this or if you just to check, you can run terraform fmt to format. You could wait to run this until later, but I have found that running it after every step helps keep the number of errors lower and easier to fix.</p><p><strong><em>Assign the security group:</em></strong></p><p>We will now assign the security group to the EC2 instance that Terraform will create.</p><p>The code below uses the expression; an expression in Terraform is anything that returns a value. In this case, we will use an expression as a reference to the name security group in our main.tf file. It won’t have an ID until it has been created. This can also be accomplished with a variable, which I will cover further down the road.</p><p>Use the syntax below to define an attribute reference.</p><p>&lt;PROVIDER&gt;_&lt;TYPE&gt;.&lt;NAME&gt;.&lt;ATTRIBUTE&gt;</p><p>The code below uses the syntax I just described.</p><pre>vpc_security_group_ids = [aws_security_group.my_jenkins_sg.id]</pre><p>To break this down for clarification, the vac_security_group is the provider (AWS), type (security group), jenkins_sg is the name that I gave to my security group, and id is the attribute that is exported by the security group.</p><p>Add the code above to your EC2 instance resource block in your main.tf file. Below you see it in line 11.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C9e5bedQV3LGacIOzTRVSg.png" /></figure><p><strong><em>Create a private S3 bucket:</em></strong></p><p>Next we will be creating an S3 bucket to store Jenkins artifacts. These artifacts are files that are generated as a result of a build or a job execution. Artifacts can include compiled binaries, build outputs, test reports, log files, documentation, or any other files that are produced during the build or deployment process.</p><p>Jenkins allows you to archive and store these artifacts so that they can be accessed and used for further analysis, deployment, or distribution. Artifacts are typically stored on the Jenkins server or on an external artifact repository; in this case it’s an S3 bucket.</p><p>Every S3 bucket has to have a globally unique name. We can use a random number generator, which I had a good time learning how to do, or we can use a variable. By using variables, you can set the bucket name based on different environments, project names, or any other configuration. The value of var.jenkins_s3_bucket_name would be defined in your Terraform configuration, either by assigning it a static value or by reading it from input variables or other sources. For this exercise, however, I will use a random number generator in my main.tf file.</p><p>The code below will create a globally unique bucket and deny access to the public</p><pre># S3 Bucket to store artifacts for my Jenkins server<br>resource &quot;aws_s3_bucket&quot; &quot;jenkins&quot; {<br>  bucket = &quot;jenkins-${random_id.randomness.hex}&quot;<br><br>  # Adding more tags to make the S3 bucket easily identifiable<br>  tags = {<br>    Name        = &quot;My-Jenkins-Artifacts&quot;<br>    Environment = &quot;Development&quot;<br>    Owner       = &quot;Aaron Bachman&quot;<br>    Project     = &quot;My Jenkins Project&quot;<br>  }<br>}<br><br># Deny public access to the S3 bucket<br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;jenkins&quot; {<br>  bucket = aws_s3_bucket.jenkins.id<br><br>  rule {<br>    object_ownership = &quot;BucketOwnerPreferred&quot;<br>  }<br>}<br><br># Create a random number to ensure a unique S3 bucket name<br>resource &quot;random_id&quot; &quot;randomness&quot; {<br>  byte_length = 12<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oyYJ-OfyBO0smFdeUZMXWA.png" /></figure><p>The third block of code will generate a random value of 12 bytes. You can set this number to anything that you want. Keep in mind that the value expressed in hexadecimal format is double what you set the number to.</p><p>The bucket name is set to “jenkins-” followed by the value of random_id.randomness.hex. This code creates an S3 bucket with a dynamically generated name using the random identifier and applies a tag to it.</p><p>After you add it to your main.tf file, save it and check for errors using terraform fmt again.</p><p>You can also download my code directly from GitHub: <a href="https://github.com/abach06/Week_20_project.git">https://github.com/abach06/Week_20_project.git</a></p><p><strong>Running Terraform/Deploy resources:</strong></p><p>To deploy our resources, the Terraform cycle uses the following commands to create, update, and delete the infrastructure resources that are created. This is where Terraform really shines in my opinion.</p><ul><li>terraform init — this downloads the plugins needed to execute config files (does note modify or apply state or config files)</li><li>terraform fmt — formats the spacing to the correct HCL format</li><li>terraform validate — validates if the file syntax is valid</li><li>terraform apply — provisions the resource infrastructure as defined in the Terraform file</li><li>terraform destroy — deletes all resources that were created with the terraform apply command (be careful with this one)</li></ul><p>To start this process, make sure that in the CLI you are in the directory with your Terraform files.</p><p>Start the process with terraform init, then terraform fmt. This should fix any spacing, but if you have errors take care of those. Once everything here is situated, the return should look like the following.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UdJcUkk8ywLa68HO5HF05g.png" /></figure><p>Next use terrraform validate to see if the syntax of the file is valid. I ended up having several issues with syntax. After I fixed these minor issues this part worked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wo0ofEtv95fUc0eJDL_-nQ.png" /></figure><p>Next terraform plan will show you the plan for all of the resources that Terraform will create for you. Take the time and scroll through the resources above the “Plan: 7 to add” line. You want to make sure that the correct resources are being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*545iJ7STyF7f72MIFA7GFw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s-QAuuvyBE1eMMRnExhUdQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LRHjTkhHNKdl2g9LKqJkgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NO_uhf5-bbJ-Ej2CjAhVbw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XDQn1gX_5GXS1js9DPJHxQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FQbz7ed6f_MzzqJDkomsNw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*44C0Li0DiYmqDwWkHb1RlA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BTYieTUhfyDHdK7Fct0XVw.png" /></figure><p>This is a lot to look at especially at first, but as you spend more time looking you will be able to see exactly what is being created, and further down the line, what you will add or destroy from existing deployments.</p><p>If the plan looks good, use terraform apply to start creating your resources. Both the plan and apply commands will ask you for a value. If you want to proceed, type yes, then hit return/enter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Bz9ayVMHeZqBXA00wmbog.png" /></figure><p><strong>Terraform apply:</strong></p><p>Once you get Terraform to start creating your resources, this can take a while. In my case, it was about 10 minutes. The output when this is done is fairly long, too long to post here, but I would suggest taking a few minutes and reading it and seeing what exactly was created. The screenshot below shows the resources that are being created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jN0k9CXkSdVV9yQhZb_TFw.png" /></figure><p>I had an issue with my subnet, I had the incorrect CIDR, and VPC ID as it turns out. I used the following commands in the AWS CLI to see my VPC.</p><pre>aws ec2 describe-vpcs<br>aws ec2 describe-subnets --filters &quot;Name=vpc-id,Values=vpc-01f17772c79b30475&quot; --query &quot;Subnets[*].{SubnetId:SubnetId, CidrBlock:CidrBlock}&quot;</pre><p>After inserting the correct ID and CIDR the rest of my resources were created after running terraform apply again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*elKvRzirNtO5b8HrqYBgwg.png" /></figure><p>Once you’ve executed terraform apply, you can use the command terraform show to inspect the current state of your Terraform resources or the execution plan. The output of this command reflects the actual state of your infrastructure, giving you a comprehensive overview of the resources provisioned by Terraform. It serves as a useful verification tool to ensure that everything you intended to create has been successfully deployed.</p><p>To get a visual representation of the resources in your AWS environment, you can navigate to the AWS Management Console. There, you’ll be able to see the infrastructure elements Terraform has orchestrated for you. As an example, you can find the my_jenkins_sg security group that Terraform has diligently created to control access to your Jenkins server.</p><p>By combining the terraform show command&#39;s insights with a visual examination in the AWS console, you gain a clear understanding of your cloud environment&#39;s current state, which is invaluable for managing and maintaining your infrastructure effectively.&quot;</p><p>Below you can see my_jenkins_sg that Terraform created for me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fNhq1h9a51PUEK-2suLEDw.png" /></figure><p>When you look at your security group, verify that the proper ports are open, so we are good to go.</p><p>If we look at the S3 bucket that was created, we see that the randomly generated value was applied to the name to create a unique name and that the public does not have access to the bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y7T-12BFPdOAhGhcp-xpVg.png" /></figure><p><strong>Verify connectivity to Jenkins server via port 8080:</strong></p><p>In AWS management console, connect to your instance via EC2 instance connect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H9Z8ExNK9W_x4WJTr-DF9A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GAjA25vKtI1H-P5spkZpw.png" /></figure><p>Jenkins should already have been installed and started because of the script that we bootstrapped to our EC2 instance. To check this, use the following command.</p><pre>systemctl status jenkins</pre><p>You should see the following result, showing that Jenkins has been installed and is running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qKs8NKAHDiHTIT-1YfHSAA.png" /></figure><p>We can also check this in the CLI. In Cloud9 use the following command with the public ip of your instance.</p><pre>curl http://&lt;instance_ip&gt;:8080</pre><p>This should return some HTML.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ocdDG696Tj76BH4Ifsbh9g.png" /></figure><p>To get the landing page for Jenkins, type the following command into your internet browser with your public ip address.</p><pre>http://&lt;instance_ip&gt;:8080</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TGQ8GZz0GmK7eUckoy6pDA.png" /></figure><p>If you don’t see this, double check to make sure your browser didn’t redirect you to the HTTPS site. If you do see this: congratulations, your Jenkins server is up and running!</p><p><strong>Clean up:</strong></p><p>Clean up in with Terraform is very easy. The following command will stop and delete all of the resources that we created in this project. This is crucial for cost optimization and workflow, and no manual cleanup! When you run this command, Terraform will ask for a response. Type yes to destroy.</p><pre>terraform destroy</pre><p>Thank you very much for joining me on my journey, and I hope to see you as I continue my learning process!</p><p>To view or use my code for this project use this link: <a href="https://github.com/abach06/Week_20_project.git">https://github.com/abach06/Week_20_project.git</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c365dcde9a59" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to build Docker containers with Python and Boto3]]></title>
            <link>https://medium.com/@abach06/how-to-build-docker-containers-with-python-and-boto3-3b559bbae308?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b559bbae308</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker-image]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Wed, 10 May 2023 17:18:40 GMT</pubDate>
            <atom:updated>2023-05-10T17:18:40.692Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PjcUchhhJkajcR8QAzjbfQ.png" /></figure><p>Welcome back, fellow techies. Today we will be working with Docker containers, Python, Boto3, and bind mounts. Let’s start by going over the details for today’s project.</p><p><strong>Docker: </strong>Docker<strong> </strong>is an open-source containerization platform that allows developers to package, deploy, and run applications in containers. Docker has many benefits. It allows for portability, scalability, isolation, and versioning. Docker allows users to package an application with all of its dependencies and libraries into a standardized unit. Please look through the Docker references <a href="https://docker-curriculum.com/">here</a>.</p><p><strong>Containerization: </strong>Containerization<strong> </strong>is a technique for packaging and deploying software applications in a portable and isolated manner, so that they can run consistently across different computing environments, such as a developer’s laptop, a test server, or a production cluster.</p><p>A container is a lightweight and standalone executable unit that contains all the dependencies, libraries, and configuration files needed to run the application, without relying on the underlying operating system or hardware. Each container runs as an isolated process with its own file system, networking, and resource allocation. Click for the <a href="https://docs.docker.com/get-started/overview/">references</a>.</p><p><strong>Docker hub: </strong><a href="https://hub.docker.com/">Docker Hub</a> is a cloud-based repository service for storing, sharing, and managing Docker container images. It allows developers to build, store, and distribute Docker images with ease. Docker Hub also provides a registry for finding and downloading pre-built images, which can be used as the basis for building new containers.</p><p><strong>Bind mount: </strong>Bind mount is a mechanism for mounting a file or directory from the host file system into a container. Bind mounts can be useful for a wide range of tasks, such as sharing configuration files, logs, or application data between the host and container. Any changes that you make to the file outside of the container is reflected inside the container. Click here for the <a href="https://blog.logrocket.com/docker-volumes-vs-bind-mounts/">references</a>.</p><p><strong>Scenario: </strong>Say a Python development team wants to simplify their environment setup and ensure consistent code execution across multiple machines. They can achieve this by creating a Dockerfile for Boto3/Python using either the Ubuntu or Python images from Dockerhub. The Dockerfile can be customized with necessary libraries or dependencies for their Python app.</p><p>Next, the team can download three repositories to their local host and create three Ubuntu containers using the customized Dockerfile. Each container can be configured with a bind mount to one of the repo directories, enabling easy code access and changes.</p><p>Finally, team members can verify access to each repo directory by logging into each container, ensuring proper configuration and effective collaboration. Using Docker allows for a portable development environment that can be easily replicated on different machines, improving workflow and collaboration efficiency.</p><p><strong>Prerequisites</strong></p><ol><li>Your IDE of choice. I am using cloud9.</li><li>The Docker engine installed in your environment.</li><li>A basic understanding of Docker commands.</li><li>Knowledge of git and github.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5CucEKqrqLlEVMLOHcHdPA.png" /><figcaption><a href="https://thehftguy.com/2017/02/23/docker-in-production-an-update/">Credit</a></figcaption></figure><p>First up, let’s download Docker to your IDE. If you already have Docker installed, you can skip this step.</p><p>Use the following commands one at a time to install.</p><pre>sudo yum update -y<br>sudo yum install docker -y<br>curl -sfL https://get.k3s.io | sh -</pre><p>After that step is complete, let’s check what version you have installed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Or7OMJQBXZ03VYSKtXXOHA.png" /></figure><p><strong>Build Dockerfile &amp; Image</strong></p><p>Now the real work begins. We are going to create a directory and Dockerfile that we will use for this project. Follow below to create your directory and file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JOoAA3eyuLlB8Rp4CzDfeg.png" /></figure><p>Make sure when you create your Dockerfile that you spell it as shown. <strong>Docker </strong>expects the name of the file to be <strong>Dockerfile </strong>which is case sensitive by default. If you spell or name it differently you would get an error when running the Docker build command.</p><p>Next we will add to our file using a file editor. I am using Vim, but you are free to use whatever you are comfortable with.</p><p>Insert the code below with the correct name for your directory into Vim.</p><pre># Update package list and install required dependencies<br>RUN apt-get update -y &amp;&amp; apt-get install -y \<br>    python3-pip<br><br># Install boto3 using pip<br>RUN pip3 install boto3<br><br># Set the working directory to /Week16ProjectDir<br># Set the working directory to /Week16ProjectDir<br>WORKDIR /wk16_Docker<br><br># Set the default command to tail the /dev/null file<br>ENTRYPOINT [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]</pre><p>To build our image we are going to use the <em>docker build </em>command.</p><pre>docker build -t &lt;name of image&gt; .</pre><p>This command builds a Docker image from a Dockerfile; the-t option used here tags the image with a name and an optional tag. At the end the . is used to specify that the Dockerfile is located in the current directory. The build process may take a minute or two. Below you can see a few of the steps that it is going through.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pn_Ej7BpoyU108i7FrS4Nw.png" /></figure><p>After it is done building, use the command docker images to see your images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SEl7pp8ZN6oKFuua9WO_9g.png" /></figure><p>If you want a specific image, you can search Dockerhub for it and pull it using the same method. If for instance you wanted to use alpine because it is lightweight, search <a href="https://hub.docker.com/">https://hub.docker.com/</a> for alpine:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M3_ju9NgB9WIbk_uhLhjYg.png" /></figure><p>To pull this image, you can use the command shown to the right of the image: docker pull alpine Run this command and check your images afterwards to see it listed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KyHgz3zZPK581j4tmaU9uA.png" /></figure><p>You can use this method for any image that is needed.</p><p><strong>Download 3 Github repo’s to local host</strong></p><p>Go to Github and choose 3 repositories you want to clone for this project. In the repo you’re interested in, locate and click the green code; the URL will show.</p><p>Use the following command to clone your repos.</p><pre>git clone &lt;Github_Repo_URL&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PGiqiNeqlHgOj3Mh3HYduw.png" /></figure><p>ls -l will show these new repos.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VpHgT5X4-6g5ldeEFnqUsQ.png" /></figure><p><strong>Create 3 Ubuntu containers</strong></p><p>Ubuntu is publicly available on Dockerhub so you do not need a log in to do this; however, if you want a specific image that is not publicly available, you need to log in using the cli.</p><p>Use the command docker login. You will be prompted for your username and password.</p><p>To create our containers we are going to use the command below.</p><pre>docker run -d -t --name &lt;containerName&gt; -v &quot;$(pwd)&quot;&lt;repositoryName&gt;:/&lt;directory_name&gt; &lt;image_name&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zXdDzSjKfOL3X0J3-I2DOA.png" /></figure><p>The -d in the command above tells Docker to run the container in detached mode, which means it runs in the background and does not attach to the console. The -t allocates a pseudo-TTY to the container, allowing you to interact with it. The -v “$(pwd)”Docker:/DockerProject mounts a directory from the host machine into the container. <strong>Docker</strong> is the name of the directory on the host machine that we are mounting, and <strong>:/wk16_Docker</strong> is the path where it is mounted inside the container. <strong>Ubuntu</strong> on the end is the name of the image to run in the container.</p><p>To see the container we just created, use thedocker ps command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TAnmhrWLBTqX_u9gA2IlmA.png" /></figure><p><strong>Inspect containers for bind mounts</strong></p><p>A bind mount in Docker is a way to mount a directory from the host machine into a Docker container. With a bind mount, the files and directories in the specified path on the host machine are made available to the container at a specified path inside the container.</p><p>Once the bind mount is created, any changes made to the files in the /data directory on the host machine will be immediately visible inside the container at the /app/data path.</p><p>Use the command below.</p><pre>docker container inspect &lt;Container_Name&gt;</pre><p>Container 1</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CPJpKSFpxTgvKJ3XXtFmtQ.png" /></figure><p>Container 2</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTWhLpXDwxX-7vN30AXpwg.png" /></figure><p>Container 3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jPaOvFvcccqDD5aUJWA3PA.png" /></figure><p>Next up we need to verify access to each of the containers. To complete this, use the following command.</p><pre>docker exec -it &lt;container_name_or_id&gt; bash</pre><p>Make sure to exit each container before trying to access the next container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P39hltJn_H4DXfCzfc-7-A.png" /></figure><p>In the command above we are utilizing docker exec ; this tells Docker to execute a command in a running container. The -it flag in Docker is used to run a container in interactive mode, with a pseudo-TTY (terminal) attached to it. Container_name or id is the container you wish to enter, and the bash command at the end of the docker exec command is used to start a new shell session inside the container.</p><p><strong>Stop and remove the containers</strong></p><p>To clean up our mess, we can either stop the containers first or force remove the containers. I’ll use the command docker container stop to stop my containers. Then I will use the docker container prune command; this command removes all of the stopped containers. Docker will prompt you to confirm the action, as it is a destructive operation that will permanently delete the stopped containers and their metadata. This will free up some space.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0fYb1Qw_WE-XrXW0u1OqWQ.png" /></figure><p>Well done everyone! In this project we built a docker file, created 3 containers, inspected each container, logged into each container, and then stopped and removed the containers! I learned a few things and I hope that you did and enjoyed the process.</p><p>I hope to see you as I continue on my journey in DevOps.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b559bbae308" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Python and Boto3 to create and manage a DynamoDB table.]]></title>
            <link>https://medium.com/@abach06/using-python-and-boto3-to-create-and-manage-a-dynamodb-table-dcb89189c2df?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/dcb89189c2df</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Wed, 19 Apr 2023 14:06:01 GMT</pubDate>
            <atom:updated>2023-04-19T14:06:01.614Z</atom:updated>
            <content:encoded><![CDATA[<p>A “ how to” for creating tables and putting items, along with how to scan and delete DynamoDB tables in Amazon Web Services (AWS) using Python and Boto3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8-17LsvlzoX0lytDChxhBw.png" /></figure><p>In this project, we will be using Python and the Boto3 module to work with DynamoDB. If you are just learning Python or are interested in the cloud, this fun project is for you!</p><p><strong>What is Python?</strong></p><p>Python is a very popular high-level programming language used for web development, data analysis, scientific computing, and AI. It is known for its simplicity, readability, and versatility. This ease of use makes it a popular language for both beginners and experienced programmers.</p><p><strong>What is DynamoDB?</strong></p><p>Amazon DynamoDB is a fully managed NoSQL database service provided by AWS. It is designed to provide low-latency, high-performance, and scalable database storage for web applications and other types of applications that require fast and flexible access to data. DynamoDB is a key-value and document database that allows you to store and retrieve any amount of data, and it automatically scales to accommodate any level of traffic. It also supports ACID transactions and provides built-in security features. DynamoDB is often used in combination with other AWS services to build highly scalable and available applications.</p><p><strong>Project Tasks:</strong></p><ul><li>Create a DynamoDB table</li><li>Use Boto3 and Python to add 10 or more items to the table</li><li>Use Boto3 and Python to scan the DynamoDB table</li><li>Query the table and remove an item</li><li>Delete the table</li></ul><p><strong>Prerequisites:</strong></p><ul><li>An AWS account</li><li>An IDE you are familiar with (I will be using Cloud 9)</li><li>General Python and AWS knowledge</li><li>Review of Boto3 documentation <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">here</a></li></ul><p>All right, now that we know the outline and what we need to complete this project, let’s get to it!</p><p><strong>Step 1: Create a DynamoDB table</strong></p><p>We are going to start by creating a DynamoDB table using Python. Below is my code that I used to create my table, and you can use it as well!</p><p><a href="https://github.com/abach06/Gold/blob/48c2814b7dbb3d3d204d95f2ee5b1d3a029c75be/Week14project.py">Gold/Week14project.py at 48c2814b7dbb3d3d204d95f2ee5b1d3a029c75be · abach06/Gold</a></p><p><strong>Note: I ran into an issue with my partition and sort key: make sure you use lowercase letters! I also had a problem with my table name; if yours does not work, look up the naming rules.</strong></p><p>After you run your code in your IDE, go to DynamoDB in the AWS console and view your new table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YKQpQ5Zs9P7ya62ptOrFqA.png" /></figure><p>Seeing that your table has been successfully created is always a good sign!</p><p><strong>Step 2: Add 10 items to the table using Python</strong></p><p>Here we are going to write some code that will add items to the table that we just created. We will use Python and the Boto3 module to add 10 items to our DynamoDB table.</p><p>Check out the Boto3 documentation for batch writing <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#batch-writing">here</a>.</p><p>The list I have created is by no means comprehensive, and it is not in order of rating or anything, just some of my favorite sources of inspiration and wonder as a guitarist. You can add single items or a batch (10 items). That helped me out in this case; there are too many great guitar albums!</p><pre>import boto3 # importing boto3 module<br><br><br><br>dynamodb = boto3.resource(&#39;dynamodb&#39;,<br>    aws_access_key_id=&#39;AKIAQUVMAJA3M2HTIREW&#39;, #insert your aws_access_key_id<br>    aws_secret_access_key=&#39;4wpwr2jO+9817HNdDvTKNznkaHEDiSAHZSuhigfd&#39;) #in<br><br><br>table = dynamodb.Table(&#39;great_guitar_albums&#39;) #variable to hold table name<br><br>with table.batch_writer() as batch: #batch writing 10 items<br>    <br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;The Essential Stevie Ray Vaughan and Double Trouble &#39;,<br>            &#39;artist&#39;: &#39;Stevie Ray Vaughan and Double Trouble&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Led Zeppelin 4&#39;,<br>            &#39;artist&#39;: &#39;Led Zeppelin&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Are You Experienced&#39;,<br>            &#39;artist&#39;: &#39;Jimi Hendrix&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Dark Side Of The Moon&#39;,<br>            &#39;artist&#39;: &#39;Pink Floyd&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Appetite for Destruction&#39;,<br>            &#39;artist&#39;: &#39;Guns n Roses&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Lucille&#39;,<br>            &#39;artist&#39;: &#39;BB King&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Back in Black&#39;,<br>            &#39;artist&#39;: &#39;AC/DC&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Revolver&#39;,<br>            &#39;artist&#39;: &#39;The Beatles&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Nevermind&#39;,<br>            &#39;artist&#39;: &#39;Nirvana&#39;<br>            }<br>        )<br>    batch.put_item(<br>        Item={<br>            &#39;album&#39;: &#39;Van Halen&#39;,<br>            &#39;artist&#39;: &#39;Van Halen&#39;<br>            }<br>        )</pre><p>After you successfully run your code, let’s head back to the AWS console to check on our table.</p><p>Click on the yellow “Explore table items” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1bPBLGZV6D9d7uJWN5dYeA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9M9a_jbHT36QDFzht9hQDg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XnlpFUcbwcuB_LBv3VnoCA.png" /></figure><p>We have successfully added items to our DynamoDB table. Great job!</p><p><strong>Step 3: Scan the table</strong></p><p>We are now going to write some code that will scan our table for us. This will show us the items in the table. This took me a while. Here is what I came up with.</p><p>In the screenshot below you can see the result of the scan.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FELqm8q5qFowhuwVXKf6Ug.jpeg" /></figure><p>When you perform a scan operation in DynamoDB, the scan reads all of the items in the table, which can be time-consuming and use a lot of read capacity units, especially for large tables. Therefore, scans are generally not recommended for tables with a lot of data, unless you need to read all items in the table.</p><p><strong>Step 4: Query the table</strong></p><p>In this step we will be querying for a result in the table instead of scanning. This method is faster and uses fewer resources than scanning so it should be your preferred method for retrieving items in your DynamoDB table with the same partition key.</p><p>You are welcome to use the code below.</p><pre>import boto3 <br><br>#importing key ***You will recieve error if not imported<br>from boto3.dynamodb.conditions import Key <br><br><br>dynamodb = boto3.resource(&#39;dynamodb&#39;, region_name=&#39;us-east-1&#39;)<br><br>table = dynamodb.Table(&#39;great_guitar_albums&#39;) #var for table name<br><br>response = table.query( <br>  KeyConditionExpression=Key(&#39;album&#39;).eq(&#39;Nevermind&#39;)   #query for items with &quot;album&quot;<br>)<br><br>print(response[&quot;Items&quot;])</pre><p>Below you can see the results of the query:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KwzIKLN_n1EzF3VUfHUi1w.png" /></figure><p><strong>Step 5: Remove an item from the table</strong></p><p>Moving on! We are going to remove an item from the table. I mentioned this in the beginning, but searching the documentation for what you need to do is very helpful for this sort of thing. Click <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/table/delete_item.html#">here</a> to look around some more.</p><p>Below is the script that I wrote to delete one of my items.</p><pre># Remove item from a DynamoDB table<br><br>import boto3<br><br>dynamodb = boto3.resource(&#39;dynamodb&#39;)<br>table_name = dynamodb.Table(&#39;great_guitar_albums&#39;)<br>response = table_name.delete_item(<br>    Key={<br>        &#39;album&#39;:&#39;Appetite for Destruction&#39;,<br>        &#39;artist&#39;:&#39;Guns n Roses&#39;<br>    } <br>)<br><br>print(&#39;Item successfully removed&#39;)<br>#this should work</pre><p>Below is the code deleting one of the items in the table — gotta love spelling errors!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ljpLj_16tgKadr4LRnqyog.png" /></figure><p>Go back to the DynamoDB dashboard and look at the items in your table. You should see the item is no longer there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aMnD1Q9hcIhsU9b9dO_hmw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2zJO9jufKTiU2OGvdclJAQ.png" /></figure><p>Fantastic. Once you have got this down, you can easily add, delete, query or scan items with Python.</p><p><strong>Step 6: Delete the table</strong></p><p>Now we are going to delete the table. You don’t want to use resources unnecessarily, and I didn’t include Robert Johnson in my list so we definitely need to delete mine. If you like delta blues, please check him out.</p><p>Go back and look at the documentation for Boto3 again <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb/table/delete.html#">here</a>.</p><pre><br># Delete a DynamoDB table<br><br>import boto3<br><br>dynamodb = boto3.client(&#39;dynamodb&#39;)<br><br>response = dynamodb.delete_table(<br>    TableName=&quot;great_guitar_albums&quot;,<br>)<br><br>print(&quot;Table successfully deleted&quot;)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Pf1_x-yAY0Rc-rFuqCpSw.png" /></figure><p>Go back and double check to make sure that the table has been deleted.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uKeAL_uX4bIOXgRp0ogu9Q.png" /></figure><p>Thank you for following along! I hope you enjoyed this project. There was more here than I originally thought, but I learned a lot, and I hope that you did as well.</p><p>I hope you continue to follow me as I continue on my journey in cloud computing and tech. Please follow me on <a href="http://www.linkedin.com/in/">social media</a> if you are inclined. If you need help with anything please feel free to message me directly. Have a great day!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dcb89189c2df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automate company savings using Python, AWS and Boto3 to stop EC2 instances.]]></title>
            <link>https://medium.com/@abach06/automate-company-savings-using-python-aws-and-boto3-to-stop-ec2-instances-e3adef6bc934?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3adef6bc934</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[boto3]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws-ec2]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Fri, 24 Feb 2023 07:45:57 GMT</pubDate>
            <atom:updated>2023-02-24T07:45:57.018Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*U8_0s5dGhg7jBNgCTm-7vQ.png" /></figure><p><strong>Introduction</strong></p><p>Today I will walk through the use of AWS SDK (software development kit) for Python, called Boto3, to stop EC2 instances automatically when you are not using them. AWS has a lot of documentation for the installation and use of Boto3: click <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html">here</a> to review it, since we will be using it later.</p><p><strong>Scenario</strong></p><p>Our DevOps engineering team often uses a development lab to test releases of our application. Managers are complaining about the rising cost of our development lab and need to save money by stopping our EC2 instances after all engineers are clocked out.</p><p><strong>Task</strong></p><ul><li>Launch at least 3 EC2 instances tagged ‘Dev’.</li><li>Launch 2 EC2 instances tagged ‘Prod’.</li><li>Create a Python script that will only stop instances tagged ‘Dev’.</li></ul><p><strong>Prerequisites</strong></p><ul><li>An AWS account with IAM user access.</li><li>Knowledge of AWS Management Console.</li><li>EC2 instances with ‘Dev’, and ‘Prod’ tags that are running.</li><li>Basic knowledge of Python.</li><li>An IDE (Integrated Development Environment) with Python installed on it. I will be using AWS Cloud9 for this exercise. Click <a href="https://www.python.org/">here</a> for more information on how to install Python.</li></ul><p><strong>Getting Started</strong></p><p>In the AWS EC2 management console, click the l<strong>aunch instance </strong>button.Name your instance and click “add additional tags”. Three of the instances we are creating are going to have a key/value tag of ‘<strong>Environment: Dev’. </strong>The other two will have a key/value tag of<strong> ‘Environment: Prod’.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7PeLjZh0dPNrAwKP.png" /></figure><p>For the rest of the options, select Amazon Linux 2 AMI, 64-bit (x86) architecture, t2.micro instance type, key pair, default VPC, default subnet, and default security group. Launch the instance, and repeat for the other instances that you are creating. Keep in mind that three will be tagged ‘Dev’ and two tagged ‘Prod’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fAaVsUFCcRvUVDrY.png" /></figure><p>Select an instance and go to Tags on the far right; you should see something similar to this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/0*mK_aN9mxW0Lqvodh.png" /></figure><p><strong>Create a Python script to stop all EC2 instances</strong></p><p>The first order of business is to install ‘<strong>Boto3’, </strong>which is the AWS SDK for Python. It will make it easy to integrate our Python application, library, or script with AWS services including Amazon S3, Amazon EC2, Amazon DynamoDB, and more.</p><p>Throughout the project we will refer to the documentation for Boto3 <a href="https://boto3.amazonaws.com/v1/documentation/api/latest/index.html">here</a>.</p><pre>pip install Boto3</pre><p>If you do not have AWS CLI installed, install it now. This will allow Boto3 to interact with our EC2 instances</p><pre>pip install awscli</pre><p>After installing Boto3 and the CLI, we are now ready to start our Python script.</p><p><strong>The Code</strong></p><p>In this first block of code that I created before writing this article, I accidentally terminated the instances instead of stopping them, as you can see. This is exactly why we get approval before we push the code to our main branch for production. I ran this code and my three instances tagged ‘Dev’ were completely erased.</p><pre>import boto3<br><br># Create an EC2 client<br>ec2 = boto3.client(&#39;ec2&#39;)<br><br># Get all instances with the &quot;Name&quot; tag equal to &quot;Dev&quot;<br>instances = ec2.describe_instances(Filters=[{&#39;Name&#39;: &#39;tag:Environment&#39;, &#39;Values&#39;: [&#39;Dev&#39;]}])<br><br># Extract the instance IDs from the response<br>instance_ids = [instance[&#39;InstanceId&#39;] for reservation in instances[&#39;Reservations&#39;] for instance in reservation[&#39;Instances&#39;]]<br><br># Terminate the instances<br>if instance_ids:<br>    ec2.terminate_instances(InstanceIds=instance_ids)<br>    print(f&quot;Instances {instance_ids} are being terminated.&quot;)<br>else:<br>    print(&quot;No instances found with the tag &#39;Name:Dev&#39;.&quot;)</pre><p>So I had to launch three more EC2 instances tagged ‘Dev’ and change my script so that it only stopped my instances. Mistakes happen all the time, and to all of us, so make sure to review, review, review!</p><pre>import boto3<br><br># Create an EC2 client<br>ec2 = boto3.client(&#39;ec2&#39;)<br><br># Get all instances with the &quot;Name&quot; tag equal to &quot;Dev&quot;<br>instances = ec2.describe_instances(Filters=[{&#39;Name&#39;: &#39;tag:Environment&#39;, &#39;Values&#39;: [&#39;Dev&#39;]}])<br><br># Extract the instance IDs from the response<br>instance_ids = [instance[&#39;InstanceId&#39;] for reservation in instances[&#39;Reservations&#39;] for instance in reservation[&#39;Instances&#39;]]<br><br># Stop the instances<br>if instance_ids:<br>    ec2.stop_instances(InstanceIds=instance_ids)<br>    print(f&quot;Instances {instance_ids} are being stopped.&quot;)<br>else:<br>    print(&quot;No instances found with the tag &#39;Name:Dev&#39;.&quot;)</pre><p>This time after running my code I received this output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fi48f8txaqv-x18e.png" /></figure><p>When checking to see if the correct instances had been stopped, they were!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*niUl9kAhoDKIphv1.png" /></figure><p>This next block of instances include the instances tagged ‘Prod’ that were not stopped and the instance that Cloud9 is currently running on.</p><p>We have successfully created a Python script using Boto3 that will stop instances in this case tagged ‘Dev’. Managing cloud environments with Python and Boto3 can save companies money and time.</p><p>I hope you enjoyed today’s article.</p><p>Please follow me for more cloud/DevOps content in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3adef6bc934" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using Python to automate file information extraction]]></title>
            <link>https://medium.com/@abach06/using-python-to-automate-file-information-extraction-2200ed53e1b6?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/2200ed53e1b6</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Tue, 21 Feb 2023 13:08:21 GMT</pubDate>
            <atom:updated>2023-02-21T13:08:21.662Z</atom:updated>
            <content:encoded><![CDATA[<p>Creating a Python script that extracts information about files in a directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_gE7e-5gr8g3BJiHv4G3Hw.png" /></figure><p>Python is one of the most popular languages in the world, and is widely used and recognized. Python is a high-level, interpreted programming language that has been designed with a focus on simplicity and ease of use. It is known for its clean syntax, which makes it easy to read and write, and its large collection of libraries and modules, which provides developers with a wide range of tools for a variety of tasks.</p><p>Python is in use all around us in fields such as data analysis, web development, and artificial intelligence, and it is also popular in data science and machine learning. Python is used extensively in web development, where it is often used to build web applications, APIs, and server-side scripts.</p><h3>Prerequisites</h3><ul><li>A basic understanding of Linux and the command line</li><li>A basic understanding of Python</li></ul><p>This basic Python script below can be created in any code editor such as VS Code, Pycharm, or in my case AWS Cloud9. The file that we are creating should be saved as a .py file.</p><pre>#Week 13 project<br><br>#!/usr/bin/env python3.7<br><br>import os<br><br># Get the current working directory<br>my_dir = os.getcwd()<br><br># Create an empty list to store file information<br>files_list = []<br><br># Loop over the entries in the directory<br>for entry in os.scandir(my_dir):<br>    # Get the file path, file name, and size<br>    file_path = entry.path<br>    file_name = entry.name<br>    file_size = entry.stat().st_size<br>    <br>    # Create a dictionary with the file information<br>    file_info = {&#39;path&#39;: file_path, &#39;file_name&#39;: file_name, &#39;size&#39;: file_size}<br>    <br>    # Append the dictionary to the list of files<br>    files_list.append(file_info)<br>    <br># Print the list of files<br>print(*files_list, sep=&quot;\n&quot;)</pre><p>I will explain each line and what it does. Any line that has a # at the start and is grayed out is a comment. These lines are not executed as part of the code.</p><pre>import os</pre><p>This is the OS module; it is a built-in library within Python that allows users to interact with the operating system. We can use this particular module to collect information about the files in our current working directory. This module is one of many libraries that are available for use in Python.</p><pre>my_dir = os.getcwd()</pre><p>‘os.getcwd()’ in this statement is a function of the OS module that we talked about earlier. It returns the current working directory where the script is being executed.</p><p>‘my_dir’ is the variable; the output from the statement ‘my_dir = os.getcwd()’ is assigned to the variable, and as a result the variable ‘my_dir’ now contains the working directory as a string.</p><pre>files_list = []</pre><p>This statement creates an empty list and named ‘files_list’ which is also the variable. The variable ‘files_list’ has been assigned the empty list shown as the square brackets ‘[]’. The empty list that was created is mutable and can be used later in the code to store data such as file names, file sizes, and other attributes.</p><pre>for entry in os.scandir(my_dir):</pre><p>This is a ‘for’ loop that will iterate over each file in the working directory. The loop executes a block of code for each item in the sequence.</p><pre>    file_path = entry.path<br>    file_name = entry.name<br>    file_size = entry.stat().st_size</pre><p>This block of code first extracts the full path of the object (file) using ‘file_path = entry.path’. Then we extract the name of the object (file) using ‘file_name = entry.name’. Last we use ‘file_size = entry.stat().st_size’ to obtain the size of the file in the working directory.</p><pre>file_info = {&#39;path&#39;: file_path, &#39;file_name&#39;: file_name, &#39;size&#39;: file_size}</pre><p>‘file_path’, ‘file_name’, and ‘file_size’ are variables in this code that hold the path, name, and size of the file. This code creates a dictionary which is then used to store information about the file or directory entry and append it to the ‘files_list’ list.</p><pre>files_list.append(file_info)</pre><p>‘file_info’ is a dictionary that contains the information from above about the file path, name and size. We then ‘append’ the file to “files_list’ where the dictionaries that contain the information about the file are stored.</p><pre>print(*files_list, sep=&quot;\n&quot;)</pre><p>This code uses the print function to print the contents of the list ‘files_list’. By using the ‘*’ operator in front of ‘files_list’, the print function separates the list into individual arguments, which is then printed onto separate lines using ‘sep=”\n”’.</p><h3><strong>Running the script:</strong></h3><p>As mentioned earlier this file should have been saved as a .py file and also saved into the directory that you want. You need to make the file executable; this can be accomplished by running the following commands.</p><pre>#Change directory<br>cd name_of_directory<br><br>#long list will list the files in the directory and the permissions associated<br>ls-l<br><br>#Change the permissions of the file<br>chmod u+x name_of_file<br></pre><p>Use the ls-l command to make sure your permissions allow you to execute the file. Then use the following command to run it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qBaM39gV1rANGJRq6QePHw.png" /></figure><p>Run the scipt</p><pre>python name_of_file.py</pre><p>My output below lists the file path, name, and size as a list of dictionaries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1rx3pyTxOTauPgNbHePdHA.png" /></figure><p>Thank you very much for reading and following along. I hope you join me for my next project that will further my exploration of Python and document my journey further into DevOps.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2200ed53e1b6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a AWS DynamoDB table with least privilege using AWS CLI]]></title>
            <link>https://medium.com/@abach06/creating-a-aws-dynamodb-table-with-least-privilege-using-aws-cli-e684b89a06ed?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/e684b89a06ed</guid>
            <category><![CDATA[aws-dynamodb]]></category>
            <category><![CDATA[ec2-instance-connect]]></category>
            <category><![CDATA[dynamodb]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Wed, 11 Jan 2023 09:21:04 GMT</pubDate>
            <atom:updated>2023-01-11T09:21:04.237Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FkzifEt__yMaBHkkcAaWNQ.png" /></figure><p><strong>DynamoDB</strong></p><p>The need for safe and reliable storage solutions for the vast amounts of data generated by today’s businesses and applications is paramount. AWS DynamoDB is a serverless NO-SQL database managed by AWS; it is fast, reliable, and flexible. DynamoDB is a highly available service that is fault tolerant.</p><p>Today I will be walking through how to create a DynamoDB table, upload sample data to the table, and create and assign an IAM role using the principle of least privilege.</p><p><strong>Partition Key</strong></p><p>Data is stored as items within a table in DynamoDB. Each item has a unique primary key, which consists of a <strong>partition key</strong>. This key is also used to distribute data across DynamoDB servers and is what allows the table to scale to accommodate a high traffic load and large amounts of data.</p><p>An optional <strong>sort key</strong> can be used in conjunction with the partition key to narrow the search field for more cost efficient search results.</p><h3>AWS Identity and Access Management (IAM) Role</h3><p>We will be creating an IAM role which we can use to grant permissions to people or AWS services or applications.</p><p>Prerequisites:</p><ul><li>An AWS account with IAM user access.</li><li>Knowledge of AWS Management Console, EC2 instances, and security groups.</li><li>Basic knowledge of Linux and AWS CLI.</li></ul><p><strong>Create a DynamoDB Table</strong></p><p>To get started, log into the AWS management console with your user account and navigate to the DynamoDB page. Click the orange “Create table” button to get started.</p><p>For my example, I will be using ten great quotes; this is not a ranking or debate, just quotes that I like. Name your table and set your partition key. I will be using the Author of the quote as the sort key. We will also be using the default table settings. Scroll down and click “create table.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DW98ltxC2QpHYU-456gMIg.png" /></figure><p>Once created, your new table will look like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LxbbzbHNNQbpcUMzQtG2HQ.png" /></figure><p>Click on your new table, and you will see we have no items in our item summary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mRpAZ5lwUtGLoKBR3RJ2Xg.png" /></figure><p>We are now going to add a few items to our table by selecting “Create item” from the Actions drop-down menu.</p><p>My attributes are the quote itself and author, so I insert the information for each. Add as many items as you want.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DTFjMFF1YagD7ydVxBgTkg.png" /></figure><p>Once you have entered all your items, you will see them all listed when you click “Explore items” under the Tables menu section of your DynamoDB dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lr_TjMuUHzu3ewXJKYpo2A.png" /></figure><p><strong>Launch an EC2 instance with an assigned IAM role</strong></p><p>In the IAM dashboard, click “Roles,” then “Create role.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WUXVishLF6Fl5yAITFpRow.png" /></figure><p>For your “trusted entity type,” select “AWS service,” and for “Use case,” select “EC2.” Click the “Next” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*229j2GDfhQfLpIQN4WkVzw.png" /></figure><p>In the search bar, type in “DynamoDB,” select the permission that “Provides Read Only Access,” then click the “Next” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4tEUp2JlxjY3u4VGPH71zA.png" /></figure><p>Give your new role a name. Below you can see the policy which is written in JSON. The policy will <strong>“Allow” </strong>the <strong>“Action” </strong>of<strong> “AssumeRole” </strong>for the EC2 instance. This will allow our EC2 instance to scan the DynamoDB table we created earlier and read the contents. After reviewing the role, click the blue “Create role” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BD3QYvAEh_b3vsj1I6CymQ.png" /></figure><p>Next we will head to the EC2 dashboard to create an instance. Click the “Launch instance” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZZe0hWXAv-HIYy0u52qLsw.png" /></figure><p>Name your instance; leave the selected AMI as “Amazon Linux.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1xohBmj48XjbHKOYZ4lXwg.png" /></figure><p>Below that you will see “Instance type;” leave this as “t2.micro (free tier eligible).” Select your Key pair, or create a new one if needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3KgO-2y4OnAriL76zoC-3w.png" /></figure><p>If you need to create a key pair, click “Create new key pair.” On the page below, name your key, and leave the key pair type as “RSA” and the private key file format as “.pem.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NWfl9x3SlJXXxbbqQPY9Eg.png" /></figure><p>In “Network settings,” click “Edit,” and you will see as below the VPC selected as default. Select a subnet (or you can use the default subnet, which should be fine), and enable “Auto-assign public IP.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VdvL6mmcBh1dv1nif5Ye7g.png" /></figure><p>For “Firewall” settings, I am creating a new security group; at the bottom click the “Add security group rule” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1AE0rIGUBQV-r3TWQMk2hA.png" /></figure><p>Add security rules to allow HTTP, and HTTPS in from anywhere. This is not best practice, but is fine for a demonstration. Launch the instance by clicking the orange button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YD7eQryl33pP6hDvX8RQIg.png" /></figure><p><strong>Add an IAM Role</strong></p><p>We are now going to attach the read-only access role we created in IAM to our new EC2 instance.</p><p>Select your instance; click the “Actions” drop-down&gt;Security&gt;Modify IAM role.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QB0n5ceGLoGk6tjfDnM0_Q.png" /></figure><p>Select the role that we created earlier, and click the “Update IAM role” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*osMQigZdxEU5JtS00-YZEA.png" /></figure><p>You should see a success message. Now we are on to the final step of today’s project: scanning our DynamoDB table!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6NclFjLUoqq9ol58uhlBww.png" /></figure><p><strong>Scan DynamoDB table with AWS CLI</strong></p><p>To test the permissions of the IAM role that we created, I will be using the AWS CLI and connecting via SSH to our EC2 instance. For more information about using AWS CLI, please read my article <a href="https://medium.com/@abach06/aws-cli-an-introduction-c9a2d446207a">AWS CLI: An Introduction</a>.</p><p>Select your instance, click “connect,” then click the “SSH client” tab and copy the example down at the bottom of the page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ln3DhjQQUf8K362lK9ymzg.png" /></figure><p>If you are using a new key pair then you will need to run this command:</p><pre>chmod 400 (your key pair)</pre><p>Paste the copied link into your terminal. You should be questioned, “Are you sure you want to connect?” Type “yes,” and we have now successfully SSH’ed into our eC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lBucL4uIJ81pPFLn9pBq0Q.png" /></figure><p>Once logged into the EC2, we are going to scan the DynamoDB table to verify that the role we created earlier works.</p><p>Run the following command:</p><pre>aws dynamodb scan --table-name &lt;table_name&gt; --region &lt;region_name&gt;</pre><p>The results will be in JSON and should look similar to this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RyRfR3NMuFZaANdkxSBCxQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CCQJnbJwthlY3YXgn3Tl7A.png" /></figure><p>There is more, but you get the idea. We have now successfully scanned the contents of our DynamoDB table.</p><p><strong>Validate “Read Only” Access to DynamoDB Table</strong></p><p>We will now verify that the IAM role that we created earlier is in fact read-only. We will test our role by trying to write to our DynamoDB table. Use the following command in the AWS CLI:</p><pre>aws dynamodb put-item --table-name &lt;table_name&gt; --item &#39;{&quot;&lt;partition_key&gt;&quot;: {&quot;S&quot;: &quot;&lt;value&gt;&quot;},&quot;&lt;sort_key&gt;&quot;: {&quot;S&quot;: &quot;&lt;value&gt;&quot;}}&#39; --region us-east-1</pre><p>After entering this command you should see this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GXNIiqLTr4pxil76jbK3RA.png" /></figure><p>This error message confirms that the IAM role created is in fact working as intended.</p><p>Thank you for reading and following along. I will be posting other content as my journey continues, and I hope you will join me!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e684b89a06ed" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Create an auto scaling group of EC2 instances for high availability automation]]></title>
            <link>https://medium.com/@abach06/create-an-auto-scaling-group-of-ec2-instances-for-high-availability-automation-ca8063c90a86?source=rss-59e8fc4e60f2------2</link>
            <guid isPermaLink="false">https://medium.com/p/ca8063c90a86</guid>
            <category><![CDATA[aws-auto-scaling]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[high-availability]]></category>
            <category><![CDATA[ec2-instance]]></category>
            <dc:creator><![CDATA[Aaron Bachman]]></dc:creator>
            <pubDate>Tue, 10 Jan 2023 03:11:07 GMT</pubDate>
            <atom:updated>2023-01-10T03:11:07.959Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UsEv21opVHD9UdMv4p-jGQ.png" /></figure><p>Utilizing the cloud to create a fault-tolerant and highly available EC2 environment is simple to set up and can save a business a lot of time and money over standard infrastructure.</p><p>An auto scaling group helps maintain the correct number of EC2 instances needed for your application by increasing the number when needed and then decreasing them. based on a set of variables that the user determines.</p><p>Today we will be building cloud infrastructure to support our auto scaling group, and in doing so will be utilizing many different AWS resources.</p><p>Today you will need:</p><ul><li>An Amazon web services account</li><li>A computer</li></ul><p>Let’s start by setting up a Virtual Private Cloud (VPC) with CIDR 10.10.0.0/16. A Virtual Private Cloud is an AWS service that provides users with a virtual private network that is logically isolated from all other networks. Every VPC has its own block of IP addresses called a CIDR block.</p><p>Navigate to the VPC page by typing “VPC” in the search bar on your AWS management console. Click the “Create VPC” button in the upper right corner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HXI9XYwNaCCqL4GVGIRTEA.png" /></figure><p>Select “VPC only” as your resource to create, name your VPC, and use the provided IPv4 CIDR of 10.10.0.0/16. Leave everything else as default.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QZ0naX5D7MQ9yRzYjIPa0w.png" /></figure><p>Click the “Create VPC” to make your new VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HrRBRyboqzDVrEE6KvyRJQ.png" /></figure><p>Next we will create three subnets. In your VPC dashboard, click on the Subnets option under the “Virtual private cloud” section, and then click the “Create subnet” button in the upper right corner. Subnets are created within an availability zone (AZ), but they cannot span across AZ’s. In order to balance our workload, we are going to create subnets in multiple AZ’s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xOE5AC3OOic4klTW9q7sBQ.png" /></figure><p>Select your new VPC. We are going to create three subnets at the same time in three different availability zones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7pW4M3lZJ75bb_q35j37yQ.png" /></figure><p>After you enter the information for your first subnet, click the “Add new subnet” button in the bottom left corner. You can then add the next two subnets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BfE6gWinbxsww923bl2C1Q.png" /></figure><h4><em>Subnet # 1</em></h4><ul><li>Name your subnet</li><li>Availability zone: us-east-1a</li><li>IPv4 CIDR block: 10.10.1.0/24</li></ul><h4>Subnet # 2</h4><ul><li>Name your subnet</li><li>Availability zone: us-east-1b</li><li>IPv4 CIDR block: 10.10.2.0/24</li></ul><h4>Subnet # 3</h4><ul><li>Name your subnet</li><li>Availability zone: us-east-1c</li><li>IPv4 CIDR block: 10.10.3.0/24</li></ul><p>Once done, you will receive a message saying that you have successfully created three subnets, and you will be able to see them in your subnet tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SxbrapiDK5xCMfjz8AOGZw.png" /></figure><p>Now click on each subnet and select the “Edit subnet settings” option in the Actions dropdown menu, which is in the upper right corner. We will enable<strong> “auto-assign public IPv4 address” </strong>so that our subnet can send and receive traffic over the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xiwXMXs0TeTWy_dM3EK58w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eDKDD_5PnQRiCuRhoQ-GFQ.png" /></figure><p>Make sure to perform this step for all three subnets.</p><p>We now want to make sure that our subnets can reach the internet, so we will create an internet gateway. Our subnets are public subnets; if they were private, we would use a NAT gateway so our subnets would be able to send traffic to the internet, but not receive.</p><p>In your VPC dashboard , select the “<strong>Internet gateways” </strong>option under the “Virtual private cloud” section, then click the orange “Create internet gateway” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xvNmCZIi84YWulzUHcvgBw.png" /></figure><p>Name your gateway and click “create internet gateway.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tB_huZP7mJG1kIAbNlvTWg.png" /></figure><p>You will receive a green message saying that you have successfully created an internet gateway. In this message bar, click “Attach to a VPC.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A7ZHwLxloSnI9lrPizTGVw.png" /></figure><p>Select your VPC from the dropdown menu, and click “Attach internet gateway.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dekTuuYyFr3zGm9O1PpSbw.png" /></figure><p>You will now see a confirmation message that your internet gateway has been successfully attached to your VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uHVsliva1BWVk1vKddnAVA.png" /></figure><p>Let’s now navigate to the <strong>Route tables</strong> option under the “Virtual private cloud” section of your VPC dashboard . Our route table was automatically created, but we are connecting to the internet using an internet gateway (IG), so we need to add our IG to our route table.</p><p>Select the correct route table by looking for the VPC name that you created for this exercise. If needed, you can expand the VPC column by dragging the column edge to the right.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pVrteosLEPQiziqNTw6q0Q.png" /></figure><p>In the Actions drop-down menu, you can select “Edit routes” or, you can click on the Routes tab in the selected route table, then click “Edit routes.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WbFIhjA2LOmboES9UYMJ_g.png" /></figure><p>Click the “Add route” button, select or input 0.0.0.0/0 (all IP addresses), and choose the internet gateway that you created. Click the “Save changes” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AiYr4XmdDoKCoN0UYGzRWg.png" /></figure><p>While we are in route tables we also have to edit the explicit subnet associations. Go to the third tab to the right (“Subnet associations”), click the button “Edit subnet associations,” and add all three of our subnets to the list.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6sHoq642thLk7WlmrRd-sA.png" /></figure><p>We are now moving on to creating an autoscaling group. In order to do this, we need to create a Launch Template. In the EC2 dashboard, click the “Launch Templates” option under the Instances section. On the next page there will be an orange “Create launch template” button: click it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fbk7LLLb28K_AiLgSDZTdA.png" /></figure><p><strong>Launch template settings:</strong></p><ul><li><strong>Name your template</strong></li><li><strong>Application and OS Images (Amazon Machine Image): Select </strong>Quick Start and Amazon Linux.</li><li><strong>Instance type:</strong> t2.micro (free tier eligible)</li><li><strong>Key Pair:</strong> Use an existing key pair or generate a pair (RSA/.pem)</li><li><strong>Network Settings:</strong> Create a Security Group by attaching your VPC and adding the following inbound rules for HTTP.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zdNoQeVJWp9r84LPH4icbA.png" /></figure><p>Under “Network settings,” select “create security group,” name your new group, and then select the correct VPC. After that, click the “Add security group rule” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6vzOP14-xt7IMDRgjQEQ1Q.png" /></figure><p>See below for the settings for Security group rule 1:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ip1JWl_R0yQefwE6N2KYtQ.png" /></figure><ul><li>Add HTTP /port 80/ source Anywhere (for this demonstration)</li><li>Add ssh /port 22/ source “My IP,” enter your IP address.</li><li>Add HTTPS /port 443/ source Anywhere (for this demonstration)</li></ul><p>Next click on “Advanced Network configuration.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzh7L9n0uAGfK8XJeteZXA.png" /></figure><p>Enable Auto-assign public IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SsztuQ2Kn20pyeTuJwuYQQ.png" /></figure><p>We are going to leave the Storage and Resource tags alone. Click on Advanced details;” we are going to go all the way to the bottom to <strong>User data.</strong></p><p>We are going to bootstrap an Apache server and updates to launch with the EC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rd9zck9D3t4OdR-WpiM5rg.png" /></figure><p>Click the orange “Create Launch Template” on the bottom. You will receive another message stating creating your launch template if everything worked, and you will see this under launch templates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QoA7b2aqCVQVBod_BcBYWA.png" /></figure><p>Over on the EC2 dashboard, go all the way to bottom and select “auto scaling groups.” Click the orange “Create auto scaling group”button. Name your auto scaling group, and then select the template that we just created. Click the “Next” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RkN-9XUL61FAJQF_Z83HgA.png" /></figure><p>Under the Network section, make sure to select the correct VPC, and then select all three subnets that we created. Click the “Next” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Sm2AEvEYlKkZ5jKkfu6sIw.png" /></figure><p>On the next page, set “Health check grace period” to 120 seconds. Make sure to check the box for “Enable group metrics collection within CloudWatch.” Click the “Next” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dKvCqdQT-M3KZpVlZNLYHw.png" /></figure><p>Here we are setting the desired group size and scaling policies:</p><ul><li>Set the Desired capacity and Minimum capacity to 2.</li><li>Set the Maximum capacity to 5.</li><li>Select “Target tracking scaling policy.”</li><li>Set Metric type to “Average CPU utilization.”</li><li>Set the target value to 50.</li><li>Click the “Next” button.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2mKWTkxnKUSt81A21ZawCA.png" /></figure><p>Forward through the next two screens without any changes by clicking the “Next” button on each screen, then review your auto scaling group. Make sure the VPC and subnets selected are correct. Click “Create auto scaling group.” If you now navigate to the EC2 instance page, you will see that we have two instances that have been provisioned and are initilizing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hg3p5OBwd8BwEmdRYvc0Zw.png" /></figure><p>Enter the public IPv4 address into your browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZMeI2d7Hn9zBY_7FIQuiOQ.png" /></figure><h3><strong>Load Balancer:</strong></h3><p>First up we need to create a target group. In your EC2 dashboard, select “Target groups” under the Load Balancing section and click the “Create target group” button in the upper right corner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ew792Tm2b5R6Z3-NEfHIOA.png" /></figure><p>Next leave the target group as “Instances,” then name your target group. The Protocol should be set to HTTP and port 80.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KIhfOOAMRym9zdF8riHnWg.png" /></figure><p>Choose the correct VPC, and click the “Next” button. On the next page, do not select the instances; click the button “Create target group” in the upper right corner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*62u8q1ZOZiy6eXxtlQV7dg.png" /></figure><p>In your EC2 dashboard, select Load balancers.” Choose to create an “Application load balancer (ALB).”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qKZhTqMZQgd63MSIFL3ARg.png" /></figure><p>Name your load balancer, make sure to select the right VPC, and choose all three of your subnets.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rTZKawgOBZSfbyUUTJpgQA.png" /></figure><p>Select your security group, and delete the default sg group.</p><p>Listeners and routing should be set to HTTP and port 80. Select your target group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jY1bc2lygRyS1ilZcJ9Bsg.png" /></figure><p>Scroll to the bottom and click “Create load balancer.” After your new load balancer is created, we are going to associate/connect it with our auto scaling group.</p><p>In your EC2 dashboard, go back down to “Auto scaling groups” under the Auto Scaling section, then select your group and click the “Edit” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nw2GnLAfdnBcYhLvxW8mag.png" /></figure><p>Scroll down to “Load balancing” and check “Application, network or gateway load balancer target groups;” select your target group, then click the orange “Update” button on the bottom right side.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9GEqUo35cuIWHzPH0hcVyw.png" /></figure><p>Back up in the “Target groups” section, you should see that we have two healthy instances running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C56qbajsCMedsp_rFTzCNg.png" /></figure><p>You can now go to the “Load Balancers” section in your EC2 dashboard, copy the DNS name of your load balancer, and paste it into your browser. When you refresh your browser, the AZ should update.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_9TfNESsyejgYN6i3ykdew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*522oIaiFzjz5CTBT1eDMCw.png" /></figure><p><strong>Congratulations!!</strong></p><p>You have created a highly available group of auto scaling EC2 web servers!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ca8063c90a86" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>