<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by David Lourenço on Medium]]></title>
        <description><![CDATA[Stories by David Lourenço on Medium]]></description>
        <link>https://medium.com/@david_lourenco?source=rss-8d45f08b600b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jS84oZKbXdDMXjxEA8ENKg.png</url>
            <title>Stories by David Lourenço on Medium</title>
            <link>https://medium.com/@david_lourenco?source=rss-8d45f08b600b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:24:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@david_lourenco/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Implement CloudFront using Edge Locations in an Application running in AWS Cloud Infrastructure]]></title>
            <link>https://medium.com/@david_lourenco/implement-cloudfront-using-edge-locations-in-an-application-running-in-aws-cloud-infrastructure-91ae4a8298aa?source=rss-8d45f08b600b------2</link>
            <guid isPermaLink="false">https://medium.com/p/91ae4a8298aa</guid>
            <category><![CDATA[ec2]]></category>
            <category><![CDATA[elastic-beanstalk]]></category>
            <category><![CDATA[cloudfront]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[David Lourenço]]></dc:creator>
            <pubDate>Tue, 02 May 2023 11:53:11 GMT</pubDate>
            <atom:updated>2023-05-02T11:53:11.091Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Note: </strong><br>This paper, was created as an improvement of another solution developed and documented <a href="https://medium.com/@david_lourenco/implementation-of-a-scalable-web-application-using-aws-elastic-beanstalk-and-dynamodb-3dd491b5dd63">here</a>. Please take a look for more context.</h3><p><strong>Implementation Must Have Concepts :<br></strong>In this section, some significant concepts are clarified for the understanding of the implementation described below.</p><p><strong>Edge Locations</strong> — Edge locations are AWS data centers designed to deliver services with the lowest latency possible. Amazon has dozens of these data centers spread across the world. They’re closer to users than Regions or Availability Zones, often in major cities, so responses can be fast and snappy.</p><p><strong>CloudFront</strong> — Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content, such as .html, .css, .js, and image files, to your users. CloudFront delivers your content through a worldwide network of data centers called edge locations. When a user requests content that you’re serving with CloudFront, the request is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance.</p><h3>How CloudFront delivers content to your users</h3><p>After configure CloudFront to deliver your content, here’s what happens when users request your objects:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/0*erfpknhBhXHdtr5y.png" /></figure><ol><li>A user accesses your website or application and sends a request for an object, such as an image file or an HTML file.</li><li>DNS routes the request to the CloudFront POP (edge location) that can best serve the request — typically the nearest CloudFront POP in terms of latency — and routes the request to that edge location.</li><li>CloudFront checks its cache for the requested object. If the object is in the cache, CloudFront returns it to the user. If the object is <em>not</em> in the cache, CloudFront does the following:</li></ol><ul><li>3a— CloudFront compares the request with the specifications in your distribution and forwards the request to your origin server for the corresponding object — for example, to your Amazon S3 bucket or your HTTP server.</li><li>3b — The origin server sends the object back to the edge location.</li><li>3c — As soon as the first byte arrives from the origin, CloudFront begins to forward the object to the user. CloudFront also adds the object to the cache for the next time someone requests it.</li></ul><h3>Create CloudFront Distribution</h3><p>Cloud Front also known as Content Delivery Network is a web service that speeds up distribution of static and dynamic web content, such as .html, .css, .js, and image files. It delivers content through edge locations and speed up response times in web applications. The user request, is smartly redirected to the right edge location and the requested content, is returned as son as possible.</p><p>Regarding implementation steps:</p><blockquote><em>In the console search bar, search for CloudFront and click</em><strong><em> Create CloudFront distribution</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/0*qPf3-hFFL44DlRKl.png" /></figure><blockquote><em>Select the </em><strong><em>Origin domain</em></strong><em> where the CloudFront will cache all the static files and leave all the configurations as default. Here, we should chose the resource that we would like to share.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/0*zbQIEpuk_k5Waewx.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/0*3N9Mhsj_hUQC_lCb.png" /></figure><blockquote><em>Click on </em><strong><em>Allowed HTTP methods</em></strong><em> and select GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE to allow the application use POST to insert the data inside the DynamoDB</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/0*hKF4nJMQlssLIHJF.png" /></figure><p>On the Settings, leave it as <strong>Use all edge locations to archive best performance.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/728/0*zOrojsJxtIpp7QCt.png" /></figure><blockquote><em>Click Create Distribution and </em>your CloudFront <em>distribution will be created.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*NyCc-kBQGnL2R01V.png" /></figure><p>After creation, we will see, our CloudFront instance as enabled. This means that we are able to test and start using our application with CloudFront to help on performance. <br>To test, we should use the new URL, generated during CloudFront instantiation.</p><blockquote><em>Copy the URL and paste into a browser to see if the application will come up</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*WJeZNz0WVXqIrMMh.png" /></figure><p><strong>Conclusion:</strong></p><p>CloudFront speeds up the distribution of your content by routing each user request through the AWS backbone network to the edge location that can best serve your content. This is basically a smart way of routing to match customer needs. Typically, this is a CloudFront edge server that provides the fastest delivery to the viewer. This solution is an amazing approach to follow when we want or need to offer slow response time to our customers.</p><p><strong>Used AWS Resources:</strong></p><p>CloudFront</p><h3>Reminder: Terminate, stop or destroy all the resources when you are done to avoid being charged for using AWS services beyond the free capacity.</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91ae4a8298aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementation of a Scalable Web Application using AWS Elastic Beanstalk and DynamoDB.]]></title>
            <link>https://medium.com/@david_lourenco/implementation-of-a-scalable-web-application-using-aws-elastic-beanstalk-and-dynamodb-3dd491b5dd63?source=rss-8d45f08b600b------2</link>
            <guid isPermaLink="false">https://medium.com/p/3dd491b5dd63</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[elastic-beanstalk]]></category>
            <category><![CDATA[amazon-web-services]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[David Lourenço]]></dc:creator>
            <pubDate>Tue, 17 Jan 2023 16:47:35 GMT</pubDate>
            <atom:updated>2023-05-02T11:54:43.755Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TQTSJlsotnynZcuw.png" /></figure><p><strong>Project description:</strong></p><p>In this project based on a real-world scenario, I was responsible for implementing an application that needs to support the high demand of a large number of users accessing it simultaneously. This application has been used in a large conference that had more than 10,000 people, in-person and online, with participants from all over the world.</p><p>This event was broadcast online and in person. During the event 10 vouchers were drawn for 3 Cloud certifications and at that moment, more than 10,000 people in the audience were able to access and register their e-mail to guarantee their participation in the raffle.</p><p>To implement this project, was used Elastic Beanstalk services. Elastic Beanstalk was responsible for deploy our web application, our database (DynamoDB) to store customer information, and a CloudFront instance to cache static and dynamic files in an Edge Location close to the users.</p><p><strong>Project Architecture:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QJ8XpwV5fRl78g48.png" /></figure><p><strong>Implementation Must Have Concepts :<br></strong>In this section, some significant concepts are clarified for the understanding of the implementation described below.</p><p><strong>Elastic Beanstalk </strong>— is an orchestration service offered by Amazon Web Services for deploying applications which orchestrates various AWS services, including EC2, S3, Simple Notification Service, CloudWatch, autoscaling, and Elastic Load Balancers. With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS Cloud without having to learn about the infrastructure that runs those applications. Elastic Beanstalk reduces management complexity without restricting choice or control. You simply upload your application, and Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.</p><p><strong>DynamoDB</strong> — Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. Basically is one NoSQL solution from Amazon.</p><p><strong>Edge Locations</strong> — Edge locations are AWS data centers designed to deliver services with the lowest latency possible. Amazon has dozens of these data centers spread across the world. They’re closer to users than Regions or Availability Zones, often in major cities, so responses can be fast and snappy.</p><p><strong>CloudFront</strong> — Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content, such as .html, .css, .js, and image files, to your users. CloudFront delivers your content through a worldwide network of data centers called edge locations. When a user requests content that you’re serving with CloudFront, the request is routed to the edge location that provides the lowest latency (time delay), so that content is delivered with the best possible performance.</p><p><strong>CloudWatch</strong> — CloudWatch enables you to monitor your complete stack (applications, infrastructure, network, and services) and use alarms, logs, and events data to take automated actions and reduce mean time to resolution (MTTR). This frees up important resources and allows you to focus on building applications and business value. A usefull use case is, integrate slack to send alarms to the team.</p><p><strong>Project Implementation:</strong></p><p>This implementation was divided in two different parts.</p><p><strong>Part 1:</strong> Creation of our database instance to store users and an Elastic Beanstalk instance to provision all the necessary infrastructures to run our application.</p><p><strong>Part 2:</strong> Perform manual testing on the application and ensure rigth functioning.</p><p><strong>Part 3:</strong> Perform Load testing on the application and ensure auto-scaling functioning.</p><h3>Part 1: Create a Dynamo DB to store user email address</h3><p>Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.</p><p>Create a database in AWS is a simple process and create a DynamoDB instance is not an exception. Search for DynamoDB in the AWS console and click to select it. This service will be used to store email addresses from users.</p><blockquote><em>Table name: users</em></blockquote><blockquote><em>Partition key: email</em></blockquote><p>Leave everything else as default and click <strong>Create table</strong> at the bottom right</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*U8cG4-y0iWnbtsQ8.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/0*FrfwXq8LEGdPWjix.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IipWBTgVk7jPghdiFwzJDw.png" /></figure><h3>CreateElastic Beanstalk</h3><p>This service will be used to deploy and manage the web application in AWS Cloud without worrying about the infrastructure that runs it. We will upload the application files and it will automatically handle the details of capacity provisioning, load balancing, scaling, and application health monitoring.</p><p>Search for elastic beanstalk in the AWS console and click to select it.</p><p>Select “<strong>Create Application” a</strong>nd start typing some basic information.</p><blockquote><em>Application name: tcb-conference</em></blockquote><blockquote><em>Platform (runtime environ.): Python 3.7+</em></blockquote><blockquote><em>Application code: Upload the application file code -&gt; The Public S3 URL is provided below</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*Hspp_znEE73hMBmd.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/0*eQSxtMJdFvhRJZOA.png" /></figure><pre>https://tcb-bootcamps.s3.amazonaws.com/bootcamp-aws/en/tcb-conf-app-EN.zip</pre><p>Our application was not developed by us because this exercise is focused on migrate applications using AWS infrastructure and not, about create applications. Despite this, the application was placed in an S3 bucket and we were able to use from there.</p><p>After basic information, we can click on <strong>“Configure more options”</strong> to make changes and adapt our Elastic Beanstalk to our use case needs.</p><p>We are able to change Platform Versions, Software requirements, Instance Defenitions, Database Instances, Network Requirements and Auto-Scaling groups.</p><p>Starting with presets, we are able to set some pre-configured options that matches in some different use cases. In our case, we are trying to build an resilient and high availability application. As such, we should set High Availability option.</p><blockquote><em>Set our presets to High availability.</em></blockquote><blockquote><em>Under Software, add the environment variable </em><strong><em>AWS_REGION</em></strong><em> with value </em><strong><em>us-east-1</em></strong><em> to the region and click Save</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/0*FKxOj4IO79WxvpLq.png" /></figure><p>High Availability preset, means that the EC2 instance that was created by elastic beans will be monitored and scaled when it is needed. Based on that, we should set a new Environment property to announce the region for the news EC2 Instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/0*PyUF66EtGInjyx3I.png" /></figure><p>Regarding instances, we are able to configure the type and the capacity for the propose that we need. For this use case, our root volume type was set to General Purpose SSD and our instance size to 8GB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*RwBzUUEtWzi-EXRN.png" /></figure><p>In capacity menu, we can configure if we will want to build a single instance aplication or if auto-scaling is needed. Based on the information that we provide, the Elastic Beanstalk will automatically create a Load Balancer for our application, to redirect trafic for the minimum and maximum of instances that was pre-configured. This will eliminate the need to respond manually in real-time to traffic spikes that merit provision of new instances</p><p><strong>Capacity:</strong></p><blockquote><em>. Instances Min: 2<br>. Instances Max: 4<br>. Instance type: t2-micro</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*rL9SH_jm0fECUJWM.png" /></figure><p>Auto-Scaling needs scalling triggers definition. In AWS Beanstalk, we can define diferent triggers to be sure that our instances were be either scale up or down. For this example, we will create triggers based on CPU utilization metrics. This allows Elastic Bean Stalk to automatically scale when our CPU hits a certain percentage of utilization.</p><p><strong>Scaling triggers:</strong></p><blockquote><em>. CPU Utilization<br>. Unit: Percent<br>. Period: 1 Min<br>. Breach durantion: 1 Min<br>. Upper threshold: 50<br>. Lower threshold: 40</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*MctVjE_j_5I074d1.png" /></figure><p>Now we will edit the security section to enable SSH connection with our instances. It&#39;s pretty easy. We’ll be using the same SSH key we created earlier, select that and then hit save!<br>If you have no SSH key defined, pelase take a look on: <a href="https://docs.aws.amazon.com/ground-station/latest/ug/create-ec2-ssh-key-pair.html">here</a></p><p><strong>Security:</strong></p><blockquote><em>Select SSH key</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*kaDOiNPLtpjfaPe9.png" /></figure><p>Last but not least, we’ll need to configure and set network configurations. For this we will set our VPC to default, add a public IP and select all subnets!</p><p><strong>Networking:</strong></p><blockquote><em>. VPC: Default:<br>. Add IP Public<br>. Select all subnets</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*KZmkZqEV4dRtnGm5.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*tgbUqQYkm7gLNiyD.png" /></figure><p>Finally, we can review all the changes that we have been made and Click on <strong>Create App.</strong></p><p>Elastic Bean Stalk will create everything for us, it will start with the EC2 instances, then the load balancer and start the application. This can usually take a bit of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*S8IfgtfyJ7i4kr8D.png" /></figure><p>As you can see below the deployment is completed and if everything was done correctly you should have an avalilible link to the web application on the top of our Elastic Beanstalk page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*aMD-2849FITtN_1j.png" /></figure><h3>Part 2: <strong>Aplication Test and Troubleshoting</strong></h3><p>Click on the URL presented on ElasticBeansTalk health dashboard and open a new browser window to test the aplication.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*SXnJgq6rQvk91aXL.png" /></figure><p>Returning to real use case, in the conference, the participants will need to enter their email address on the web page and the application will insert the email address into the DynamoDB.</p><p>But, if we try to register as is, we will not be able.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*uhk7FG4gyM9RTCJX.png" /></figure><p>Let’s try to understand and troubleshoot the error message in the Elastic Beanstalk logs.</p><blockquote><em>Go to logs and click the last 100 lines for the log file of the two instances</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*04CCXuLzL8IT-GYh.png" /></figure><p>After analysis, we are able to see that our EC2 instance does not have the required permission to perform a PUT operation on DynamoDB.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*Q26BtgFzCtU1b8r_.png" /></figure><p>We will need to make changes to the IAM role before the application is fully operational</p><p><strong><em>To fix this, we need to:</em></strong></p><blockquote><em>Navigate to IAM| Click on roles | Click on </em><a href="https://us-east-1.console.aws.amazon.com/iamv2/home?region=us-east-1#/roles/details/aws-elasticbeanstalk-ec2-role"><em>aws-elasticbeanstalk-ec2-role</em></a><em> |</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/0*DPTaQj-evveAOu6F.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*D1grDah7i9m7XGfA.png" /></figure><blockquote><em>Add permission | Attach policies| Type ‘DynamoDB’ | Select AmazonDynamoDBFullAccess and then click attach policies to associate to the EC2 instances</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/795/0*vQen_ZhrN7n0XrkK.png" /></figure><blockquote>Click Attach Policies and test your application again. Now, will work !</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bjLnhC4BRG_6HXmOtSIJ1Q.png" /></figure><p>By default, ElasticBeansTalk role, do not have write/post permissions because this is an invasive behaviour. To perform this, a permission needs to be added, as we saw below.</p><h3>Part 3: Perform Load Test</h3><p>Load test induce load on CPU, RAM, HDD SSD and mutch more but for the aim of this project we will induce CPU load. With this test, we want to create and simulate a real world environment and test the behaviour of our application under stress.</p><p>When a server is crashing or its hardware otherwise is misbehaving under load, it is helpful to induce ‘fake load’ in the system to help understanding the problem. In order to create this specific conditions we can use <strong>stress</strong> tool that is available in most Linux distributions.</p><p>Next, we will start our step by step to instal, configure and perform our stress test.</p><p>First we need to access our EC2 Instance via SSH to start our instalation process.</p><blockquote><em>Copy the IP-address from one of the EC2 instances in our EC2 instances board.</em></blockquote><blockquote><em>Use PowerShell or Git Bash terminal and perform an SSH call using the SSH Key previously created.</em></blockquote><pre>ssh ec2-user@3.81.228.68 -I ssh-key.pem</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*BJai8l42SYa_WuM72au5-Q.png" /></figure><p>After SSH connection, we need to instal stress tool using:</p><pre>sudo install epel -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/0*Fb18vOeVmqEv6gYM.png" /></figure><pre>sudo yum install stress -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/0*p_1UweT9du12hjIM.png" /></figure><p>Now we have our tool instaled and we just need to start our test: With the below command we will be able to run 4 workers to work and to bump our EC2 CPU utilization.</p><pre>stress -c 4</pre><p>After this step, we will see our Elastic Beanstalk making some changes and adaptations to meet the new demand. To start, we will see a warning for high CPU Utilization in our instance dashboard. Going back a few steps, we configured our Auto-Scaling group to create a new instance when CPU utilization is up to 50%.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*rBP1c5c0ii2HSR9n.png" /></figure><p>During this process, check Elastic Beanstalk health dashboards to see the changes:</p><blockquote>Check Health dashboards</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*jcTT_fnRi1dxObWW.png" /></figure><blockquote>Check Elastic Beanstalk Health Overview</blockquote><p>The instances will be marked as degraded and the Elastic Beanstalk will add one new instances with the Auto-Scaling group to keep up with the workload and making sure the users have more instances available to use the application without any lag.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*SxvPtPjE36h8NxR3.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*KjKNbzCRD8-1vJkr.png" /></figure><p>After a few minutes we will be able to see that our infrastructure is adapted to the new reality.</p><blockquote>Check the Auto-Scaling Group</blockquote><p>Now we are able to see three Instances of <em>Auto-Scaling groups </em>instead of one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*FpjO4SkjfDYank-T.png" /></figure><blockquote>Check again Elastic Beanstalk Health Overview</blockquote><p>And now we will see our aplication after changes, stable and ready to the new demands.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*JpI6DnhPbXLYqMFP.png" /></figure><blockquote>Check EC2 Instances</blockquote><p>And as expected, 2 new instances were created and now, working without warnings and issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*fonnpsHFKBp9xAZE.png" /></figure><blockquote>Test your aplication and you should see the same behaviour that we saw before</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*SXnJgq6rQvk91aXL.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bjLnhC4BRG_6HXmOtSIJ1Q.png" /></figure><h3>Accessory Work</h3><p>As an accessory work, we can do the reverse process and take a look how our infrastucture will recover after a big CPU SPIKE.</p><p>Lets start by:</p><blockquote><em>Stop our stress process by running the command below</em></blockquote><pre>ps -ef |grep stress<br>killall &lt;PID&gt;</pre><p>And after the load test is stopped, the auto-scaling group will scale down our infrastructure depending on the CPU utilization.</p><h3><strong>Reminder: Terminate, stop or destroy all the resources when you are done to </strong>avoid being charged for using aws services beyond the free capacity<strong>.</strong></h3><h3>Improvement and Next Steps:</h3><p>We will implement CloudFront and Edge Location to give response power to this project. <br>Take a look on: <a href="https://medium.com/@david_lourenco/implement-cloudfront-using-edge-locations-in-an-application-running-in-aws-cloud-infrastructure-91ae4a8298aa">here</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3dd491b5dd63" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migration of a Workload running in a Corporate Data Center to AWS using the Amazon EC2 and RDS…]]></title>
            <link>https://medium.com/@david_lourenco/migration-of-a-workload-running-in-a-corporate-data-center-to-aws-using-the-amazon-ec2-and-rds-5d9f78b17273?source=rss-8d45f08b600b------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d9f78b17273</guid>
            <category><![CDATA[aws-rds-instances]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[ec2]]></category>
            <dc:creator><![CDATA[David Lourenço]]></dc:creator>
            <pubDate>Tue, 06 Dec 2022 01:13:19 GMT</pubDate>
            <atom:updated>2022-12-20T23:55:32.115Z</atom:updated>
            <content:encoded><![CDATA[<h3>Migration of a Workload running in a Corporate Data Center to AWS using the Amazon EC2 and RDS service</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BylhtXE1S8YFksMg.png" /></figure><p><strong>Project description:</strong></p><p>In another project based on a real-world scenario, I acted as the Cloud Specialist responsible for migrating a workload running in a Corporate DataCenter to AWS.<br>The application and database were migrated to AWS using the Lift &amp; Shift (rehost) model, moving both application and database data.</p><p>I followed some migration steps: Planning (sizing, prerequisites, resource naming), Execution (resource provisioning, best practices), Go-live (validation test — Dry-run, final migration — Cutover) and Post Go-live (ensure the operation of the application and user access).</p><p>During execution, some important steps were done and next, I will explain and detail the most important ones.</p><p><strong>Solution Architecture:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qyCoemD8V4hVGyYa.png" /></figure><p><strong>Solution Must Have Concepts :<br></strong>In this section, some significant concepts are clarified for the understanding of the implementation described below.</p><p>VPC — <em>A V</em><strong><em>irtual Private Cloud</em></strong><em> is a private cloud computing environment contained within a public cloud. Essentially, a VPC provisions logically isolated sections of a public cloud in order to provide a virtual private environment.</em></p><p>EC2 — <strong>Amazon Elastic Compute Cloud</strong> is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers. Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction.</p><p>RDS — <strong>Amazon Relational Database Service</strong> is a collection of managed services that makes it simple to set up, operate, and scale databases in the cloud.</p><p>SS — An <strong>AWS security group</strong> acts as a virtual firewall for your EC2 instances to control incoming and outgoing traffic. Both inbound and outbound rules control the flow of traffic to and traffic from your instance, respectively.</p><p>IG — <strong>Internet Gateway </strong>is a horizontally scaled, redundant, and highly available VPC component that allows communication between your VPC and the internet. Internet Gateway enables resources (like EC2 instances) in public subnets to connect to the internet.</p><p><strong>Project Implementation:</strong></p><p>As we all know, security is one of the most significant topics when we are talking about data. Data is a dedicated topic and the process of build a secure application starts when we need to designs and make decisions about our architecture. In AWS, we are able to use VPC.<br> Inside which VPC, we are able to create private and public subnets. Making use of this capability, was created a private network to host our RDS (SQL database) and a public network to host our EC2 instance with our Application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OyrtCobDe892LfTTn5wERw.png" /><figcaption>Virtual Private Cloud</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rcCSaEqAJYxc6pcNpGwVmw.png" /><figcaption>Public &amp; Private Subnets</figcaption></figure><p>By default, when we create a VPC in AWS, it is not linked with internet. If from our machine, we try to access one EC2 inside our public subnet, we are not able. But why, if we are talking about a public subnet ? To solve this need, it is required to create an Internet Gateway. Inside this Internet Gateway, we will need to create a route table and specific the range of IP´s that we want to open to connect to the VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*AbSvWWDik53rUUzBVlU27A.png" /><figcaption>Network representation</figcaption></figure><p>After network creation, we started our resources provisioning. Our application was implemented in python and to prepare our EC2 environment we preformed some dependencies installations. During EC2 resource creation, is generated an SSH Key. This SSH Key was used to access to our machine and instal dependencies like:</p><pre>sudo apt update<br>sudo apt install -y python3-dev<br>sudo apt install -y libmysqlclient-dev<br>sudo apt install -y unzip<br>sudo apt install -y libpq-dev python-dev libxml2-dev libxslt1-dev libldap2-dev<br>sudo apt-get install libsasl2-dev libffi-dev<br>curl -O  https://bootstrap.pypa.io/pip/3.6/get-pip.py ; python3 get-pip.py --user<br>export PATH=$PATH:/home/ubuntu/.local/bin/<br>pip3 install flask<br>pip3 install wtforms<br>pip3 install flask_mysqldb<br>pip3 install passlib</pre><p>To withstand our application, a Relational Database Instance was created. With RDS we are able to create relational databases like MySQL, PostgreSQL, SQLServer and Oracle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*jKIGcrV7oqlWzjvsUQmf6Q.png" /></figure><p>After resources provisioning was created, a Security Group to create a secure association between our Application and our Database. When we need to create a Security Group, we have a lot of features to take advantage. For this scenario, were created two inbound rule to open database access for a specific range of IP’s. First was created a rule to open PORT 8080 to HTTP connections to withstand database interactions from application needs. Second, was opened another PORT (22) to enable SSH connection and withstand maintaining and provisioning actions.<br>After Security Group creation, it was time to create and populate our tables with information from the on-premisses environment. To perform this action, was used a dump file with a backup of all the needed information.</p><p>When we create our Database Instance, we can also create users. During this process, a user was created to be used to connect via SSH to our Database Instance.</p><p>After Database provisioning and Database User creation, we are able to access this Instance from our machine to create our tables and populate with all we require, that’s it ? NO, we are not able ! Do you know why ? Time to shine ✨</p><p>If we came back in this text, we talked about security. Our Database Instance was created in a private subnet. This private subnet is only accessibly to our EC2 Instance. By this, I mean that we need to access our EC2 Instance first, and only then connect via SSh to our Database Instance.</p><pre>//to connect to our EC2 instance <br>//ip-publico-ec2<br>ssh -i chavessh ubuntu@ip-publico-ec2</pre><pre>//to connect to our database<br>mysql -h &lt;endpoint_do_rds&gt; -P 3306 -u admin -p</pre><pre>//to create tables and populate with dump file information<br>  show databases;<br>  use wikidb;<br>  show tables;<br>  source dump.sql;</pre><p>After VPC provisioning, EC2 preparation and RDS configuration, we were able to start our application. Connected to our EC2 Instance, our application startup was triggered.</p><p><strong>Test Solution and evidences:</strong></p><p>The access to our application was granted via the public IP of the EC2 Instance. As such, the web application was available from PORT 8080 as configured in the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cldd_tL1MptotB6LQByNjQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9BkE_5uGtp1E-2KmVai9jQ.png" /></figure><p><strong>Used AWS Resources:</strong></p><p>Virtual Private Cloud (VCP), Amazon Elastic Compute Cloud (EC2), Amazon Relational Database Service (RDS), Internet Gateway.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d9f78b17273" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated user migration and management of AWS Identity and Access Management (IAM) resources]]></title>
            <link>https://medium.com/@david_lourenco/automated-user-migration-and-management-of-aws-identity-and-access-management-iam-resources-fe97e91b1ce8?source=rss-8d45f08b600b------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe97e91b1ce8</guid>
            <category><![CDATA[aws-cloud-platform]]></category>
            <category><![CDATA[aws-iam]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[multi-cloud]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[David Lourenço]]></dc:creator>
            <pubDate>Fri, 25 Nov 2022 01:12:13 GMT</pubDate>
            <atom:updated>2022-12-15T23:32:31.126Z</atom:updated>
            <content:encoded><![CDATA[<p>In this project, based on a real-world needs, I acted as Cloud Specialist with the mission to migrate users in an automated way and manage AWS IAM (Identity and Access Management) resources.</p><p>There are 100 users with different roles and privileges that need to be migrated. For all users, Multi-factor authentication (MFA) needs to be enabled, and password security policies applied.</p><p>This project, was built to increase hands-on experience with AWS IAM, AWS CLI, WSL, and Ansible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pr78CMgUIOC2px6rFws54g.png" /><figcaption>Provider and Thecnologies</figcaption></figure><p><strong>Solution Architecture:</strong></p><p>In addition to migrating these users, it will be necessary to associate permissions to groups and users must have MFA enabled, adding an additional layer of security.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-RTS0j52RzHKrwWtakcSrw.png" /></figure><p>To avoid repetitive and manual tasks in the AWS console, I thought of a solution to automate this process. Ansible was the technology chosen to simplify this challenge.</p><blockquote>Ansible is a suite of software tools that enables infrastructure as code. It is open-source and the suite includes software provisioning, configuration management, and application deployment functionality</blockquote><p>An Excel file with all the users to be migrated was used as a starting point. For the interaction and use of Ansible, it is necessary to create Playbooks. A playbook was created, where resource provisioning and configuration was divided into a few distinct steps.<br>It required installing Ansible and Python remotely, running on a Linux WSL.</p><p><strong>Solution Implementation:</strong></p><p><strong>Step 1 </strong>— Read CSV and convert to a list:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/524/1*LOVAtuFOcubt2MhUG--C3Q.png" /></figure><p><strong>Step 2 </strong>— Create users and passwords:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*dUiMEpsRZ9auhBSjHXKf-w.png" /></figure><p><strong>Step 3</strong> — Create CloudAdminAnsible Group and Set Policies:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/1*ZXdVCwyBB87YXdUEC7iZig.png" /></figure><p><strong>Step 4</strong> — Create and set MFA policy for all groups in the list:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*gvoQZ64fNeMAiWoLxjJXIA.png" /></figure><p><strong>Step 5 </strong>— Set Password policies:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*zkbtKY2HoiGtKchR_flulA.png" /></figure><p>After Ansible playbook creation, it’s time to prepare our Linux machine to run AWS CLI and interact with AWS infrastructure. <br>As first step, we start running <em>aws configure</em> command.</p><blockquote>For general use, the aws configure command is the fastest way to set up your AWS CLI installation.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/495/1*SviJbTKC-mL6oipRJFbywQ.png" /></figure><p>To test if CLI was well configured, was used <em>aws iam list-users</em> command to try to retrieve all the AWS created users.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*HEOyoDPKuyzPGk69o94PZA.png" /></figure><p>After AWS configuration with a pre-configured user, was created a file with aws_access_key and aws_secret_key to be used as parameter on Ansible play command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*rQdTXSOYpbMpdO0eRz1Erw.png" /></figure><p>Next, was run the command where all the magic happens:<br> <em>ansible-playbook -i creds.yml ansible-hero.yml</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8eFUVLWi0xEHFpW82zQxvA.png" /></figure><p><strong>Evidences:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bK7TixYQpLuDtMuOYe2Vzg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u4weHFzzAMt086KXrZEICA.png" /></figure><p><strong>Conclusion: <br></strong>With Ansible, it was possible to solve and automate some steps that otherwise would have become repetitive and boring to do. Creating users, groups, and policies is something that is easily done using AWS IAM, but the objective of this project was to automate the process, so that the task is delivered as quickly, efficiently and with as much quality as possible. Certainly, we could migrate 100 users, create difference groups and associate them to different policies manually, but it is also true that we would easily make more mistakes and take longer to do so.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe97e91b1ce8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>