<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rajpreet Gill on Medium]]></title>
        <description><![CDATA[Stories by Rajpreet Gill on Medium]]></description>
        <link>https://medium.com/@hellomeenu1?source=rss-8ba88fe42f63------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*jvPnC0VuRoyn5a61B0tjfA.jpeg</url>
            <title>Stories by Rajpreet Gill on Medium</title>
            <link>https://medium.com/@hellomeenu1?source=rss-8ba88fe42f63------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 09:27:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hellomeenu1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Automating AWS Lambda Deployments Using AWS CodePipeline and CodeBuild]]></title>
            <link>https://medium.com/@hellomeenu1/automating-aws-lambda-deployments-using-aws-codepipeline-and-codebuild-91fdaec242d7?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/91fdaec242d7</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Tue, 17 Dec 2024 21:53:01 GMT</pubDate>
            <atom:updated>2024-12-17T22:01:15.038Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R_FYYoeDh8IFtsOYOZljJA.jpeg" /></figure><blockquote><strong>Introduction</strong></blockquote><blockquote>In this blog, we’ll explore how to automate deployments of an AWS Lambda function using <strong>AWS CodePipeline</strong> and <strong>AWS CodeBuild</strong>. By integrating code hosted on <strong>GitHub</strong>, we will create a <strong>CI/CD</strong> pipeline that automatically updates and publishes new versions of the Lambda function whenever changes are made.</blockquote><p><strong>Example</strong>:<br><em>“By automating the deployment of AWS Lambda functions with CodePipeline and CodeBuild, we can ensure faster, error-free, and seamless code updates directly from GitHub.”</em></p><blockquote><strong>What is AWS CodePipeline?</strong></blockquote><blockquote><strong>AWS</strong> <strong>CodePipeline</strong> is a fully managed <strong>CI/CD (Continuous Integration and Continuous Delivery)</strong> service that automates the software release process. It allows you to define and automate steps such as:</blockquote><blockquote><strong>Source</strong>: Fetching the code from repositories (e.g., GitHub).</blockquote><blockquote><strong>Build</strong>: Compiling, packaging, or testing the application.</blockquote><blockquote><strong>Deploy</strong>: Deploying the code to target environments (e.g., Lambda, EC2, or S3).</blockquote><blockquote><strong>What is AWS CodeBuild?</strong></blockquote><blockquote><strong>AWS CodeBuild</strong> is a fully managed <strong>build service</strong> that compiles source code, packages it, runs tests, and produces deployment-ready artifacts. CodeBuild executes <strong>commands defined in a </strong><strong>buildspec.yml file</strong>, such as:</blockquote><blockquote><strong>Zipping</strong> the code files.</blockquote><blockquote><strong>Updating</strong> an AWS Lambda function.</blockquote><blockquote><strong>Running</strong> post-deployment tasks.</blockquote><h3>How the CI/CD Pipeline Works</h3><p>Here’s how the entire CI/CD process flows for automating AWS Lambda deployments:</p><h3>1. Source Stage (GitHub Integration)</h3><ul><li>The source code (lambda_function.py and buildspec.yml) is hosted in a GitHub repository.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n8eREFRC8LCVjmbYKM2pJA.png" /></figure><blockquote><strong>Note</strong>: To integrate my <strong>GitHub repository</strong> with AWS CodePipeline, I selected <strong>GitHub (Version 2)</strong> as the source provider while creating the pipeline. AWS prompted me to authorize the <strong>AWS Connector for GitHub</strong>, granting it access to my repository Rajpreet12/ci-cd. CodePipeline then monitored the main branch for changes and automatically triggered the pipeline, pulling the updated code for deployment.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aRF7zHCLGWuPA3L851ZBxg.png" /></figure><h3>“CodePipeline continuously monitors the main branch for commits and triggers the pipeline automatically whenever changes are detected.”</h3><pre>git add lambda_function.py<br>git commit -m &quot;Updated Lambda function code&quot;<br>git push origin main</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lSc5QufEqGYKvJXKBjdaCA.png" /></figure><h3>2. Build Stage (AWS CodeBuild)</h3><p><em>“In CodeBuild, I configured the build environment as follows: Managed Image (Amazon Linux 2), Python 3.11 runtime, x86_64 architecture. I used the existing </em><em>buildspec.yml file in the GitHub repository, which defined the build and deployment commands.&quot;</em></p><pre>version: 0.2<br><br>phases:<br>  install:<br>    runtime-versions:<br>      python: 3.11<br>  build:<br>    commands:<br>      - echo &#39;Building Lambda function...&#39;<br>      - zip -r lambda_function.zip lambda_function.py<br>  post_build:<br>    commands:<br>      - echo &#39;Updating Lambda function...&#39;<br>      - aws lambda update-function-code --function-name MyDemoLambda --zip-file fileb://lambda_function.zip<br>      - echo &#39;Waiting for 30 seconds before publishing version...&#39;<br>      - sleep 30  # Add a delay to ensure update completes<br>      - echo &#39;Publishing new Lambda version...&#39;<br>      - aws lambda publish-version --function-name MyDemoLambda</pre><h3>3. Deployment to AWS Lambda</h3><ul><li><strong>Creating the Initial Lambda Function</strong>:</li><li>I started by creating a simple AWS Lambda function named <strong>MyDemoLambda</strong> using the AWS Lambda Console.</li><li>The function had a basic handler that returned a message like <strong>“Hello from MyDemoLambda!”</strong>.</li><li><strong>Setting Up CodePipeline</strong>:</li><li>I integrated my GitHub repository with AWS CodePipeline and configured it to monitor changes in the main branch.</li><li><strong>Automatic Code Updates</strong>:</li></ul><blockquote><em>“When I pushed changes to </em><em>lambda_function.py, CodePipeline triggered CodeBuild. CodeBuild packaged the updated code into a ZIP file, used the </em><em>update-function-code command to upload the new version to AWS Lambda, and then published it automatically.&quot;</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1GsL-wZl5wlXpD78fz42tg.png" /></figure><h3>Step-by-Step Workflow of the Pipeline</h3><p>Here is the visual workflow:</p><ol><li><strong>Code Changes</strong>:<br>You edit the lambda_function.py file and push it to your GitHub repository.</li><li><strong>CodePipeline (Source Stage)</strong>:<br>CodePipeline detects the commit and pulls the updated code from GitHub.</li><li><strong>CodeBuild (Build Stage)</strong>:<br>CodeBuild runs the following tasks:</li></ol><ul><li>Packages the code into a ZIP file.</li><li>Updates the AWS Lambda function with the new deployment package.</li><li>Publishes a new version of the Lambda function.</li></ul><p><strong>4. AWS Lambda</strong>:<br>The new code is deployed, and a version is published. You can test the function to confirm the update.</p><h3>Key Benefits of Using CodePipeline and CodeBuild</h3><ol><li><strong>Automation</strong>:<br>Automates the entire process of building, packaging, and deploying code.</li><li><strong>Reliability</strong>:<br>Ensures consistent deployments without manual intervention.</li><li><strong>Speed</strong>:<br>Delivers faster releases by triggering pipelines as soon as code changes are detected.</li></ol><p><strong>4. Integration</strong>:<br>Easily integrates with GitHub, Lambda, and other AWS services.</p><h3>Testing the Result:</h3><p>Once the pipeline runs successfully:</p><ol><li>Go to the <strong>AWS Lambda Console</strong>.</li><li>Open the <strong>MyDemoLambda</strong> function.</li><li>Test the function using a test event:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pMwGmuWXOXYq6NtdIJ6cNw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m_wxjEEv9z_UmMO2o0gUIw.png" /></figure><h3>Conclusion:</h3><blockquote>“This approach not only simplifies deployments but also provides a foundation for integrating automated testing and monitoring into the CI/CD pipeline for enhanced reliability.”</blockquote><p>#AWS #CloudComputing #CodePipeline #CodeBuild #Lambda #GitHub #DevOps #CICD</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91fdaec242d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ECS vs EKS: Which AWS Container Service is Right for You?]]></title>
            <link>https://medium.com/@hellomeenu1/ecs-vs-eks-which-aws-container-service-is-right-for-you-c71bddd7d01c?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/c71bddd7d01c</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Fri, 06 Dec 2024 19:20:35 GMT</pubDate>
            <atom:updated>2024-12-06T19:20:35.170Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>Introduction</strong></blockquote><blockquote>Containers have revolutionized the way we build, ship, and run applications. AWS offers two robust services to orchestrate containers: <strong>Amazon ECS</strong> (Elastic Container Service) and <strong>Amazon EKS</strong> (Elastic Kubernetes Service). While both serve the same end goal — managing containers at scale — they differ in approach, complexity, and use cases.</blockquote><blockquote>In this blog, we’ll break down the <strong>key differences</strong> between ECS and EKS, highlight their strengths, and help you choose the best option for your workloads.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aQad92QTlkZbGPffkEjjfw.png" /></figure><h3>What is ECS?</h3><p>Amazon ECS is AWS’s native container orchestration service. Think of ECS as the “easy button” for running containers in AWS. It abstracts much of the complexity of container orchestration, letting you focus on running your applications.</p><p><strong>Core Concepts</strong>:</p><ul><li><strong>Cluster</strong>: Logical grouping of tasks.</li><li><strong>Task</strong>: A running instance of a container (or multiple containers).</li><li><strong>Service</strong>: Manages and scales tasks.</li></ul><blockquote><strong>Example ECS Use Case:</strong></blockquote><blockquote><em>A startup needs to deploy a microservices-based e-commerce app quickly without worrying about underlying infrastructure.</em></blockquote><h3>What is EKS?</h3><p>Amazon EKS is AWS’s managed Kubernetes service. It lets you harness the power and flexibility of Kubernetes for container orchestration while AWS manages the control plane.</p><p><strong>Core Concepts</strong>:</p><ul><li><strong>Cluster</strong>: Includes the Kubernetes control plane and worker nodes.</li><li><strong>Pod</strong>: Smallest deployable unit, encapsulating one or more containers.</li><li><strong>Node Group</strong>: EC2 instances or Fargate profiles where pods are deployed.</li></ul><blockquote><strong>Example EKS Use Case:</strong></blockquote><blockquote><em>An enterprise with a multi-cloud strategy wants to standardize on Kubernetes to deploy a machine-learning platform across AWS and on-premises environments.</em></blockquote><h3>ECS vs EKS: A Comparison</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vlORask9ssqm5HaR3URgmA.png" /></figure><h3>When to Use ECS</h3><p>ECS is ideal for:</p><p><strong>AWS-first Workloads</strong>:</p><ul><li>Fully leveraging AWS services like S3, RDS, and DynamoDB.</li></ul><p><strong>Simple Applications</strong>:</p><ul><li>Quickly deploying microservices, batch processing, or APIs.</li></ul><p><strong>Serverless Workloads</strong>:</p><ul><li>Combining ECS with AWS Fargate for serverless container execution.</li></ul><blockquote><strong>Example</strong>: A fintech startup needs to deploy an API for processing transactions. They use ECS with Fargate for cost-efficiency and scalability.</blockquote><h3>When to Use EKS</h3><p>EKS is the right choice for:</p><p><strong>Kubernetes Workloads</strong>:</p><ul><li>Applications that already run on Kubernetes or need Kubernetes-specific features like Helm charts or service meshes.</li></ul><p><strong>Hybrid or Multi-cloud Deployments</strong>:</p><ul><li>Standardizing across environments (AWS, on-premises, or other clouds).</li></ul><p><strong>Complex Orchestration</strong>:</p><ul><li>Deploying large-scale, stateful applications like machine-learning pipelines or distributed databases.</li></ul><blockquote><strong>Example</strong>: An enterprise adopts EKS to standardize Kubernetes workloads across AWS and their on-premises VMware environment using <strong>EKS Anywhere</strong>.</blockquote><h3>Visualizing ECS vs EKS</h3><p>Here’s a visual comparison of ECS and EKS architectures:</p><h4>ECS Architecture:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*yT51T8B4Zzfvk_W4Zvfqbg.png" /></figure><p><strong>EKS Architecture</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/908/1*V0HuoTcJdSoaMS4U-zQ0_Q.png" /></figure><h3>ECS and EKS in Action</h3><p><strong>Scenario</strong>: A gaming company is deploying two workloads:</p><ol><li><strong>API Backend</strong>: Requires simple orchestration and integrates deeply with AWS services. Solution: <strong>ECS with Fargate</strong>.</li><li><strong>Game Analytics Platform</strong>: Uses custom Kubernetes operators and requires multi-cloud support. Solution: <strong>EKS</strong>.</li></ol><h3>Final Thoughts</h3><p>Both ECS and EKS are powerful container orchestration services, but the best choice depends on your specific requirements:</p><ul><li><strong>ECS</strong>: Simplicity, AWS-native workloads, and fast deployment.</li><li><strong>EKS</strong>: Kubernetes flexibility, multi-cloud strategies, and complex workloads.</li></ul><p>AWS also offers tools like <strong>AWS App Runner</strong> for even simpler deployment and <strong>Amazon Fargate</strong> for serverless compute. Your container journey starts with understanding your workload needs and scaling goals.</p><p>Are you on team <strong>ECS</strong> or <strong>EKS</strong>? Share your container journey with us. Unsure which service suits you best? Allow us to guide you through the decision-making process.</p><p>#AWS #cloudComputing #ECS #EKS #CloudWhistler #Cloudnaire</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c71bddd7d01c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Currency Normalization System with AWS CDK: A Story of Automation and Resilience]]></title>
            <link>https://medium.com/@hellomeenu1/building-a-currency-normalization-system-with-aws-cdk-a-story-of-automation-and-resilience-c41983417bb4?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/c41983417bb4</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Fri, 15 Nov 2024 19:48:52 GMT</pubDate>
            <atom:updated>2024-11-15T19:48:52.722Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>Introduction</strong></blockquote><blockquote>Data consistency is critical, especially when dealing with financial records like property sales. My project aimed to tackle a common yet complex problem: <strong>normalizing property sales data from multiple currencies into USD</strong>. Leveraging AWS services, I designed a serverless system that automates this process with high scalability and efficiency.</blockquote><blockquote>This blog narrates my journey of building the project, the architectural design, and the challenges I overcame. If you’re curious about <strong>AWS Cloud Development Kit (CDK),</strong> serverless technologies, or solving real-world cloud automation problems, this is the story for you.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GuRH_CB_JaeeQz4P" /><figcaption>Photo by <a href="https://unsplash.com/@eduschadesoares?utm_source=medium&amp;utm_medium=referral">Eduardo Soares</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Problem</h3><p>Imagine dealing with property sales data in various currencies (e.g., CAD, USD). Manually normalizing such data is tedious and error-prone. Automation could not only save time but also improve accuracy.</p><blockquote><strong>Goals:</strong></blockquote><blockquote>Automatically process property sales data <strong>uploaded</strong> to a <strong>raw bucket.</strong></blockquote><blockquote>Normalize all prices to <strong>USD.</strong></blockquote><blockquote>Store processed data <strong>securely </strong>and <strong>enable</strong> easy retrieval.</blockquote><h3>The Architecture</h3><h3>Flow Diagram</h3><p>Here’s the flow of our system:</p><ol><li><strong>Raw Properties S3 Bucket</strong>: A user uploads a CSV file containing raw property sales data.</li><li><strong>Event Trigger</strong>: The upload triggers an AWS Lambda function.</li><li><strong>Lambda Function</strong>:</li></ol><ul><li>Reads the uploaded file.</li><li>Normalizes the currency from CAD to USD.</li><li>Stores the normalized data in a DynamoDB table.</li><li>Moves the processed file to another S3 bucket.</li></ul><blockquote><strong>Processed Properties S3 Bucket</strong>: Holds the normalized files for further use.</blockquote><blockquote><strong>DynamoDB Table</strong>: Contains the structured, normalized property sales records.</blockquote><h3>Infrastructure as Code</h3><p>Using AWS CDK, I implemented this architecture as Infrastructure as Code (IaC). This approach allows for repeatable, version-controlled deployments. Here’s how I set it up:</p><ul><li><strong>Raw and Processed S3 Buckets</strong>:</li></ul><pre>raw_bucket = s3.Bucket(self, &quot;RawPropertiesBucket&quot;,<br>    removal_policy=RemovalPolicy.DESTROY,<br>    auto_delete_objects=True<br>)<br><br>processed_bucket = s3.Bucket(self, &quot;ProcessedPropertiesBucket&quot;,<br>    removal_policy=RemovalPolicy.DESTROY,<br>    auto_delete_objects=True<br>)</pre><ul><li><strong>DynamoDB Table</strong>:</li></ul><pre>table = dynamodb.Table(self, &quot;PropertiesTable&quot;,<br>    partition_key=dynamodb.Attribute(name=&quot;id&quot;, type=dynamodb.AttributeType.STRING),<br>    billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST,<br>    removal_policy=RemovalPolicy.DESTROY<br>)</pre><ul><li><strong>Lambda Function</strong>:</li></ul><pre>lambda_function = _lambda.Function(<br>    self, &quot;CurrencyNormalizer&quot;,<br>    runtime=_lambda.Runtime.PYTHON_3_9,<br>    handler=&quot;lambda_function.lambda_handler&quot;,<br>    code=_lambda.Code.from_asset(&quot;src/CurrencyNormalizer/&quot;),<br>    environment={<br>        &quot;RAW_BUCKET&quot;: raw_bucket.bucket_name,<br>        &quot;PROCESSED_BUCKET&quot;: processed_bucket.bucket_name,<br>        &quot;TABLE_NAME&quot;: table.table_name<br>    }<br>)</pre><ul><li><strong>Permissions</strong>: I granted the Lambda function permissions to interact with S3 buckets and DynamoDB:</li></ul><pre>raw_bucket.grant_read_write(lambda_function)<br>processed_bucket.grant_read_write(lambda_function)<br>table.grant_write_data(lambda_function)</pre><h3><strong>The Workflow</strong></h3><ol><li><strong>File Upload</strong>: A CSV file with raw data is uploaded to the <strong>Raw Properties S3 Bucket</strong>.</li><li><strong>Trigger</strong>: This event triggers the <strong>Lambda function</strong>.</li><li><strong>Data Processing</strong>: The Lambda function:</li></ol><ul><li>Reads the CSV file from the raw bucket.</li><li>Normalizes the currency in Lambda</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ywXZ0CyDv6hpigU8Tb8tjw.png" /></figure><blockquote><strong>Step-by-Step Currency Normalization:</strong></blockquote><blockquote><strong>Read the Input Data</strong> The raw data is uploaded to the <strong>Raw Properties S3 Bucket</strong> in .csv format. The Lambda function is triggered by an S3 event whenever a file is uploaded.</blockquote><blockquote>The Lambda function retrieves the file using the s3:GetObject API.</blockquote><blockquote>It reads and parses the .csv data to extract the necessary fields for processing (e.g., price, currency, etc.).</blockquote><blockquote><strong>Identify the Currency</strong></blockquote><blockquote>Each row in the .csv contains a currency field (e.g., &quot;CAD&quot;, &quot;USD&quot;).</blockquote><blockquote>If the currency field is already in USD, no conversion is needed.</blockquote><blockquote>If the currency is in another format (e.g., CAD), the conversion logic is applied.</blockquote><h3><strong>Sample Lambda function code:</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RZ9mEZEKAs7oaqnSxBuGtw.png" /></figure><ul><li>After run Test, Monitor Lambda’s logs in <strong>CloudWatch</strong> for validation.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3IDe3gXK3ns2zDg6Bo_cJA.png" /></figure><ul><li>Saves the normalized data to the DynamoDB table.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6V7ipE7SlNSEx2KfE-kUZw.png" /></figure><ul><li>Copies the processed file to the <strong>Processed Properties S3 Bucket</strong> and deletes it from the raw bucket.</li></ul><p><strong>4. Data Verification</strong>:</p><ul><li>Check the processed data in DynamoDB using aws dynamodb scan.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tuQUDfcUZwqdyJe3dTMCsQ.png" /></figure><ul><li>Ensure the processed file appears in the processed bucket.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zMl7s4KWfPixey-G8rS_Bw.png" /></figure><h3>Challenges and Resolutions</h3><h3>1. Permission Errors</h3><p>Initially, the Lambda function couldn’t access the S3 bucket or DynamoDB table due to insufficient permissions. By carefully crafting IAM policies, I resolved these issues. For example:</p><pre>{<br>  &quot;Effect&quot;: &quot;Allow&quot;,<br>  &quot;Action&quot;: [<br>    &quot;s3:GetObject&quot;,<br>    &quot;s3:PutObject&quot;,<br>    &quot;s3:DeleteObject&quot;,<br>    &quot;dynamodb:PutItem&quot;<br>  ],<br>  &quot;Resource&quot;: &quot;*&quot;<br>}</pre><h3>2. Debugging with Logs</h3><p>When the Lambda function failed, <strong>CloudWatch logs</strong> became my go-to tool for debugging. Issues like incorrect bucket names, DynamoDB table names, or key errors were resolved through log analysis.</p><h3>Testing the System</h3><h3>End-to-End Testing</h3><ol><li>Uploaded a file to the <strong>Raw Properties S3 Bucket</strong>:</li></ol><pre>aws s3 cp properties_data.csv s3://&lt;raw-bucket-name&gt;/</pre><p>2. Monitored the Lambda function’s execution logs in CloudWatch.</p><p>3. Queried the DynamoDB table to verify normalized data:</p><pre>aws dynamodb scan --table-name &lt;table-name&gt;</pre><p>4. Checked the <strong>Processed Properties S3 Bucket</strong> to confirm the processed file was moved successfully.</p><h3>Lessons Learned</h3><ol><li><strong>Infrastructure as Code is Powerful</strong>: AWS CDK streamlined the setup and made it easy to manage and redeploy resources.</li><li><strong>Logs Are Your Best Friend</strong>: CloudWatch logs played a crucial role in debugging.</li><li><strong>Fine-Grained Permissions Matter</strong>: Tightening IAM policies ensured security while resolving access issues.</li></ol><h3>Conclusion</h3><p>This project was a journey of learning and overcoming challenges. From designing a scalable architecture to solving IAM permission issues, every step contributed to my understanding of cloud-native solutions. The result was a robust, automated system that normalized property sales data efficiently.</p><p>If you’re building something similar, focus on logging, permissions, and testing every component. Remember, each challenge is an opportunity to learn and grow.</p><p>Here are some hashtags for your project:</p><p>#AWS #CloudComputing #Serverless #DynamoDB #LambdaFunctions #S3 #CloudDevelopment #InfrastructureAsCode #CDK #DevOps #CurrencyNormalization</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c41983417bb4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Connecting Ubuntu to AWS S3 Using IAM Roles, Access Keys, and Secret Keys]]></title>
            <link>https://medium.com/@hellomeenu1/connecting-ubuntu-to-aws-s3-using-iam-roles-access-keys-and-secret-keys-61b05426370a?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/61b05426370a</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Mon, 04 Nov 2024 04:07:58 GMT</pubDate>
            <atom:updated>2024-11-06T06:35:02.565Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Introduction</strong> In today’s cloud-based environment, efficient interaction between on-premises systems and cloud services is crucial. This guide provides a comprehensive walkthrough on how to connect an Ubuntu system to AWS S3 using IAM roles, access keys, and secret keys, empowering users to manage their S3 buckets securely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*twQFlgPj_G5NArcLzM5SVQ.png" /></figure><blockquote><strong>What is Ubuntu?</strong></blockquote><blockquote>Ubuntu is a popular, open-source Linux-based operating system known for its user-friendliness, stability, and extensive community support. It is widely used for both personal computing and server environments due to its robust security features and compatibility with various software applications. Ubuntu is often chosen for cloud-based tasks because it offers flexibility, ease of use, and seamless integration with development tools.</blockquote><blockquote><strong>What is Python?</strong></blockquote><blockquote>Python is a versatile, high-level programming language that is popular for its readability and ease of use. It is widely adopted in cloud computing for automation, data manipulation, and server-side operations due to its extensive libraries and frameworks. With the help of Boto3, the official AWS SDK for Python, developers can interact with AWS services programmatically, making Python an excellent choice for automating cloud tasks such as managing S3 buckets and securely handling IAM roles.</blockquote><p><strong>Step 1: Setting Up an IAM User with Access Keys</strong> To establish a secure connection between your Ubuntu system and AWS, you need an IAM user with appropriate permissions. Follow these steps:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kQO79N2MZKhWuOX-puqYXw.png" /></figure><p><strong>Create an IAM User</strong>:</p><ul><li>Log in to the AWS Management Console.</li><li>Navigate to <strong>IAM (Identity and Access Management)</strong>.</li><li>Create a new user (e.g., ‘Test User’) and select “Programmatic access” as the access type.</li><li>Attach the “AmazonS3FullAccess” policy or create a custom policy with the necessary permissions.</li></ul><p><strong>Generate Access Keys</strong>:</p><ul><li>After creating the user, download the access key and secret key. These keys are essential for programmatic authentication and connecting to AWS services from your system.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ghDWVvvJ_Tkb1cl3UETGcg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HqYaps9Ou4eOh0lxQnSJog.png" /></figure><blockquote><strong>Note</strong>: Under the user that was created, select the option &#39;Security Credentials&#39; to create an access key.</blockquote><p><strong>Create a New IAM Role for AssumeRole:</strong></p><ul><li>In the IAM console, go to <strong>Roles</strong> &gt; <strong>Create role</strong>.</li><li>Choose <strong>Another AWS account</strong> as the trusted entity.</li><li>Enter your <strong>Account ID</strong>.</li><li>In <strong>Policies</strong>, attach the <strong>AmazonS3FullAccess</strong> policy to allow this role to interact with S3.</li><li>Give the role a name (e.g., “pythonrole”) and create it.</li></ul><p><strong>Edit the Role Trust Relationship:</strong></p><ul><li>Go to the role you just created (e.g., “pythonrole”).</li><li>Select the <strong>Trust relationships</strong> tab.</li><li>Click <strong>Edit trust relationship</strong> and add your user ARN in the Principal section like this:</li></ul><pre>{<br>  &quot;Version&quot;: &quot;2012-10-17&quot;,<br>  &quot;Statement&quot;: [<br>    {<br>      &quot;Effect&quot;: &quot;Allow&quot;,<br>      &quot;Principal&quot;: {<br>        &quot;AWS&quot;: &quot;arn:aws:iam::YOUR_ACCOUNT_ID:user/TestUser&quot;<br>      },<br>      &quot;Action&quot;: &quot;sts:AssumeRole&quot;<br>    }<br>  ]<br>}</pre><blockquote><strong>Note</strong>: Replace YOUR_ACCOUNT_ID with your actual AWS account ID.</blockquote><p><strong>Step 2: Installing AWS CLI on Ubuntu</strong> To interact with AWS services from Ubuntu, you need to install the AWS Command Line Interface (CLI).In my case, I used VSCode with WSL Ubuntu to create it.</p><pre>curl &quot;https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip&quot; -o &quot;awscliv2.zip&quot;<br>unzip awscliv2.zip<br>sudo ./aws/install</pre><p><strong>Step 3: Configuring AWS CLI with Access Keys</strong> Configure your AWS CLI to use the access keys obtained earlier:</p><pre>aws configure</pre><p>Enter the following details when prompted:</p><blockquote><strong>AWS Access Key ID</strong>: Your generated access key.</blockquote><blockquote><strong>AWS Secret Access Key</strong>: Your generated secret key.</blockquote><blockquote><strong>Default region name</strong>: Choose a region (e.g., us-east-1).</blockquote><blockquote><strong>Default output format</strong>: Typically json.</blockquote><p><strong>Step 4: Connecting to S3 Using Python and Boto3</strong> Boto3 is the official AWS SDK for Python, enabling programmatic interactions with AWS services:</p><ol><li><strong>Install Boto3</strong>:</li></ol><pre>pip3 install boto3</pre><ol><li><strong>Python Script to Connect to S3</strong>: Use the following script to assume an IAM role, connect to S3, and perform operations like bucket creation and file uploads:</li></ol><pre>import boto3<br><br># Create an S3 client using access keys directly<br>s3_client = boto3.client(<br>    &#39;s3&#39;,<br>    aws_access_key_id=&#39;YOUR_ACCESS_KEY&#39;,<br>    aws_secret_access_key=&#39;YOUR_SECRET_KEY&#39;,<br>    region_name=&#39;us-east-1&#39;<br>)<br><br># List buckets<br>response = s3_client.list_buckets()<br>print(&#39;Buckets:&#39;, [bucket[&#39;Name&#39;] for bucket in response[&#39;Buckets&#39;]])</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1AFUwZtwD96Aupq2tQ36mw.png" /></figure><p><strong>Step 5: Securing Your Credentials</strong> For best practices:</p><blockquote><strong>Avoid hard-coding credentials</strong> in scripts; use environment variables or IAM roles when running on AWS services.</blockquote><blockquote><strong>Store credentials securely</strong> using tools like AWS Secrets Manager.</blockquote><p><strong>Conclusion</strong> Connecting Ubuntu to AWS S3 using IAM roles and access keys ensures secure and efficient interaction with cloud resources. By following these steps, you can confidently set up, configure, and manage S3 buckets from your Ubuntu system.</p><p>#AWS #Ubuntu #S3 #IAM #Python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61b05426370a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Scalable and Highly Available AWS Portfolio Website]]></title>
            <link>https://medium.com/@hellomeenu1/building-a-scalable-and-highly-available-aws-portfolio-website-be403c1f2411?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/be403c1f2411</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Mon, 21 Oct 2024 21:02:33 GMT</pubDate>
            <atom:updated>2024-10-21T21:02:33.599Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sC87nkAy8Exg85FTVkskSw.png" /></figure><blockquote><strong>In this mini project</strong>, I created a highly scalable and fault-tolerant portfolio website on AWS, showcasing my certifications and dynamic blog posts. The project focused on migrating a traditional monolithic website into a cloud-native architecture using several AWS services. This post will cover the key steps and technologies used in the project, including <strong>EC2</strong>, <strong>S3</strong>, <strong>Lambda</strong>, <strong>SNS</strong>, <strong>Route 53</strong>,<strong>IAM</strong> and <strong>CloudFront</strong>, <strong>DynamoDB</strong> and how they contributed to building a reliable and performant solution.</blockquote><h4>Project Overview</h4><blockquote><strong>The Challenge:</strong> The client, a startup, had been running a monolithic website on a single server for over a decade. Maintaining the site was becoming difficult due to performance issues and the risks associated with every change made to the application. To modernize the infrastructure and make it more reliable and scalable, the client wanted to migrate to AWS. This required splitting the application into microservices and leveraging cloud-native architectures to ensure the site was always available.</blockquote><h3>Technologies Used and Why</h3><h4>1. EC2 (Elastic Compute Cloud):</h4><ul><li><strong>Why We Used It:</strong> EC2 is a scalable virtual server that allows us to deploy and manage applications in the cloud. For the initial stage of the migration, EC2 was chosen to host the website as it closely mirrored the on-premise environment the client was familiar with, providing an easy first step in the migration.</li><li><strong>How We Used It:</strong> We launched an EC2 instance using a pre-configured Amazon Machine Image (AMI) that contained the portfolio website files (HTML, CSS, and JavaScript). This included important pages like the main portfolio, blog, AWS certifications, and contact form.</li></ul><p><strong>Important Steps in EC2 Setup:</strong></p><ul><li><strong>VPC (Virtual Private Cloud):</strong> We configured a VPC to isolate our EC2 instance within a secure network. The VPC allowed us to control traffic in and out of our instance.</li><li><strong>Security Group:</strong> We created a security group that allowed incoming traffic on port 80 (HTTP) and 22 (SSH), ensuring that the website was accessible and that we could SSH into the instance for management.</li><li><strong>Key Pair:</strong> A key pair was generated to securely SSH into the EC2 instance. The private key file (labpair-key.pem) was used to connect to the instance.</li></ul><blockquote><strong>IAM Role:</strong> An IAM role was attached to the EC2 instance, granting the instance necessary permissions, such as:</blockquote><blockquote><strong>SNS Full Access</strong>: For publishing notifications to SNS.</blockquote><blockquote><strong>S3 Full Access</strong>: To allow the instance to interact with S3 for hosting assets and managing files.</blockquote><blockquote><strong>CloudWatch Full Access</strong>: For monitoring and logging metrics.</blockquote><blockquote><strong>EC2 Full Access</strong>: For complete control over EC2 instances.</blockquote><ul><li>After SSH into the instance, we modified and updated pages, such as certs.html, using:</li></ul><pre>sudo nano /var/www/html/certs.html</pre><ul><li>This allowed us to customize the website’s content and display the certifications dynamically.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1XJMwevyiofkS7UUnkDdvQ.png" /></figure><h4>2. S3 (Simple Storage Service):</h4><ul><li><strong>Why We Used It:</strong> S3 is an object storage service that offers industry-leading scalability, availability, and security. We used S3 to store static assets (such as images and blog posts), and later, as part of the serverless migration, to host the static version of the website.</li><li><strong>How We Used It:</strong> S3 hosted assets such as images for certifications. It also received YAML files for blog posts, which were then processed by Lambda functions to generate HTML content for the blog section. Once processed, the posts were stored back in S3 in a format that the website could display dynamically.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oIL7ZjvyHL7ff5hROWxtDA.png" /></figure><h4>3. Lambda (Serverless Compute):</h4><ul><li><strong>Why We Used It:</strong> Lambda enables running code without provisioning or managing servers. This is perfect for processing events (like new blog posts or form submissions) because it scales automatically and only incurs costs when the function is triggered.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B0Bk4Ut-YMedcOMFuFRGmA.png" /></figure><ul><li><strong>How We Used It:</strong> Lambda functions were used for:</li></ul><blockquote><strong>Blog Post Processing:</strong></blockquote><blockquote>When a <strong>YAML</strong> file was uploaded to <strong>S3</strong>, the<strong> Lambda</strong> function processed and transformed it into <strong>HTML</strong> using a pre-configured function. This allowed the blog posts to be dynamically added to the website without embedding server-side logic into the<strong> EC2 </strong>instance itself. The processed<strong> HTML</strong> was stored back in S3, ready to be displayed on the blog section of the site.</blockquote><blockquote><strong>Contact Form Processing:</strong></blockquote><blockquote>The contact form on the website was powered by a Lambda function, which captured the form data (name, email, message) submitted by users. This data was then sent via <strong>Amazon SNS</strong> (Simple Notification Service), triggering an email notification to me with the details of the submission, ensuring timely communication with visitors of the website.</blockquote><blockquote><strong>DynamoDB Integration:</strong> DynamoDB was used to store additional data dynamically, such as website interactions or blog-related metadata. Lambda functions were configured to retrieve and update this data in DynamoDB, ensuring real-time interaction with the database when required.</blockquote><h4>4. SNS (Simple Notification Service):</h4><ul><li><strong>Why We Used It:</strong> SNS is used to send messages (notifications) to subscribers via email, SMS, or other endpoints. It ensures reliable and scalable delivery of messages from the application to the user.</li><li><strong>How We Used It:</strong> SNS was integrated with the contact form on the portfolio site. When a user filled out the form, the Lambda function triggered SNS, which sent an email notification with the submission details. This automated the contact process, making it seamless for users to get in touch.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pJPOou3jzqMWqcflVQUhsQ.png" /></figure><h4>5. Route 53 (DNS Service):</h4><ul><li><strong>Why We Used It:</strong> Route 53 is a scalable DNS service that provides reliable and low-latency routing of end-user traffic to AWS resources. We used it to manage domain name resolution and routing for our custom domain.</li><li><strong>How We Used It:</strong> Route 53 managed the DNS records for the domain <em>innovatewithraj.org</em>. It allowed me to create an alias record that pointed to the Application Load Balancer (ALB), ensuring that all traffic to the domain was directed to the correct resources.</li><li>The DNS configuration allowed the domain to be reachable globally:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Tw_Nl3hJoZ364utBs-n2Q.png" /></figure><pre>curl http://innovatewithraj.org</pre><h4>6. Application Load Balancer (ALB):</h4><ul><li><strong>Why We Used It:</strong> ALB is used to distribute incoming traffic across multiple targets, such as EC2 instances, to improve the availability and fault tolerance of the application.</li><li><strong>How We Used It:</strong> In the initial phase of the project, the ALB distributed traffic between EC2 instances, ensuring that the portfolio website could handle varying traffic loads without becoming overwhelmed. The ALB also worked seamlessly with Route 53 to route domain traffic to the correct EC2 instances.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*777HUHKhwIrbUA5lf9RDPA.png" /></figure><h4>7. CloudFront (Content Delivery Network):</h4><ul><li><strong>Why We Used It:</strong> CloudFront is a CDN that distributes content to users globally with low latency and high transfer speeds. It is essential for improving the performance and speed of websites with users spread across multiple regions.</li><li><strong>How We Used It:</strong> CloudFront was used to cache and deliver the website’s content from the nearest edge locations to users. This reduced latency and ensured faster load times for visitors accessing the website from different geographical locations.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6H3nRlAOTmL__m86YSagkQ.png" /></figure><h4>8. DynamoDB Integration:</h4><p>In addition to S3 and Lambda, DynamoDB was used to manage and store specific data. This data-driven approach leveraged DynamoDB for fast, scalable storage, providing high availability and performance for the AWS Latest News page and other dynamic elements in the application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*428z_hKdtfkGjrcgB8UJbA.png" /></figure><h3>Stage 1: EC2-Based Website Deployment</h3><p>I first deployed the portfolio website on an EC2 instance, allowing easy management and customization of the site. I SSH’d into the instance, uploaded assets such as certifications, and integrated the necessary microservices (blog, contact form, etc.).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*95DNl05fRCfH1eZ8Nq88Mw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9B9pHBRSRqsKkinJSpvbSA.png" /></figure><h3>Stage 2: Migrating to Serverless</h3><p>In the second phase, I migrated the static content from EC2 to S3, setting up the site as an S3-hosted static website. CloudFront was used to optimize content delivery, and Route 53 was updated to route traffic directly to the CloudFront distribution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7jWHJPLWsG6X5W8wNJeVHA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9B9pHBRSRqsKkinJSpvbSA.png" /></figure><h3>Conclusion</h3><p>This project demonstrated how AWS services can be used to modernize and optimize a legacy website, making it highly available, scalable, and fault-tolerant. By leveraging services like EC2, Lambda, IAM,S3, SNS, CloudFront, and Route 53,DynamDB. I was able to create a portfolio website that is both reliable and scalable, with minimal management overhead.</p><p>Next, I plan to continue refining the website’s architecture, focusing on further cost optimization by completing the transition to a fully serverless environment.</p><p>#AWS #CoudComputing #EC2 #S3 #CloudFront #Lambda #Route53 #SNS</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=be403c1f2411" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implement Secure Authentication in Your Application with Amazon Cognito]]></title>
            <link>https://medium.com/@hellomeenu1/implement-secure-authentication-in-your-application-with-amazon-cognito-994d67629342?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/994d67629342</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Tue, 01 Oct 2024 23:55:06 GMT</pubDate>
            <atom:updated>2024-10-01T23:55:06.282Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><blockquote>Imagine building a to-do list application where users can create and manage their tasks effortlessly. But here’s the challenge: how do you securely manage user sign-ups and logins without having to build a complex authentication system from scratch? This is where <strong>Amazon Cognito</strong> comes in — your gateway to a secure, scalable, and easy-to-use user authentication system for both web and mobile applications.</blockquote><blockquote>In this post, we’ll walk through how Amazon Cognito can be used to secure a simple to-do list dashboard application, giving users the ability to log in safely using a pre-built hosted UI. No headaches, no managing passwords — just a smooth, secure experience.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qe9fDqdmAv79RvB-GDGz5w.png" /></figure><h3><strong>Why Amazon Cognito?</strong></h3><p>Amazon Cognito helps you handle user management (like sign-ups, sign-ins, and user profiles) with minimal effort. With Cognito, you get:</p><p><strong>💫 Secure Authentication</strong>: Integration with OAuth2, SAML, and other identity standards.</p><p><strong>💫 Scalable User Management</strong>: Cognito scales automatically to handle millions of users.</p><p><strong>💫 Hosted UI</strong>: A pre-built UI for sign-in and sign-up to simplify integration.</p><p>With Cognito, you don’t need to worry about storing passwords or designing secure authentication flows — AWS takes care of it for you.</p><h3><strong>Step-by-Step: Using Amazon Cognito in a To-Do List Dashboard</strong></h3><p>Let’s take a look at how we integrated Amazon Cognito into a <strong>simple to-do list dashboard</strong> to make it secure.</p><h3>1. The Project Overview</h3><p>The objective is to ensure secure access to a to-do list dashboard constructed with <strong>HTML, JavaScript</strong>, and <strong>Bootstrap</strong>. The dashboard, as depicted in the file, provides a straightforward interface for users to <strong>create</strong>, <strong>view</strong>, and<strong> manage</strong> their to-do items.</p><pre>    &lt;!DOCTYPE html&gt;<br>    &lt;html lang=&quot;en&quot;&gt;<br>    &lt;head&gt;<br>        &lt;meta charset=&quot;UTF-8&quot;&gt;<br>        &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot;&gt;<br>        &lt;title&gt;MyDashboard&lt;/title&gt;<br>        &lt;link href=&quot;https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css&quot; rel=&quot;stylesheet&quot;&gt;<br>        &lt;link rel=&quot;stylesheet&quot; href=&quot;style.css&quot;&gt;<br>        &lt;style&gt;<br>            /* Custom styles */<br>            body {<br>                padding-top: 20px;<br>            }<br>            .todo-item {<br>                margin-bottom: 5px;<br>            }<br>        &lt;/style&gt;<br>    &lt;/head&gt;<br>    &lt;body&gt;<br>    &lt;div class=&quot;container&quot;&gt;<br>        &lt;div class=&quot;jumbotron&quot;&gt;<br>        &lt;div class=&quot;float-right&quot;&gt;<br>            &lt;button id=&quot;signOut&quot; class=&quot;btn btn-outline-secondary&quot;&gt;Sign Out&lt;/button&gt;<br>        &lt;/div&gt;<br>        &lt;h1 class=&quot;display-4&quot; id=&quot;greeting&quot;&gt;Welcome to My to-do-list!&lt;/h1&gt;<br>        &lt;p class=&quot;lead&quot;&gt;This is a simple personal dashboard app where you can manage your tasks.&lt;/p&gt;<br>    &lt;/div&gt;<br>        &lt;!-- To-Do List Section --&gt;<br>        &lt;div class=&quot;todo-section&quot;&gt;<br>            &lt;h2&gt;To-Do List&lt;/h2&gt;<br>            &lt;div id=&quot;todoList&quot; class=&quot;list-group&quot;&gt;<br>                &lt;!-- To-Do items will be added here dynamically --&gt;<br>            &lt;/div&gt;<br>            &lt;div class=&quot;input-group mb-3&quot;&gt;<br>                &lt;input type=&quot;text&quot; id=&quot;todoInput&quot; class=&quot;form-control&quot; placeholder=&quot;Add new task&quot;&gt;<br>                &lt;div class=&quot;input-group-append&quot;&gt;<br>                    &lt;button class=&quot;btn btn-outline-secondary&quot; type=&quot;button&quot; id=&quot;addTodo&quot;&gt;Add&lt;/button&gt;<br>                &lt;/div&gt;<br>            &lt;/div&gt;<br>        &lt;/div&gt;<br>    &lt;/div&gt;<br><br>    &lt;script src=&quot;https://code.jquery.com/jquery-3.5.1.slim.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script src=&quot;https://cdn.jsdelivr.net/npm/@popperjs/core@2.5.4/dist/umd/popper.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script src=&quot;https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js&quot;&gt;&lt;/script&gt;<br>    &lt;script src=&quot;app.js&quot;&gt;&lt;/script&gt; &lt;!-- Link to the JavaScript file --&gt;<br>    &lt;/body&gt;<br>    &lt;/html&gt;</pre><blockquote><strong>Note</strong>: To view the file (index.html) in a browser, run it through the Live Server extension in VScode. It will display as shown below.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x4TEu3zMZ0SJJjmKOD3kWw.png" /></figure><h3>2. Setting Up Cognito User Pool</h3><p>To manage user authentication, an Amazon Cognito User Pool is set up — a directory that assists in user management. Here’s the process:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pCWTKErP0VS3u5KR0cxKZw.png" /></figure><ol><li>Access the <strong>AWS Management Console </strong>and go to the <strong>Amazon Cognito</strong> <strong>service</strong>.</li><li>Click on <strong>‘Create a user pool</strong>’, then select ‘<strong>Email</strong>’.</li><li>Proceed by clicking ‘<strong>Next</strong>’, opt for Cognito defaults, choose ‘<strong>No MFA</strong>’, and keep the remaining defaults. Continue by clicking <strong>‘Next</strong>’ twice.</li><li>For the email provider, choose ‘<strong>Send Email with Cognito</strong>’ and proceed.<br>Name your pool ‘<strong>ApplicationUserPool’</strong> and check the option ‘Use the Cognito Hosted UI’. Select a Cognito Domain and enter a preferred domain name.</li></ol><blockquote><strong>Example</strong>: In my situation, I have the<strong> domain name</strong> “<strong>InnovateWithRaj</strong>” and have chosen a <strong>pre-configured domain </strong>to host the UI <strong>(innovatewithraj.auth.us-east-1.amazoncognito.com).</strong></blockquote><blockquote><strong>Note</strong>: For guidance on creating a <strong>domain</strong>, refer to my other<strong> blog post</strong>, “<strong>Exploration on Amazon Route 53: A Comprehensive Guide with Examples</strong>.”</blockquote><p>5. Name your App Client with any desired name.</p><p>6. For the callback URL, use the ‘<strong>Go Live</strong>’ feature on the `<strong>index.html</strong>` file within VScode, copy the URL from the address bar, and set it as the <strong>callback URL</strong>, replacing<strong> 127.0.0.1</strong> with ‘<strong>localhost</strong>’, e.g., <a href="http://localhost:5500/index.html"><strong>http://localhost:5500/index.html</strong></a><strong>.</strong></p><p>7. In the Advanced app client settings and<strong> OAuth 2.0</strong> grant types, select <strong>‘Implicit grant</strong>’ and deselect ‘<strong>Authorization code grant</strong>’.<br>8. Finally, click ‘<strong>Next</strong>’ and select <strong>‘Create pool</strong>’.</p><h3>3. Login Button: Redirecting to Cognito</h3><p>In the login.js file, we’ve configured an event listener for the <strong>Cognito sign-in button</strong>:</p><ul><li>When a user clicks the sign-in button, they are redirected to the Cognito-hosted login page, where they authenticate.</li><li>This is done using a pre-built URL that includes parameters like <strong>client ID</strong>, <strong>redirect URI</strong>, and <strong>response type</strong> (in our case, token)​(login).</li></ul><pre>document.addEventListener(&#39;DOMContentLoaded&#39;, function() {<br> // Check if the user is coming back from Cognito with authentication tokens<br> checkAuthentication();<br><br> // Event listener for the Cognito sign-in button<br> document.getElementById(&#39;cognitoSignIn&#39;).addEventListener(&#39;click&#39;, redirectToCognitoSignIn);<br>});<br><br>function redirectToCognitoSignIn() {<br> // Construct the Cognito hosted UI URL<br> const clientId = &#39;2hilv11a1gumem5f04u23624gg&#39;;<br> const responseType = &#39;token&#39;; // Use &#39;token&#39; for Implicit grant<br> const redirectUri = encodeURIComponent(&#39;http://localhost:5500/index.html&#39;); // make sure it&#39;s HTTP not HTTPS<br> const cognitoDomain = &#39;innovatewithraj.auth.us-east-1.amazoncognito.com&#39;; // do not copy the HTTPS:// part just the domain<br><br> const cognitoUrl = `https://${cognitoDomain}/login?response_type=${responseType}&amp;client_id=${clientId}&amp;redirect_uri=${redirectUri}`;<br><br> // Redirect to the Cognito sign-in page<br> window.location.href = cognitoUrl;<br>}<br><br>function checkAuthentication() {<br> const urlParams = new URLSearchParams(window.location.hash.substring(1)); // Parsing the hash by removing &#39;#&#39;<br> const idToken = urlParams.get(&#39;id_token&#39;);<br><br> if (idToken) {<br>     // Store the token in local storage or session storage<br>     localStorage.setItem(&#39;id_token&#39;, idToken);<br><br>     // Redirect to index.html since the user is authenticated<br>     window.location.href = &#39;index.html&#39;;<br> } else {<br>     // User is not authenticated, stay on the login page or handle accordingly<br> }<br>}</pre><blockquote><strong>Note</strong>: In the login.js file, update the congnitoDomain, client ID, and redirect URI within the Applicationuserpool that we have established under the App Integration.</blockquote><h3>4. Authentication Flow</h3><p>Once users sign in through Cognito, they are redirected back to our application (index.html) with an <strong>authentication token</strong> in the URL hash. We then parse this token to verify the user and manage access:</p><ul><li>The token is stored in <strong>local storage</strong>, allowing us to keep track of whether the user is authenticated.</li><li>Once authenticated, users can see their to-do list and interact with the dashboard securely.</li></ul><p><strong>Snippet for Token Management</strong>:</p><pre>function checkAuthentication() {<br>    const urlParams = new URLSearchParams(window.location.hash.substring(1));<br>    const idToken = urlParams.get(&#39;id_token&#39;);<br><br>    if (idToken) {<br>        localStorage.setItem(&#39;id_token&#39;, idToken);  // Store the ID token<br>        window.location.href = &#39;index.html&#39;;  // Redirect to the dashboard<br>    } else {<br>        // User is not authenticated<br>    }<br>}</pre><p>This simple authentication flow lets users access the dashboard securely, ensuring that only logged-in users can manage their tasks.</p><blockquote><strong>Note</strong>: After running the live server from index.html and signing in with Cognito, the output will appear as follows.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t3-JcZWRcp99se-MC7pvqA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PZ9A5vPs5GHu8fL0aKlpbQ.png" /></figure><h3>5. Benefits of Using Cognito for Your App</h3><p>Using Cognito for our app brought several advantages:</p><ul><li><strong>Security</strong>: Cognito handled password policies, multi-factor authentication, and secure login flows, ensuring our app was well protected.</li><li><strong>Speed of Integration</strong>: By using the hosted UI and Cognito API, we reduced the time to implement user authentication.</li><li><strong>Scalability</strong>: Amazon Cognito scales automatically, so our to-do list app can handle new users as it grows without us worrying about performance.</li></ul><h3>Conclusion</h3><p>In this post, we explored how <strong>Amazon Cognito</strong> can easily integrate into a simple web app to provide a secure user authentication experience. Whether you’re building a to-do list app, a personal dashboard, or a complex web platform, Cognito offers an easy, scalable solution for user management.</p><p>#CloudWhistler #Cloudnaries #AWS #CloudComputing #AmazonCognito</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=994d67629342" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Winning the CloudWhistler Blog Challenge: A Comprehensive Step-by-Step Guide]]></title>
            <link>https://medium.com/@hellomeenu1/winning-the-cloudwhistler-blog-challenge-a-comprehensive-step-by-step-guide-86cc68d5a714?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/86cc68d5a714</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Tue, 24 Sep 2024 02:14:20 GMT</pubDate>
            <atom:updated>2024-09-24T02:14:20.187Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to Win the CloudWhistler Blog Challenge: A Step-by-Step Guide</h3><p>Participating in the <strong>CloudWhistler blog challenge</strong> is not just about showcasing your cloud knowledge, it’s about creativity, engagement, and making your voice stand out among the crowd. Whether you’re passionate about AWS, Azure, or Google Cloud, crafting the perfect blog that wins over judges and resonates with the community is key. As a winner on the list, Here are some tips and strategies to help you succeed and win the CloudWhistler challenge:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/799/1*efCZ202uD4c8fwomVoTBYQ.png" /></figure><h3>1. Choose a Unique Cloud Topic</h3><p>Winning the challenge starts with selecting a unique and relevant topic. Many participants will write about generic cloud concepts, so <strong>choosing a niche</strong> topic or diving deep into a specific cloud service can make your blog stand out.</p><h4>How to Find Your Winning Topic:</h4><ul><li><strong>Focus on Emerging Technologies</strong>: Write about newer AWS services like <strong>AWS CodeCatalyst</strong>, or <strong>Azure Arc</strong>, or innovations in <strong>multi-cloud strategies</strong>.</li><li><strong>Solve Real-World Problems</strong>: Show how a specific cloud solution solves a complex business problem. For example, explain how <strong>serverless architecture</strong> can help scale a startup.</li><li><strong>Case Studies</strong>: Share real-life experiences or case studies of implementing cloud solutions in specific industries such as healthcare, finance, or e-commerce.</li></ul><p><strong>Pro Tip</strong>: Before finalizing your topic, research what’s already out there. If the topic has been overdone, find a new angle, like offering insights into common pitfalls and how to avoid them.</p><h3>2. Tell a Compelling Story</h3><p>CloudWhistler judges and readers will look for more than technical details. They want stories that connect. Make your blog <strong>relatable</strong>, whether through personal experiences, anecdotes, or by walking your audience through a real-world problem and solution.</p><h4>How to Tell a Story with Your Blog:</h4><ul><li><strong>Start with a Problem</strong>: Begin your blog with a scenario, perhaps a business facing scalability issues or a startup unsure about data security. Frame the <strong>cloud solution</strong> as the hero of your story.</li><li><strong>Use a Narrative Arc</strong>: Take the reader on a journey where you describe the challenges, the solution (cloud technology), and the outcome. Show the transformation powered by the cloud.</li><li><strong>Use Visuals and Examples</strong>: Include diagrams, charts, or even screenshots to break down complex ideas into digestible content. Use storytelling techniques that make cloud topics easier to grasp.</li></ul><p><strong>Pro Tip</strong>: Use the <strong>hero’s journey</strong> structure: Present a challenge, show how the cloud becomes the hero, and conclude with how it transforms the business.</p><h3>3. Demonstrate Hands-On Expertise</h3><p>Anyone can talk about cloud concepts, but demonstrating <strong>hands-on knowledge</strong> sets you apart. Cloud Whistler&#39;s audience is full of technical experts, and they appreciate when you dive into practical steps.</p><h4>How to Showcase Practical Expertise:</h4><ul><li><strong>Step-by-Step Guides</strong>: Write how-toss or walkthroughs on deploying cloud resources, automating with serverless, or securing multi-cloud environments.</li><li><strong>Code Samples and Scripts</strong>: Include snippets of Python, YAML, or Terraform scripts that solve real problems. Readers appreciate being able to implement what you’ve shared directly.</li><li><strong>Include Performance Metrics</strong>: If you’re showcasing a cloud optimization solution, back it up with <strong>before-and-after performance metrics</strong>. Showing how you improved latency or reduced costs with an AWS Lambda solution will resonate well.</li></ul><p><strong>Pro Tip</strong>: Take screenshots of your cloud console or infrastructure diagrams that show the architecture you’re describing in your blog.</p><h3>4. Optimize for SEO and Engagement</h3><p>Winning the challenge means your blog needs to reach a wider audience, and that starts with <strong>search engine optimization (SEO)</strong> and engaging your readers.</p><h4>How to Optimize for SEO and Reader Engagement:</h4><ul><li><strong>Use Keywords Strategically</strong>: Include relevant keywords like “AWS serverless,” “Kubernetes automation,” or “Azure security” to help your blog appear in search results.</li><li><strong>Short and Engaging Sentences</strong>: Keep sentences and paragraphs short to improve readability. Break up your blog with headings, bullet points, and visuals.</li><li><strong>Call to Action (CTA)</strong>: End your blog with a CTA to engage your readers. Whether it’s inviting comments, encouraging readers to try a cloud tool, or providing a downloadable guide, interaction boosts visibility.</li><li><strong>Promote on Social Media</strong>: Share your blog post across LinkedIn, Twitter, and relevant forums like Dev.to or Reddit’s cloud subreddits. The more readers you drive to your blog, the more likely it will get noticed.</li></ul><p><strong>Pro Tip</strong>: Use tools like <strong>Google Keyword Planner</strong> to research what terms people are searching for related to your topic. Use these keywords to shape your title, headers, and content.</p><h3>5. Emphasize Innovation and Thought Leadership</h3><p>The <strong>CloudWhistler blog challenge</strong> is all about showcasing what makes your approach unique. Judges want to see that you’re a <strong>thought leader</strong> in the space and that your blog brings <strong>new insights</strong>.</p><h4>How to Highlight Innovation:</h4><ul><li><strong>Predict the Future</strong>: Include predictions on where the cloud industry is heading. Will multi-cloud architecture dominate the next decade? Will <strong>AI-driven cloud management</strong> replace manual operations?</li><li><strong>Highlight Industry Trends</strong>: Discuss how emerging trends like <strong>edge computing</strong>, <strong>AI-driven cloud optimization</strong>, or <strong>sustainability in cloud computing</strong> are shaping businesses.</li><li><strong>Offer Opinions</strong>: Don’t just explain how a technology works — offer opinions on whether certain trends are worth the hype or how they might evolve. Show that you’re thinking critically about the future of cloud.</li></ul><p><strong>Pro Tip</strong>: Finish your blog with <strong>actionable advice</strong>. For example, “Start by building a small-scale serverless architecture today, and you’ll be prepared for the next phase of cloud evolution.”</p><h3>Conclusion: Bring It All Together</h3><p>Winning the CloudWhistler blog challenge takes more than just technical knowledge — you need to <strong>tell a story</strong>, <strong>engage your audience</strong>, and <strong>demonstrate expertise</strong> in a way that sets you apart. By focusing on a niche topic, using real-world examples, and incorporating hands-on steps, you can craft a blog that not only wins the competition but also helps grow your presence in the cloud community.</p><p>Remember to keep it fresh, keep it actionable, and most importantly, keep it <strong>cloudy</strong>!</p><p>#AWS #CloudComputing #Cloudwhistler #cloudnaires</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=86cc68d5a714" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CrowdStrike Chaos: How a Software Update Brought Airports and Banks to Their Knees]]></title>
            <link>https://medium.com/@hellomeenu1/crowdstrike-chaos-how-a-software-update-brought-airports-and-banks-to-their-knees-d7c0e289076c?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/d7c0e289076c</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Sun, 22 Sep 2024 22:06:45 GMT</pubDate>
            <atom:updated>2024-09-22T22:06:45.471Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine waking up on a normal day, grabbing a cup of coffee, and preparing for your flight. You check in online, only to find your booking has mysteriously disappeared. At the airport, travelers are standing in long lines, with check-in systems completely offline. The chaos spreads. Banks are having trouble processing transactions, ATM machines are frozen, and news broadcasters are suddenly off the air.</p><p>It sounds like the plot of a thriller movie, but this was reality for countless individuals and businesses across the globe. And the culprit? A well-intentioned <strong>CrowdStrike software update</strong> that spiraled out of control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/804/1*eXSp6v84EHU8GnChz7SFjA.png" /></figure><h3>What Happened?</h3><p>CrowdStrike, a leading cybersecurity firm, released an update to its Falcon platform — a tool designed to safeguard systems against cyber threats. Ironically, the very update meant to enhance security ended up causing massive disruptions by interacting poorly with <strong>Windows systems</strong>, resulting in widespread outages. The issue manifested in various forms, from computers freezing and showing the infamous <strong>Blue Screen of Death (BSOD)</strong>, to locking users out of their machines due to <strong>BitLocker recovery</strong> issues​.</p><p>It didn’t take long for the problem to spread globally. Airports from <strong>Tokyo to New Delhi</strong>, major banks, and even news outlets faced unprecedented technical failures. Flights were delayed or grounded, transactions were halted, and millions of users were unable to access their systems.</p><h3>A Global Disaster Unfolds</h3><p>For those in the travel industry, the timing couldn’t have been worse. Airlines like <strong>Delta</strong>, <strong>United</strong>, and <strong>Ryanair</strong> were unable to manage their booking systems. Passengers were stuck, unable to check in, while airline staff resorted to manual processes — something that felt as outdated as writing a letter in the age of instant messaging.</p><p>In airports like <strong>Amsterdam</strong>, <strong>Tokyo</strong>, and <strong>Berlin</strong>, long queues formed as travelers grew restless, trying to figure out why nothing seemed to work. Some passengers had to wait for hours just to get an update, while others missed their flights entirely.</p><p>But it wasn’t just airports. Banks reported that their ATMs were going offline, and some branches were forced to turn away customers as their systems were unable to process any transactions. The financial sector, which relies on seamless operations to ensure the flow of money, was paralyzed​.</p><p>Meanwhile, <strong>Sky News</strong>, one of the UK’s major broadcasters, went completely off the air, unable to deliver live broadcasts to their millions of viewers. What should have been another routine day in news quickly turned into a technical nightmare, leaving viewers in the dark.</p><h3>Not a Cyberattack, Just a Bad Update</h3><p>Given the severity and the spread of the problem, many initially feared it was a <strong>cyberattack</strong>. Governments and security officials scrambled to assess the situation, but it soon became clear that this was not a coordinated hack. <strong>CrowdStrike</strong> acknowledged the issue, confirming that it was a <strong>technical failure</strong> stemming from a single update within their Falcon platform​.</p><p>The problem was rooted in how the update interacted with <strong>Windows systems</strong>. Specifically, it triggered system crashes and invoked <strong>BitLocker</strong> recovery, which caused machines to lock down, leaving users helpless without their recovery keys. The Falcon update, meant to improve security, had unfortunately broken the very systems it was designed to protect​.</p><h3>How CrowdStrike Responded</h3><p>CrowdStrike was quick to address the issue. Within hours of the outages, they released a <strong>fix</strong>. The company reassured its users that the problem wasn’t malicious but purely technical in nature. They advised affected organizations to <strong>reboot</strong> their systems after applying the patch, and in most cases, this worked to restore functionality​.</p><p>However, the fix wasn’t as simple for everyone. For companies with large networks, the recovery process was slow, requiring manual intervention and multiple reboots. The outage highlighted just how interconnected and fragile our digital systems can be, where a single flawed update can bring industries to a standstill.</p><h3>How to Get Rid of the Chaos</h3><p>For businesses and individuals affected by the outage, the steps to recover were relatively straightforward but time-consuming.</p><ol><li><strong>Apply the CrowdStrike Fix</strong>: If you were affected by the outage, the first step was to install the fix issued by CrowdStrike. The patch was designed to reverse the problematic update and restore system functionality.</li><li><strong>Reboot Systems</strong>: After applying the fix, companies were advised to reboot their systems. In many cases, a full reboot cleared the errors and got systems back online.</li><li><strong>Retrieve BitLocker Keys</strong>: For those locked out due to <strong>BitLocker</strong>, retrieving the recovery key from a <strong>Microsoft account</strong> or company key management system was necessary to unlock the machine. This process was straightforward for organizations with centralized IT support, but more challenging for smaller businesses without strong key management practices​.</li><li><strong>Prepare for Future Incidents</strong>: The incident served as a wake-up call for many organizations to bolster their contingency plans. <strong>Backup and recovery systems</strong>, <strong>redundant infrastructure</strong>, and <strong>testing updates in controlled environments</strong> became even more critical following the chaos.</li></ol><h3>A Wake-Up Call for the Digital World</h3><p>The CrowdStrike update incident exposed the vulnerabilities that lie at the heart of our digital infrastructure. It showed just how dependent industries have become on software that must function flawlessly to keep the world running. Whether it’s booking a flight, making a bank transfer, or delivering the news, we rely on technology to operate smoothly.</p><p>However, even with the best cybersecurity measures in place, incidents like this can still happen. The key takeaway is that <strong>businesses must always be prepared</strong>. Strong backup systems, robust disaster recovery plans, and the ability to quickly respond to technical failures are essential for minimizing the damage caused by such events.</p><p>The CrowdStrike outage will be remembered as a stark reminder of how even small software glitches can spiral into massive, global disruptions. But with the right precautions and recovery strategies, organizations can mitigate the impact and prevent future incidents from having the same level of chaos.</p><p>#AWS #CloudComputing #CrowdStrike #Cybersecurity</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d7c0e289076c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Seamless Serverless Architecture: Integrating AWS Lambda with VPC, EC2, and API Gateway]]></title>
            <link>https://medium.com/@hellomeenu1/building-a-seamless-serverless-architecture-integrating-aws-lambda-with-vpc-ec2-and-api-gateway-a0a377f1d670?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0a377f1d670</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Sat, 21 Sep 2024 23:40:58 GMT</pubDate>
            <atom:updated>2024-09-22T00:04:21.069Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s fast-paced world, serverless architectures have become the backbone of many applications. AWS Lambda, with its ability to run code without provisioning or managing servers, is a core player in building these architectures. But serverless isn’t isolated — it often needs to interact with traditional server-based resources, like EC2 instances, databases, and RESTful APIs.</p><p>In this post, we will explore how to integrate <strong>AWS Lambda</strong> with critical infrastructure components like <strong>VPCs</strong>, <strong>EC2 instances</strong>, and <strong>API Gateway</strong> to create a powerful, scalable, and flexible architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZOpnVNMgHSMHaR9Nj3cZpw.png" /></figure><h3>Why Serverless?</h3><p>Serverless computing provides:</p><ul><li><strong>No server management</strong>: AWS manages the infrastructure.</li><li><strong>Automatic scaling</strong>: Lambda scales with incoming requests.</li><li><strong>Cost efficiency</strong>: You pay only for what you use.</li></ul><p>Despite its numerous advantages, many applications require Lambda functions to interact with other AWS services, including EC2, S3, and others inside a VPC.</p><h3>Step 1: Setting Up Lambda to Connect with a VPC</h3><p>While AWS Lambda functions run in an isolated environment by default, they often need access to resources running inside a <strong>Virtual Private Cloud (VPC)</strong>, like RDS databases or private EC2 instances. Here’s how to achieve this.</p><h4>The Architecture: Lambda with VPC</h4><p>When a Lambda function needs to interact with resources inside a VPC, such as a private subnet, you must:</p><ol><li><strong>Place the Lambda function in the VPC</strong>: This ensures that the Lambda function can access resources within private subnets.</li><li><strong>Attach Security Groups</strong>: Configure security groups to allow traffic between Lambda and the required VPC resources.</li><li><strong>Use a NAT Gateway</strong>: If your Lambda function requires internet access, ensure that a <strong>NAT Gateway</strong> is set up in a public subnet to route outbound internet traffic.</li></ol><h4>Implementation: Lambda and VPC Setup</h4><ul><li><strong>Create a VPC</strong> with two public and two private subnets.</li><li><strong>Attach Lambda to the VPC</strong> by selecting private subnets and assigning a security group that allows the necessary traffic.</li></ul><p>Once the Lambda function is associated with the VPC, it can now securely access any EC2 instances or databases inside the VPC.</p><h3>Step 2: Lambda Interacting with EC2</h3><p>AWS Lambda’s power doesn’t end at VPC connectivity. It can also interact directly with services running inside EC2 instances. For example, imagine you have a simple web server running on an EC2 instance that serves metadata (like instance ID and availability zone). You can create a Lambda function that retrieves data from this server and stores it in <strong>Amazon S3</strong>.</p><h4>The Architecture: Lambda and EC2 Interaction</h4><p>In this setup:</p><ul><li><strong>Lambda</strong> fetches data from the <strong>EC2 web server</strong> running in a private subnet.</li><li>The fetched data is then uploaded to an <strong>S3 bucket</strong> for storage or further processing.</li></ul><h4>Implementation: Fetching and Uploading Data</h4><ol><li><strong>Launch an EC2 instance</strong> inside the private subnet and set up a web server that returns metadata (instance ID and availability zone).</li><li><strong>Modify the Lambda function</strong> to retrieve this data using an HTTP request and then upload it to <strong>S3</strong>.</li></ol><p>Here’s the code to accomplish this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7ll6_5gM4g9l7-p2-F8Rjg.png" /></figure><h4>Key Considerations:</h4><ul><li><strong>Security Groups</strong>: Ensure that the security group for the Lambda function allows outbound traffic to the EC2 instance.</li><li><strong>NAT Gateway</strong>: If Lambda needs internet access, set up a NAT gateway.</li></ul><h3>Step 3: Exposing Lambda via API Gateway</h3><p>API Gateway acts as a <strong>front door</strong> to serverless applications by enabling developers to create RESTful APIs that invoke Lambda functions. This allows external users to interact with your Lambda functions through HTTP requests.</p><h4>The Architecture: API Gateway Integration</h4><ul><li><strong>API Gateway</strong> handles incoming HTTP requests and routes them to the <strong>Lambda function</strong> using the <strong>LAMBDA_PROXY</strong> integration.</li><li>This architecture allows you to expose the functionality of your Lambda function to the web securely.</li></ul><h4>Implementation: Building the API</h4><ol><li><strong>Create the Lambda Function</strong>: Write a simple Lambda function that returns a random message.</li><li><strong>Build the API</strong> in API Gateway:</li></ol><ul><li>Create a new <strong>REST API</strong> and configure a /hello resource.</li><li>Set the method to <strong>GET</strong> and integrate it with your Lambda function using <strong>LAMBDA_PROXY</strong>.</li></ul><p><strong>3. Deploy the API</strong>:</p><ul><li>Deploy the API to a stage (e.g., prod) and obtain the invoke URL.</li><li>Test the API by sending a GET request to the /hello endpoint.</li></ul><p>Here’s an example <strong>Lambda function</strong> that generates a random greeting:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8kYT4l5I_m0GkWtljAUgNQ.png" /></figure><p>Once deployed, your Lambda function is now available via an API URL, allowing you to interact with it from the web.</p><h3>Step 4: Using Lambda with API Gateway and S3</h3><p>To expand the use case, let’s integrate all components:</p><ul><li>Use API Gateway to trigger the Lambda function.</li><li>The Lambda function retrieves data from an EC2 instance.</li><li>Lambda uploads the data to <strong>Amazon S3</strong>.</li></ul><p>This creates a <strong>seamless flow</strong> between multiple AWS services — allowing users to invoke complex workflows through a simple HTTP API.</p><h3>Conclusion: The Power of Serverless Integration</h3><p>By combining AWS Lambda with VPC, EC2, and API Gateway, we unlock the ability to create powerful, serverless applications that can interact with traditional server-based resources while providing the scalability and cost efficiency of serverless architecture.</p><p>With this architecture, developers can create highly efficient workflows that are resilient, scalable, and adaptable, making it easier to manage infrastructure and focus on building robust applications.</p><p>Serverless is more than just code execution — it’s a bridge between modern and traditional cloud architectures, enabling innovative solutions that can drive the next generation of applications.</p><p>#AWS #Lambda #APIGateway #NatGatway #S3Bucket #VPC</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0a377f1d670" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Amazon Prime Video Reverted to a Monolith: A Case Study on Cloud Architecture Evolution]]></title>
            <link>https://medium.com/@hellomeenu1/why-amazon-prime-video-reverted-to-a-monolith-a-case-study-on-cloud-architecture-evolution-bd2582b438a5?source=rss-8ba88fe42f63------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd2582b438a5</guid>
            <dc:creator><![CDATA[Rajpreet Gill]]></dc:creator>
            <pubDate>Fri, 20 Sep 2024 20:47:31 GMT</pubDate>
            <atom:updated>2024-09-20T20:56:21.288Z</atom:updated>
            <content:encoded><![CDATA[<p>In the cloud computing world, microservices are often hailed as the ultimate solution for scalability and agility. However, Amazon Prime Video — a leading video streaming platform — made a surprising architectural decision: they reverted certain critical services from a microservices-based architecture back to a monolithic design. This blog post explores the reasoning behind this shift, the challenges it addresses, and what this means for cloud architects worldwide.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D3JQ9KOfdYxrkIElGNbNpw.png" /></figure><h3>What is Amazon Prime Video?</h3><p>Amazon Prime Video is a global streaming service offering thousands of movies, TV shows, and original content like <em>The Boys</em> and <em>The Marvelous Mrs. Maisel</em>. Integrated as part of the larger Amazon Prime membership, it reaches millions of users globally. This platform handles enormous traffic, streaming content to viewers worldwide while managing petabytes of data and billions of API calls daily.</p><h3>Prime Video’s Journey: From Monolith to Microservices</h3><p>In the early days, Prime Video operated on a <strong>monolithic architecture</strong>, where all services — such as user authentication, video streaming, and recommendations — were tightly coupled within a single codebase. As demand grew and the platform scaled globally, this monolith became harder to maintain, slower to deploy, and more prone to failures.</p><p>To overcome these issues, Prime Video embraced a <strong>microservices architecture</strong>, breaking down the monolith into hundreds of smaller, independent services. Each microservice was responsible for a specific function, allowing teams to deploy updates independently, scale services based on traffic, and isolate faults.</p><h3>Why Did Prime Video Shift Back to a Monolithic Architecture?</h3><p>Despite the flexibility microservices offer, Amazon Prime Video made a strategic decision to return certain critical functions to a monolithic architecture. Here’s why:</p><h4>1. Latency and Operational Complexity</h4><p>Microservices introduce <strong>latency</strong> due to the increased communication between independent services. For instance, when a user clicks “Play” on a video, multiple services — such as authentication, content catalog, and playback — must communicate before the video starts. This caused millisecond-level delays, but at scale, even tiny lags can degrade the user experience.</p><p>By reverting to a monolithic setup for these core services, Amazon Prime Video reduced the number of API calls between services, minimizing latency and improving video start times.</p><h4>2. Simplifying Development and Operations</h4><p>Managing hundreds of services increases the operational complexity of monitoring, deploying, and debugging. This can lead to <strong>microservice sprawl</strong>, where the sheer number of services becomes difficult to track and manage. For Prime Video, consolidating these services simplified the development pipeline, reduced the cognitive load on developers, and streamlined operational overhead.</p><h4>3. Performance Optimization</h4><p>While microservices are ideal for scalability, performance optimizations sometimes require tight coupling of services. Amazon Prime Video’s playback and recommendation engines needed to function seamlessly together to deliver personalized content quickly. The switch back to a <strong>modular monolith</strong> allowed for better performance tuning and reduced inter-service communication, leading to an overall smoother user experience.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J1rqPzXg_x3YbSmmjS2Trw.png" /></figure><h3>The Benefits of a Hybrid Approach</h3><p>Amazon Prime Video didn’t abandon microservices entirely. Instead, they opted for a <strong>hybrid architecture</strong>, using a monolith for critical services like video playback while keeping non-essential services, such as billing and content management, in microservices. This hybrid approach allowed Prime Video to:</p><ul><li><strong>Optimize Core Features</strong>: Performance-sensitive services were consolidated to reduce latency and improve efficiency.</li><li><strong>Maintain Flexibility</strong>: Microservices are still used where independent scaling and fault isolation are beneficial.</li></ul><p>This strategic combination enables Prime Video to leverage the strengths of both architectures.</p><h3>Customer Impact: Faster, More Reliable Streaming</h3><p>The shift back to a monolithic architecture directly benefits Amazon Prime Video’s customers:</p><ul><li><strong>Faster Video Playback</strong>: The reduced latency between services means videos load faster, providing a more seamless experience for users.</li><li><strong>Increased Reliability</strong>: With fewer moving parts in the critical areas of the application, there are fewer chances of system failures or service disruptions.</li></ul><p>In a globally distributed service like Prime Video, these improvements are crucial for maintaining user satisfaction and minimizing churn.</p><h3>Real-World Example: The Video Playback Service</h3><p>Let’s break this down with an example: <strong>video playback</strong>.</p><p>In a microservices setup, when a user initiates playback:</p><ol><li><strong>Authentication Service</strong> verifies the user.</li><li><strong>Catalog Service</strong> retrieves metadata about the video.</li><li><strong>Playback Service</strong> starts streaming the video.</li><li><strong>Recommendations Service</strong> suggests additional content.</li></ol><p>Each of these services communicates via APIs, creating potential delays. In contrast, under the <strong>monolithic setup</strong>, these services are consolidated into a single module. This reduces the number of interactions between services and streamlines the entire process, leading to quicker response times.</p><h3>Challenges with Microservices: Lessons from Prime Video</h3><p>Prime Video’s transition highlights some of the real-world challenges associated with microservices:</p><ul><li><strong>Latency Issues</strong>: The overhead of inter-service communication can slow down performance.</li><li><strong>Operational Complexity</strong>: Managing hundreds of independent services increases the burden on DevOps teams.</li><li><strong>Microservice Sprawl</strong>: As the number of services grows, tracking, managing, and debugging becomes increasingly difficult.</li></ul><p>Amazon Prime Video’s decision to centralize certain services into a monolith was aimed at reducing these challenges while retaining the benefits of microservices where needed.</p><h3>Lessons for Cloud Architects: When to Use a Monolith vs. Microservices</h3><p>This shift in architecture provides valuable insights for cloud architects and developers:</p><ul><li><strong>Monolith</strong>: Ideal for performance-critical, tightly coupled systems where minimizing latency is a top priority.</li><li><strong>Microservices</strong>: Best for applications that need independent scaling and flexibility, especially for non-core services.</li><li><strong>Hybrid Approach</strong>: Combining both architectures allows businesses to benefit from scalability and agility without compromising on performance or complexity.</li></ul><h3>Conclusion: Architecture is Always Evolving</h3><p>Amazon Prime Video’s architectural shift reminds us that there is no one-size-fits-all solution in cloud architecture. The choice between microservices and monolith depends on the specific needs of your application, such as performance, complexity, and scalability. As technology evolves, so too should your approach to architecture. Constantly evaluate your infrastructure and be prepared to adapt as your business and user needs change.</p><p>#AWS #cloudcomputing #Amazon #Monolith #Microservices # Hybrid</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd2582b438a5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>