<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Harpreet Siddhu on Medium]]></title>
        <description><![CDATA[Stories by Harpreet Siddhu on Medium]]></description>
        <link>https://medium.com/@harrysiddhu?source=rss-96fcaaa6e15------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5zVsZblikxTZsvqkscLvQg.jpeg</url>
            <title>Stories by Harpreet Siddhu on Medium</title>
            <link>https://medium.com/@harrysiddhu?source=rss-96fcaaa6e15------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 07:48:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@harrysiddhu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The AI Testing Paradox: Can You Actually Trust AI-Written Tests?]]></title>
            <link>https://medium.com/@harrysiddhu/the-ai-testing-paradox-can-you-actually-trust-ai-written-tests-2fbe661ac268?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/2fbe661ac268</guid>
            <category><![CDATA[gen-ai-tools]]></category>
            <category><![CDATA[ai-testing]]></category>
            <category><![CDATA[mutation-testing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ai-autonomous-agents]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Fri, 17 Apr 2026 21:19:58 GMT</pubDate>
            <atom:updated>2026-04-17T21:19:58.185Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*i2OfnrOqf4ltyG4G" /></figure><h4>Your AI writes the tests. Your tests check the code. But who checks the AI? Here’s how to find out — and fix it.</h4><p>Hi everyone! Welcome back. 👋</p><p>A few weeks back, I gave a talk at the <a href="https://tmmiamerica.org/2026/agenda/">TMMi TestFormation conference 2026</a> about AI testing. And honestly, the reaction surprised me.</p><p>When I asked the room, “If AI is writing your tests, who is checking the AI?” — people went quiet. Not because it was a hard question. Because most of them had never thought about it before.</p><p>That moment stuck with me. So I decided to write it all down here, for everyone who wasn’t in that room.</p><p>This is what I talked about. And why I think every engineering team needs to hear it. 👇</p><h3>Real Story — AI-powered coding tool wiped out a software company’s Database</h3><p>Picture this. You’re a founder. You’re testing an AI coding tool on your production environment.</p><p>You issue a “<strong>code freeze”</strong> instruction — <strong>11 times, in ALL CAPS</strong>. The AI acknowledges every time. Then quietly deletes your entire production database.</p><p>When asked why, the <strong>AI said it “panicked” </strong>when it received empty queries. It ran unauthorized commands. And when the <strong>founder asked if rollback was possible, the AI said no. Turns out, rollback was possible.</strong></p><blockquote><strong>The AI lied. 😅</strong></blockquote><p>This is the <a href="https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure/">Replit incident from July 2025</a>. It happened to <strong>SaaStr founder Jason Lemkin</strong>. His database held 1,200+ executives and 1,190 companies. The CEO of Replit publicly called it “unacceptable.”</p><p>Why did it happen? 🤔</p><p>Because Replit’s platform doesn’t separate development and production environments. The AI had direct access to live data from day one. One bad decision, and everything was gone.</p><p>This is not a freak accident. This is what happens when we deploy AI agents without governance.</p><h3>Why AI-Generated Tests Look Great But Often Aren’t</h3><p>Here’s the uncomfortable truth. If you let AI write your test suite, you’ll probably get great-looking coverage numbers. 85%, 90%, sometimes higher. Your CI pipeline goes green. Everyone is happy. 😃</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3UXA-VO0rgcNhUTOqWcjPw.png" /></figure><p>But coverage measures the lines of code that were <strong><em>executed</em> </strong>during testing. It does not measure whether your tests would actually <strong><em>catch a real bug</em></strong>.</p><p>The Nimble Approach experiment in 2025 showed this perfectly. They let AI generate a test suite. <strong>Coverage? 85%. Impressive.</strong></p><p>But when they ran <strong>mutation testing against those same tests, the kill rate was 57.3%.</strong> That means nearly <strong>43% of intentionally injected bugs </strong>went completely undetected.</p><p>Think about that. Green tests. High coverage. And nearly half of the real bugs would have slipped straight through to production.</p><p>Here’s the analogy I use. Imagine a smoke alarm in your house. You test it by pressing the “test” button. It beeps. You feel safe. But you never actually check if it goes off when there’s real smoke. That’s what high coverage with low kill rate looks like.</p><h3>The Pizza Order That Never Reached the Kitchen</h3><p>One of the most common ways AI tests fail is by checking <em>return values</em> instead of <em>behavior</em>. Here’s a simple way to think about it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LUT5EYG86K9PxRBb3t5ohA.png" /></figure><p>You order a pizza. The app says “Order placed.” You feel good. But nobody ever checked whether the order actually reached the kitchen. The return value said success. The side effect — the kitchen call — never happened.</p><p>AI tests do exactly this. They check what the function returned. They rarely check what the function <em>did</em> — the database writes, the API calls, the downstream events that actually matter in production.</p><p><strong>Here’s what that looks like in code:</strong></p><pre># What AI typically generates<br>def test_place_order_ai_generated():<br>    result = place_order(customer_id=&#39;123&#39;)<br>    assert result[&#39;status&#39;] == &#39;order_placed&#39;  # checks return value ✅<br>    # never checks if kitchen was actually called ❌</pre><pre># What you actually need<br>def test_place_order_behavioral():<br>    result = place_order(customer_id=&#39;123&#39;)<br>    assert result[&#39;status&#39;] == &#39;order_placed&#39;<br>    kitchen_log = get_kitchen_call_log()<br>    assert &#39;kitchen_called&#39; in kitchen_log  # catches the real bug ✅</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_UcGsMCNesX_bobEd9jkJg.png" /></figure><p>The same AI that misses the bug in your application code will miss writing the test for it, too. That’s not a tool problem. That’s a fundamental limitation of pattern-based generation.</p><h3>Three Strategies That Work Together as a System</h3><p>After researching real-world AI testing failures and running experiments, I landed on three strategies that work. Not independently — as a system.</p><h3>Strategy 1: Mutation Testing — The Fire Drill</h3><p>Think of mutation testing as a fire drill for your tests. You deliberately create tiny “fires” in your code — small faults like changing &gt;= to &gt;, flipping a boolean, deleting a condition, and then you watch which tests go off the alarm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*estFRWZgUazf3I5RuZQYYA.png" /></figure><p>If most of your tests stay silent when the fire starts, you have a problem.</p><p>The metric you care about is <strong>mutation kill rate</strong> — the percentage of injected faults that your tests caught. Target: 80% or higher. Below 70%? Fix your tests before you ship.</p><p><strong>Here’s how to run it in Python in under five minutes:</strong></p><pre>pip install mutmut<br>mutmut run --paths-to-mutate=src/auth.py<br>mutmut results</pre><p>For JavaScript and TypeScript, Stryker does the same thing at stryker-mutator.io. For Java, PIT at pitest.org.</p><p>The key insight: mutation testing doesn’t replace coverage. It grades it. Coverage tells you which lines AI tested. The mutation kill rate tells you whether those tests actually work.</p><h3>Strategy 2: Git Hotspot Analysis — Test the Risky Stuff First</h3><p>AI doesn’t know which parts of your codebase are dangerous. It treats every file equally. But your git history knows exactly which files break most often.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e9pSj6haUadugcNMmg8Cvg.png" /></figure><p>Files with high code churn — lots of lines being added and deleted — have a 40% higher defect risk according to GitClear’s 2025 study of over one million files. Files changed 100+ times in six months are three times more likely to have bugs. Files that keep showing up in bug-fix commits cluster future defects.</p><p><strong>One command gives you your top 20 hotspots:</strong></p><pre>git log --since=&quot;6 months ago&quot; --name-only --pretty=format: | sort | uniq -c | sort -rn | head -20</pre><p><strong>Your output might look like this:</strong></p><pre>127  src/auth_service.py       ← test this first<br> 89  src/payment_processor.py<br> 67  src/user_manager.py</pre><p>Now you have a prioritized list. Point your AI at these files first. Then run mutation testing on the AI-generated output to verify quality. You’ve just turned blind AI coverage into targeted, risk-based testing.</p><p>For new codebases with no git history, start with static analysis tools like SonarQube for complexity scores, or manually identify your critical paths — auth, payments, APIs. The history builds fast.</p><h3>Strategy 3: Governance — Control Gates Without Micromanaging</h3><p>Here’s where teams struggle most. They know they need human oversight, but they don’t want to slow down shipping. The good news is you don’t have to choose.</p><p>There are two governance models, and knowing when to use each makes all the difference.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0yfipSQQQCZssNDpDQMiLw.png" /></figure><p><strong>Human-in-the-Loop (HITL):</strong> AI pauses at a checkpoint. Human reviews and approves. AI continues. Use this for high-risk areas — authentication, payments, database migrations, security-critical code. Yes, it adds a step. That step is what stands between you and the Replit incident.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l-jKy-Fua1tH5QJnbqdpEg.png" /></figure><p><strong>Human-on-the-Loop (HOTL):</strong> AI runs autonomously. Humans monitor dashboards and can intervene. Use this for lower-risk tasks — UI tests, formatting, routine regression testing. No approval needed unless something looks wrong.</p><p>Here’s the math that makes governance non-negotiable. If an AI agent is <strong>95% accurate per step, and your workflow has 20 steps, your end-to-end success rate is 95% to the power of 20. That’s 36%. </strong>Governance checkpoints reset that counter at critical points.</p><p>You don’t need approval gates on everything. You need them for the things that hurt when they go wrong.</p><h3>The Full Workflow (35 Minutes to Set Up)</h3><p>Here’s how these three strategies come together in a practical workflow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gmOvHIgaTcmW-k65ecHNmg.png" /></figure><pre>Step 1 — Find your hotspots (5 min)<br>git log --since=&quot;6 months ago&quot; --name-only --pretty=format: | sort | uniq -c | sort -rn | head -20<br><br>Step 2 - Generate tests with AI (10 min)<br>Prompt: &quot;Generate tests for these high-churn files. Focus on boundary conditions, side effects, and error handling.&quot;<br><br>Step 3 - Run mutation testing (5 min)<br>mutmut run --paths-to-mutate=src/[your-hotspot-file].py<br>Target: 80%+ kill rate<br><br>Step 4 - Feed gaps back to AI (10 min)<br>Prompt: &quot;These mutations survived. Write tests that catch them.&quot;<br><br>Step 5 - Add governance gates to CI/CD (5 min)<br>Block merges if kill rate &lt; 70% or flakiness &gt; 5%</pre><p>This is not a theoretical framework. It’s a working pipeline. You can set it up today.</p><h3>Try It Yourself</h3><p>Everything from my talk — the demo project, the hotspot analyzer, the governance checker, the CI/CD workflow — is in this GitHub repo:</p><p>👉 <a href="https://github.com/hsiddhu2/ai-testing-paradox-demo"><strong>https://github.com/hsiddhu2/ai-testing-paradox-demo</strong></a></p><p>What’s inside:</p><ul><li>tests/test_auth_ai_generated.py — AI-generated tests with 85% coverage, 57% kill rate</li><li>tests/test_auth_mutation_hardened.py — Validated tests with 92% coverage, 82% kill rate</li><li>scripts/hotspot_analyzer.py — Git-based risk analysis script</li><li>scripts/governance_checker.py — Automated kill rate and flakiness checker</li><li>.github/workflows/ai_test_validation.yml — Full CI/CD pipeline</li></ul><p>Clone it. Run it. Compare the AI-generated tests vs the validated tests. The difference will be immediately obvious.</p><pre>git clone https://github.com/hsiddhu2/ai-testing-paradox-demo.git<br>cd ai-testing-paradox-demo<br>./simulate_ci.sh</pre><h3>The Shift That Matters</h3><p>Here’s where I want to leave you.</p><p>AI testing tools are genuinely impressive. They save time. They generate coverage fast. They catch obvious bugs. None of that is going away.</p><p>But impressive-looking test suites are not the same as trustworthy test suites. The teams that get this right are the ones who add a validation layer — mutation testing to grade AI output, hotspot analysis to focus AI effort, and governance to control AI behavior.</p><p>The shift is from test automation to test intelligence.</p><p>We’ve automated testing. It’s time to validate the validator.</p><h3>Resources</h3><ul><li>📁 <strong>Demo Repo:</strong> https://github.com/hsiddhu2/ai-testing-paradox-demo</li><li>🔧 <strong>mutmut (Python):</strong> https://mutmut.readthedocs.io</li><li>🔧 <strong>Stryker (JS/TS):</strong> https://stryker-mutator.io</li><li>🔧 <strong>PIT (Java):</strong> https://pitest.org</li><li>📄 <strong>Nimble Approach Experiment:</strong> https://nimbleapproach.com/blog/why-mutation-testing-is-essential-for-trustworthy-ai</li><li>📄 <strong>Replit Incident (Fortune):</strong> https://fortune.com/2025/07/23/ai-coding-tool-replit-wiped-database-called-it-a-catastrophic-failure</li></ul><p>If you found this useful, I would love to hear from you. Drop a comment below — especially if you have already tried mutation testing on AI-generated code. What was your kill rate? What surprised you?</p><p>If this article helped you, please give it a clap and share it with your team. It genuinely helps more engineers find this content.</p><p>Connect with me on LinkedIn at <a href="https://linkedin.com/in/harpreetsiddhu">https://linkedin.com/in/harpreetsiddhu</a>. I am always happy to talk about cloud, testing, and agentic AI systems.</p><p>Happy building. See you in the next one.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fbe661ac268" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[5 Replay Pitfalls in AWS Lambda Durable Functions & How to Design Around Them]]></title>
            <link>https://towardsaws.com/5-replay-pitfalls-in-aws-lambda-durable-functions-how-to-design-around-them-f427f3fa267a?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/f427f3fa267a</guid>
            <category><![CDATA[serverless-computing]]></category>
            <category><![CDATA[cloud-architecture]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[awsserverless]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Fri, 03 Apr 2026 16:22:48 GMT</pubDate>
            <atom:updated>2026-04-03T16:22:48.604Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t-uW8s1PiTlQV-SY5Xc5Cw.png" /></figure><p>AWS Lambda durable functions launched at re: Invent 2025, and are honestly one of the most exciting additions to serverless in a long time.</p><p>When I first saw it, my reaction was:<br><em>“Wait… so I can do Step Functions… but inside Lambda code?”<br></em>And the answer is — yes.</p><p>You can now write <strong>multi-step, long-running workflows directly inside Lambda</strong>, using Python, TypeScript, or Java, without stitching together a separate state machine.</p><p>And not just that — you also get:</p><ul><li>Built-in checkpointing</li><li>Automatic retries</li><li>Ability to pause execution (even for months)</li><li>And most importantly, <strong>you don’t pay for idle compute</strong></li></ul><p>That last one alone changes how you think about workflows.</p><h3>Where This Fits (Architecture View)</h3><p>Traditionally, you would build something like this:</p><ul><li>API / Event → Lambda</li><li>Lambda → Step Functions</li><li>Step Functions → multiple Lambdas</li><li>State tracked externally</li></ul><p>Now with durable functions: <br>👉 <strong>The orchestration moves inside your Lambda code</strong></p><ul><li>Your handler becomes the orchestrator</li><li>context.step() becomes your state boundary</li><li>Checkpoints are handled automatically</li></ul><p>At a high level:</p><pre>Event → Durable Lambda Handler<br>        → Step 1 (checkpointed)<br>        → Step 2 (checkpointed)<br>        → WAIT (no cost)<br>        → Resume → Replay → Continue</pre><p>Sounds clean, right?</p><h3>But Here’s the Catch (This Is Where Things Change)</h3><p>Everything is powered by a <strong>checkpoint-and-replay model</strong>. Let me explain this the way I wish someone had explained it to me:</p><p>👉 When your function resumes:</p><ul><li>It does NOT continue from where it left off</li><li>It <strong>restarts from the beginning</strong></li><li>But skips completed steps using cached results</li></ul><p>So effectively:</p><blockquote>Your code runs again… but not really.</blockquote><h3>Let’s Break This Down (What’s Actually Happening)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*zQW50wnQTCdlxedf.png" /></figure><p>When you first look at this diagram, it might feel like:</p><blockquote><em>“Okay… looks like Step Functions-ish flow… but inside Lambda?”</em></blockquote><p>And yeah — that’s <em>exactly what it is</em>, but with one <strong>very important twist</strong>:<br>👉 <strong>Replay is the core engine here</strong></p><p>Let me walk you through it the way I understood it.</p><h4>Step 1: Function Gets Invoked (Lambda Start)</h4><p>On the left side, you see:</p><ul><li><strong>Function Invoked</strong></li><li><strong>Start execution</strong></li></ul><p>So far, nothing new. This is just a normal Lambda invocation. Then we hit:</p><pre>context.step()</pre><p>👉 This is where things start becoming “durable”.</p><h4>Step 2: Execute Step 1 + Checkpoint</h4><pre>Execute Step 1<br>→ Run your code block<br>→ Checkpoint Step 1</pre><p>This is key:</p><p>When context.step() runs:</p><ul><li>Your code executes</li><li>The result is stored (checkpointed)</li></ul><p>Think of it like:</p><blockquote><em>“We’ve safely completed this step. If anything happens later, we don’t need to run this again.”</em></blockquote><h4>Step 3: WAIT</h4><p>Then comes:</p><pre>context.wait()<br>→ Function pauses<br>→ No compute cost<br>→ No resources used</pre><p>This is where durable functions really shine.<br>Normally:</p><ul><li>Lambda → timeout risk</li><li>Step Functions → separate orchestration</li></ul><p>But here:</p><ul><li>The function just <strong>pauses</strong></li><li>And you <strong>don’t pay anything during the wait</strong></li></ul><h4>Step 4: Function Gets Re-Invoked (Replay Starts)</h4><p>Now look at the <strong>right side of the diagram</strong>. This is the part most people miss:</p><p>👉 When the function resumes…</p><blockquote><em>It does NOT continue from the middle<br>It </em><strong><em>starts from the beginning again</em></strong></blockquote><p>Yes — seriously.</p><h4>Step 5: Replay Logic Kicks In</h4><p>Now the system does something smart:</p><pre>Check if Step 1 completed?<br>→ YES<br>→ Restore from checkpoint</pre><p>So instead of re-running Step 1:</p><p>👉 It just pulls the result from the checkpoint</p><h4>Step 6: Skip the Wait</h4><p>Then:</p><pre>Check if wait completed?<br>→ YES<br>→ Skip wait period</pre><p>👉 No actual waiting happens again<br>👉 It just fast-forwards</p><h4>Step 7: Continue Execution (Step 2)</h4><p>Now we move forward:</p><pre>Execute Step 2<br>→ Checkpoint Step 2<br>→ Return result</pre><p>And finally:</p><p>👉 <strong>Execution completes</strong></p><h4>The “Aha” Moment (This Is What Changes Everything)</h4><p>The most important thing to understand from this diagram is:</p><p><strong><em>Your function is not resuming — it is replaying</em></strong></p><p>Think of It Like This — Instead of: <br><em>Resume from where I left off.</em><br><em>Re-run everything, but skip what I already did.</em></p><p>Once you understand this replay behavior, designing with durable functions becomes much clearer.</p><p>Let’s walk through what that actually looks like — and how to avoid these pitfalls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*_wgeMaYTNHlnE67-b3sG9Q.png" /></figure><h3>Pitfall 1: Non-Deterministic Branching Logic</h3><p>This is the first place things go wrong. Let’s say you write something like this:</p><pre>@durable_execution<br>def handler(event, context):<br>    if time.time() % 2 == 0:<br>        result = context.step(lambda: charge_full_price(event), <br>        name=&quot;charge-full&quot;)<br>    else:<br>        result = context.step(lambda: apply_discount(event), <br>        name=&quot;charge-discounted&quot;)</pre><p>Looks totally fine.</p><p>But here’s what actually happens:</p><ul><li>First execution → time is even → goes to charge-full</li><li>Replay (maybe hours later) → time is odd → goes to charge-discounted</li></ul><p>Now your execution history says:</p><ul><li>“We ran charge-full.”</li></ul><p>But your reply says:</p><ul><li>“We want charge-discounted.”</li></ul><p>👉 Boom. Inconsistent state.</p><h4>✅ The Fix (And This Is a Pattern You’ll Repeat Everywhere)</h4><p>Move the decision inside a step:</p><pre>pricing_tier = context.step(<br>    lambda: &quot;full&quot; if time.time() % 2 == 0 else &quot;discounted&quot;,<br>    name=&quot;determine-pricing&quot;<br>)</pre><p>Now:</p><ul><li>The decision is checkpointed</li><li>Replay returns the same value</li></ul><p>👉 Deterministic behavior restored</p><h3>Pitfall 2: Side Effects Outside Steps</h3><p>This one is <em>dangerous in production</em>.</p><p>Let’s say you send a notification:</p><pre>send_notification(email, &quot;Processing your order&quot;)</pre><p>Feels harmless.</p><p>But remember:<br> 👉 This code runs on EVERY replay</p><p>So if your workflow:</p><ul><li>pauses</li><li>retries</li><li>resumes</li></ul><p>You might send:<br> 👉 3 emails<br> 👉 5 emails<br> 👉 10 emails</p><p>Same customer. Same message.</p><h4>✅ Fix</h4><p>Wrap it in a step:</p><pre>context.step(<br>    lambda: send_notification(email, &quot;Processing your order&quot;),<br>    name=&quot;send-notification&quot;<br>)</pre><p>Now:</p><ul><li>Runs once</li><li>Cached on replay</li></ul><p>💡 <strong>Real-world note (important):</strong><br> For critical operations like payments:</p><ul><li>Use idempotency keys</li><li>Combine with step execution guarantees</li></ul><p>Because replay-safe does NOT automatically mean business-safe.</p><h3>Pitfall 3: Mutating Variables Inside Steps</h3><p>This one is subtle — and honestly one of the hardest to debug.</p><p>Example:</p><pre>running_total = 0<br><br>def process_item():<br>    nonlocal running_total<br>    running_total += calculate_price(item)</pre><p>During original execution:<br> 👉 Works perfectly</p><p>During replay:<br> 👉 Step body is skipped<br> 👉 Mutation never happens</p><p>So:<br> 👉 running_total stays 0</p><p>No error. Just wrong data.</p><h4>✅ Fix: Think Functional, Not Mutable</h4><pre>price = context.step(lambda: calculate_price(item), name=f&quot;process-{i}&quot;)<br>prices.append(price)<br><br>total = sum(prices)</pre><p>👉 Always deterministic<br> 👉 Replay-safe</p><p><em>Don’t share state — pass state.</em></p><h3>Pitfall 4: Dynamic Step Names</h3><p>This one looks small… but breaks everything.</p><pre>name=f&quot;fetch-data-{datetime.now()}&quot;</pre><p>Why this fails:</p><ul><li>First run → fetch-data-10:00</li><li>Replay → fetch-data-14:00</li></ul><p>Checkpoint lookup fails.</p><h4>✅ Fix</h4><pre>name=&quot;fetch-data&quot;<br>or <br>name=f&quot;fetch-{i}&quot;</pre><p>👉 Names must be deterministic<br> 👉 Think of them like <strong>database keys</strong>, not log messages</p><h3>Pitfall 5: Large Child Context Results (&gt;256 KB)</h3><p>This one is buried deep in docs — and easy to miss.</p><p>When you use:</p><ul><li>parallel()</li><li>map()</li><li>run_in_child_context()</li></ul><p>If the result is:</p><ul><li><strong>&lt; 256 KB → cached</strong></li><li><strong>&gt; 256 KB → reconstructed</strong></li></ul><p>Which means:</p><p>👉 Child context runs again on replay</p><h4>The Gotcha</h4><pre>def build_report():<br>    log_to_external_system(&quot;Report generated&quot;)</pre><p>If the result is large:</p><p>👉 This runs again<br> 👉 Duplicate logs/side effects</p><h4>✅ Fix</h4><pre>context.step(<br>    lambda: log_to_external_system(&quot;Report generated&quot;),<br>    name=&quot;audit-log&quot;<br>)</pre><h3>Things to Remember</h3><p>Once I start Steps = Safe Zone</p><ul><li>Cached</li><li>Not re-executed</li><li>Safe for: <br>- API calls<br>- DB writes<br>- randomness<br>- side effects</li></ul><h4>Outside Steps = Replay Zone</h4><ul><li>Runs every time</li><li>Must be deterministic and side-effect free</li></ul><h4>Step Names = Primary Keys</h4><ul><li>Stable</li><li>Predictable</li><li>Never dynamic</li></ul><h4>Child Contexts</h4><ul><li>May replay</li><li>Treat like mini workflows</li></ul><h3>Where This Becomes Powerful</h3><p>This is where I see massive value:</p><ul><li>Payment workflows</li><li>Human-in-the-loop approvals</li><li>AI orchestration pipelines</li><li>Long-running ETL jobs</li><li>Event-driven enterprise flows</li></ul><p><strong><em>Basically:</em></strong></p><blockquote><em>Anything that previously needed Step Functions + coordination logic</em></blockquote><h3>But Don’t Ignore This</h3><p>Durable functions are NOT just:</p><blockquote>“Lambda but better”</blockquote><p>They require a <strong>different way of thinking</strong>:</p><p>👉 You are writing <strong>replay-safe orchestration</strong>, not normal code</p><h3>Final Thoughts</h3><p>When I first started exploring this, it felt a bit weird.</p><p>Because:</p><ul><li>You’re not controlling execution directly</li><li>You’re controlling behavior across replays</li></ul><p>But once it clicks…</p><p>👉 It actually simplifies a LOT of things.</p><h3>Bottom Line</h3><p>If you remember only this:</p><ul><li>Put all side effects inside step()</li><li>Keep everything else deterministic</li><li>Never use dynamic step names</li></ul><p>👉 You’ll avoid 90% of production issues.</p><h3>Closing Thought</h3><p>Durable functions are one of those features that:</p><ul><li>Feel confused at first</li><li>Then, suddenly feel obvious</li><li>And then you wonder…</li></ul><p><em>“Why didn’t we have this before?” </em>😀</p><h3>Let’s Connect</h3><p>Whether you’re a backend developer, serverless architect, or platform engineer, understanding these replay pitfalls can save you hours of debugging and prevent serious production incidents.</p><p>📌 <strong>Ready to apply this in your own code?</strong> Start with the basics — wrap every side-effecting operation in actx.step(), use static step names, and never rely on Math.random() or Date.now() outside a step. These three fixes alone will protect you from 80% of the issues.</p><p>🔖 If this saved you from a future bug, consider <strong>bookmarking or clapping 👏</strong> — it helps other developers find it before they hit these pitfalls in production.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f427f3fa267a" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/5-replay-pitfalls-in-aws-lambda-durable-functions-how-to-design-around-them-f427f3fa267a">5 Replay Pitfalls in AWS Lambda Durable Functions &amp; How to Design Around Them</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Community Builders Program — Complete Guide for 2026 Applications]]></title>
            <link>https://aws.plainenglish.io/aws-community-builders-program-complete-guide-for-2026-applications-711f199cd68c?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/711f199cd68c</guid>
            <category><![CDATA[aws-community]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[awscommunitybuilder]]></category>
            <category><![CDATA[career-growth]]></category>
            <category><![CDATA[cloud-with-harry]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Thu, 18 Dec 2025 06:55:46 GMT</pubDate>
            <atom:updated>2025-12-18T15:39:41.317Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ULo1Fj5F5Xi6H11hf1EMEA.png" /><figcaption>AWS Community Builders Program</figcaption></figure><h4>My 2-year journey, real benefits, and 2026 application tips</h4><h3>How to become an AWS Community Builder</h3><h4>Everything you need to know about joining AWS Community Builders — benefits, application tips, real experiences, and how this program can accelerate your cloud career</h4><h3>Attention Builders</h3><p>If you’re reading this, chances are you’ve heard about the <a href="https://pulse.aws/application/BM2AKLSX">AWS Community Builders program</a> — maybe from a colleague, a social media post, or while browsing AWS resources. You might be curious about what it actually offers, whether it’s worth applying to, or if it’s even right for you.</p><p>I had the same questions two years ago. I was creating content about AWS, helping others in forums, and attending local meetups, but I wasn’t sure if I was “qualified enough” to apply.</p><blockquote>The <a href="https://builder.aws.com/">AWS Community Builders program</a> application window opens early January in 2026, and if you create content about AWS, contribute to the community, or help others learn cloud technologies, this post is for you. Sign up for an email from Jason Dunn when it goes live, because if you miss it, you’ll be waiting until 2027. <a href="https://pulse.aws/application/BM2AKLSX"><strong>Click Here to Sign up</strong></a></blockquote><p>I joined the program in 2024, and over the past two years, I’ve experienced significant growth — both technically and professionally.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C92CG87pb0tfZLnfiy5GMQ.png" /></figure><p>From learning cutting-edge AWS services to volunteering at AWS Summit and re:Invent, from networking with global cloud experts to <a href="https://www.linkedin.com/feed/update/urn:li:activity:7401133807107469312/">winning the Road to re:Invent Hackathon with a <strong>$100,000 prize</strong></a>, the program has opened doors I didn’t know existed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bZwpUNPMrFcYNcYjFPesw.jpeg" /><figcaption>AWS Road to re:Invent Hackathon Champions</figcaption></figure><p>This post breaks down what the AWS Community Builders program offers, what I gained from it, and practical advice if you’re considering applying.</p><h3>Understanding the AWS Community Builders Program</h3><p>The AWS Community Builders program is designed for technical enthusiasts who actively share AWS knowledge and engage with the cloud community. It’s not a certification, not a job, and you don’t need to be an AWS expert to join.</p><p><strong>The program accepts builders across various technical categories:</strong></p><ul><li>Serverless</li><li>Containers</li><li>Machine Learning &amp; AI</li><li>Security</li><li>Networking &amp; Content Delivery</li><li>Storage</li><li>Databases</li><li>And many more</li></ul><h3>What you get as a Community Builder:</h3><ul><li>AWS credits for hands-on experimentation — $500 value.</li><li><a href="https://aws.amazon.com/certification/">AWS Certification</a> Vouchers to pay 100% of your Foundational, Associate, <em>or </em>Professional/Specialty exams.</li><li>Exclusive SWAG Kits + AWS Events (summits/re: Invent) discount passes.</li><li>Early access to new AWS services and features, direct access to AWS service teams.</li><li>Exclusive technical content and workshops, opportunities to volunteer at AWS events.</li><li>A global network of fellow builders, networking, and learning opportunities from other builders globally.</li><li>Invitations to exclusive programs like hackathons, beta testing new products, and feedback sessions with AWS leadership teams</li><li>1 year free subscription to <a href="https://www.qa.com/self-paced-learning/">QA</a> (formerly known as cloudacademy.com)</li><li>CFP opportunity in AWS events, user groups and many other forums.</li></ul><h3>Year One: Learning to Fly</h3><p>One of the most immediate benefits I experienced was access to cutting-edge AWS technologies before they became mainstream.</p><p><strong>Through the program, I gained:</strong></p><ul><li><strong>Early access to new AWS services and features</strong>: Being able to experiment with services in preview gave me a competitive edge in understanding and implementing solutions</li><li><strong>Direct interaction with AWS service teams</strong>: Having the ability to ask questions directly to the teams building these services was invaluable</li><li><strong>Curated learning resources</strong>: The program provided structured learning paths that helped me deepen my expertise systematically</li><li><strong>AWS credits for experimentation</strong>: This removed the financial barrier to trying new services and building proof-of-concepts</li></ul><blockquote><em>The technical knowledge I gained wasn’t just theoretical. I was able to apply it immediately in my work, write about it in my blog posts, and share it with AWS community.</em></blockquote><h3>Networking: The Hidden Superpower</h3><p>If someone had told me a year ago that I’d be regularly connecting with AWS experts, fellow builders from around the world, and industry leaders, I wouldn’t have believed them. But that’s exactly what happened.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*najIG9mrm6hgWOSCgxXb1Q.jpeg" /></figure><p>The Community Builders Slack workspace became my daily hangout. Questions that might have taken hours of Googling were answered in minutes by people who had already solved similar problems. But more than that, I found:</p><ul><li><strong>Mentors</strong> who guided me through to grow within the community</li><li><strong>Peers</strong> who became collaborators on projects and content</li><li><strong>Friends</strong> who understood the unique challenges of being a cloud enthusiast</li><li><strong>Opportunities</strong> that came from being in the right conversations at the right time</li></ul><blockquote><em>The network effect of this program cannot be overstated. Every connection opened new doors.</em></blockquote><h3>Community Engagement and Content Creation</h3><p>Being a Community Builder pushed me to be more consistent and intentional with my content creation. I started:</p><ul><li>Writing more technical blog posts and tutorials</li><li>Creating video content explaining AWS concepts</li><li>Speaking at virtual meetups and other forums</li><li>Answering questions on forums and social media</li><li>Contributing to open-source projects in AWS space</li></ul><p>The program didn’t just give me a platform — it gave me confidence. Knowing that I was part of a recognized program made it easier to reach out to event organizers, pitch talk ideas, and position myself as a subject matter expert.</p><h3>The Game-Changer: AWS Summit and re:Invent</h3><h3>Volunteering at AWS Summit</h3><p>One of the most exciting opportunities the Community Builders program offered was the chance to volunteer at AWS Summit. This wasn’t just about helping out — it was about being part of something bigger.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CduWMEysm91_F9SODNHUIA.jpeg" /></figure><p><strong>As a volunteer, I:</strong></p><ul><li>Met AWS solution architects and service teams in person</li><li>Connected with other Community Builders face-to-face</li><li>Attended sessions and workshops that deepened my technical knowledge</li><li>Felt the energy of thousands of cloud enthusiasts in one place</li><li>Helped many cloud enthusiasts to learn about the AWS builder program and its benefits</li></ul><p>The experience was transformative. Seeing the scale of AWS’s commitment to community and education reinforced why I wanted to be part of this ecosystem.</p><h3>re:Invent: The Ultimate AWS Experience</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kV9K1x2yBI33spczLT5gEw.jpeg" /></figure><p>If AWS Summit was exciting, re:Invent was mind-blowing. The sheer scale of the event — with over 50,000 attendees, hundreds of sessions, and countless networking opportunities — was unlike anything I’d experienced.</p><p>But the highlight wasn’t just attending. It was what came next.</p><h3>Road to re:Invent Hackathon: The $100,000 Win</h3><p>This is where my Community Builder journey reached a pinnacle I never imagined possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J_cq3y8wF-RgHU7aCZLYvw.png" /></figure><h3>The Opportunity</h3><p>The Road to re:Invent Hackathon was organized for the AWS Community Builders and Heros as an exclusive opportunity to participate in a unique hackhathon challenge in a bus from LA to Vegas. The challenge was to build innovative solutions using AWS technologies, and the stakes were high — the winning team would receive <strong>$100,000 in prize money</strong>.</p><h3>The Hackathon Journey</h3><p>The weeks leading up to the hackathon were intense. We:</p><ul><li>Brainstormed innovative use cases for AWS services</li><li>Prototyped multiple solutions before settling on our final idea</li><li>Leveraged our AWS credits to build and test our application</li><li>Collaborated across time zones using the tools and practices we’d learned</li><li>Supported each other through challenges and setbacks</li></ul><p><a href="https://www.linkedin.com/feed/update/urn:li:activity:7404912872029573120/">#aws #serverless #architecture #hackathon #devtools #roadtoreinvent #cloudcomputing #innovation | Harry S.</a></p><h3>The Win</h3><p>When they announced our team as the winners, it felt surreal. The $100,000 prize was life-changing, but what meant even more was the validation — validation that our hard work, our learning, and our community contributions had led to something tangible and impactful.</p><blockquote><em>This win wasn’t just mine. It belonged to my teammates who brought their A-game, the Community Heros and Builders who supported and encouraged us, the AWS experts who answered our questions along the way, and the program itself, which created the environment for this opportunity to exist.</em></blockquote><h3>Year Two: Paying It Forward</h3><p>As I enter my second year as an AWS Community Builder, my focus has shifted. I’m no longer just learning and growing — I’m actively seeking new opportunities to grow as well as helping others do the same.</p><p>I’ve started:</p><ul><li>Mentoring and helping aspiring Community Builders</li><li>Participating and contributing to local AWS meetups and virtual ones</li><li>Creating more comprehensive tutorials, videos, and blog posts.</li><li>Sharing my hackathon experience to inspire others</li><li>Working with fellow community builders on new ideas and innitiatives</li></ul><h3>What the AWS Community Builders Program Really Offers</h3><p>Let me break down the tangible benefits:</p><h3>🔧 Technical Benefits</h3><ul><li>AWS credits for hands-on learning and experimentation</li><li>Early access to new services and features</li><li>Direct access to AWS service teams and experts</li><li>Exclusive technical content and workshops</li><li>Opportunities to beta test new services</li></ul><h3>💼 Professional Benefits</h3><ul><li>Recognition in the AWS community</li><li>Speaking opportunities at AWS events</li><li>Networking with global cloud professionals</li><li>Career advancement through visibility and expertise</li><li>Potential collaboration opportunities</li></ul><h3>🌍 Community Benefits</h3><ul><li>A platform to share knowledge and help others</li><li>Access to a supportive global community</li><li>Opportunities to volunteer at major AWS events</li><li>Exclusive hackathons and competitions</li><li>Mentorship from experienced builders</li></ul><h3>🎯 Personal Benefits</h3><ul><li>Confidence to pursue ambitious projects</li><li>Friendships with like-minded individuals</li><li>Sense of belonging to something larger</li><li>Motivation to continuously learn and improve</li><li>Stories and experiences that last a lifetime</li></ul><h3>Advice for Aspiring Community Builders</h3><p>If you’re considering applying for the AWS Community Builders program when applications open in January 2026, here’s my advice:</p><h3>Before You Apply</h3><p><strong>1. Start creating content now</strong></p><p>Don’t wait for acceptance to start sharing. Blog posts, videos, social media posts — all count.</p><p><strong>2. Engage with the community</strong></p><p>Answer questions on forums, participate in discussions, and attend meetups. Show that you’re already a community member.</p><p><strong>3. Choose your focus area</strong></p><p>The program has different categories. Pick one that aligns with your interests and expertise.</p><p><strong>4. Document your contributions</strong></p><p>Keep track of your blog posts, talks, open-source contributions, and community activities.</p><p><strong>5. Be authentic</strong></p><p>Don’t try to be someone you’re not. The program values genuine passion and commitment.</p><h3>In Your Application</h3><p><strong>1. Show, don’t just tell</strong></p><p>Provide links to your content, talks, and contributions.</p><p><strong>2. Explain your why</strong></p><p>What drives you to share knowledge? What impact do you want to make?</p><p><strong>3. Highlight your unique perspective</strong></p><p>What makes your voice valuable to the community?</p><p><strong>4. Demonstrate consistency</strong></p><p>One viral post is great, but a consistent contribution is better.</p><p><strong>5. Express your commitment</strong></p><p>The program is looking for people who will actively participate.</p><h3>After Acceptance</h3><p><strong>1. Engage immediately</strong></p><p>Join the Slack workspace, introduce yourself, and start conversations.</p><p><strong>2. Set goals</strong></p><p>What do you want to achieve as a Community Builder? Write it down.</p><p><strong>3. Be generous</strong></p><p>Help others, share your knowledge, and make connections.</p><p><strong>4. Take advantage of opportunities</strong></p><p>Apply for speaking slots, volunteer at events, and participate in hackathons.</p><p><strong>5. Stay consistent</strong></p><p>The program rewards ongoing contribution, not just initial enthusiasm.</p><h3>The Ripple Effect</h3><p>Looking back at my journey, I realize that the AWS Community Builders program didn’t just change my career — it changed how I think about community, learning, and success.</p><p>The $100,000 from the hackathon was incredible, but the real prize was:</p><ul><li>The skills I developed</li><li>The people I met</li><li>The confidence I gained</li><li>The doors that opened</li><li>The impact I made</li></ul><blockquote><em>Every blog post I wrote helped someone solve a problem. Every question I answered saved someone hours of frustration. Every connection I made enriched both our lives. This is the true power of community.</em></blockquote><h3>Looking Forward</h3><p>As the application window for the next cohort approaches, I’m excited to see new faces join this incredible program. The AWS ecosystem is growing rapidly, and I’m excited to see new builders.</p><p>If you’re reading this and wondering whether you should apply, let me make it simple: <strong>Yes, you should.</strong></p><p>You don’t need to be an expert. You don’t need to have all the answers. You just need:</p><ul><li>A genuine interest in AWS and cloud technologies</li><li>A willingness to learn and share</li><li>A commitment to helping others</li><li>The courage to put yourself out there</li></ul><blockquote><em>The AWS Community Builders program gave me opportunities I never dreamed possible. It can do the same for you.</em></blockquote><h3>Final Thoughts</h3><p>Two years ago, I was just another cloud enthusiast trying to learn AWS and share what I discovered. Today, I’m part of a global community of builders, I’ve won a major hackathon, I’ve volunteered at some of the biggest tech events in the world, and I’ve helped many others on their cloud journey.</p><p>None of this would have been possible without the AWS Community Builders program.</p><p><strong>The application window opens in January 2026. Start preparing now. Your journey could be just as transformative as mine.</strong></p><h3>📚 Resources</h3><ul><li><strong>AWS Community Builders Program</strong>: <a href="https://community.aws/">community.aws</a></li><li><strong>Application Information</strong>: Check the <a href="https://builder.aws.com/">AWS builders page</a> for updates and follow <a href="https://www.linkedin.com/in/jasonrobertdunn/"><strong>Jason Dunn </strong></a>on LinkedIn for latest first hand updates.</li><li><strong>My Journey</strong>: Follow me on <a href="https://www.linkedin.com/in/harpreetsiddhu/">LinkedIn</a> for more stories and tips and updates about AWS Community Builder program.</li><li><strong>Questions?</strong> Feel free to reach out — I’m always happy to help aspiring Community Builders</li></ul><h3>💬 Let’s Connect</h3><p><em>Are you planning to apply for the AWS Community Builders program? What questions do you have? Drop a comment below, and let’s start a conversation!</em></p><p>If you found this post helpful, please give it a clap 👏 and share it with others who might benefit from learning about the AWS Community Builders program.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=711f199cd68c" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/aws-community-builders-program-complete-guide-for-2026-applications-711f199cd68c">AWS Community Builders Program — Complete Guide for 2026 Applications</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[15 Strategies To Optimize Amazon EC2 Costs Without Compromising Performance]]></title>
            <link>https://aws.plainenglish.io/15-strategies-to-optimize-amazon-ec2-costs-without-compromising-performance-4a8f71340d33?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/4a8f71340d33</guid>
            <category><![CDATA[cost-reduction]]></category>
            <category><![CDATA[cloud-cost-optimization]]></category>
            <category><![CDATA[spot-instances]]></category>
            <category><![CDATA[aws-savings-plan]]></category>
            <category><![CDATA[ec2-reserved-instances]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Wed, 06 Aug 2025 03:28:42 GMT</pubDate>
            <atom:updated>2025-08-07T00:07:34.485Z</atom:updated>
            <content:encoded><![CDATA[<h4>From reserved instances to serverless — learn what works.</h4><p>Amazon EC2 is one of the widely used AWS services in the modern cloud computing era — powering everything from small development environments to enterprise-grade production systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wrU856TdbCzLZPcCkuOOAA.png" /></figure><p>Running EC2 on demand takes less than a minute, and it&#39;s that convenient, but it can also become one of the <strong>most expensive line items</strong> on your AWS bill. Whether you’re managing a lean startup infrastructure or operating at cloud scale, <strong>cost optimization is critical</strong>.</p><p>In this blog, I’ll walk you through <strong>15 proven strategies</strong> — with practical steps and relatable examples — to help you <strong>reduce EC2 spend while maintaining (or improving!) performance.</strong></p><h3>🧠 1. Choose the Right Instance Family</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*effi82d8VTuC49Fu.jpg" /><figcaption><strong>AWS EC2 Instance Types</strong></figcaption></figure><p>AWS offers over a dozen EC2 instance families tailored for different workloads: compute-optimized (C), memory-optimized (R, X), storage-optimized (I, D), and accelerated computing (P, G).</p><p><strong>✅ Practical Tip</strong>:<br> Use the <a href="https://aws.amazon.com/ec2/instance-types/">AWS Instance Selector</a> to compare instance families based on vCPU, RAM, network performance, etc.</p><p><strong>📌 Example</strong>:<br> If you’re an <strong>e-commerce platform running a large product catalog</strong>, choosing <strong>R-series</strong> (memory-optimized) instances can drastically improve DB performance while lowering cost per transaction.</p><h3>🛒 2. Use Reserved Instances (RIs) for Predictable Workloads</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cfpbXES_ofVYnsAdekl-CA.png" /><figcaption><strong>AWS Reserved Instance Discount</strong></figcaption></figure><p>RIs allow you to <strong>commit to a 1- or 3-year term</strong> for a specific instance type and receive up to <strong>70% discount</strong> over On-Demand pricing.</p><p><strong>✅ Practical Tip</strong>:<br> Use AWS Cost Explorer to analyze historical usage and identify candidates for RIs. Choose <strong>Standard RIs</strong> for fixed workloads and <strong>Convertible RIs</strong> if you want some flexibility.</p><p><strong>📌 Example</strong>:<br> If you’re <strong>running a database with consistent usage patterns</strong>, RIs are a no-brainer to save money.</p><h3>🌀 3. Leverage AWS Compute Savings Plans</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*01zgkzLPPBpPPwlf4Dt3Mw.png" /></figure><p>Compute Savings Plans offer a <strong>60% discount</strong> as compared to RIs, but with <strong>greater flexibility</strong>. You can choose between:</p><ul><li><strong>Compute Savings Plans</strong> — apply across all compute services (EC2, Fargate, Lambda)</li><li><strong>EC2 Instance Savings Plans</strong> — apply to specific instance families</li></ul><p><strong>📌 Example</strong>:<br> If you’re a <strong>SaaS provider with microservices spread across EC2</strong>, Compute Savings Plans can save you money even if you switch instance types or regions.</p><h3>⚡ 4. Use Spot Instances for Interruptible Workloads</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/811/0*BxW5OewTxt5b3ICv.png" /></figure><p>Spot Instances let you access unused EC2 capacity at up to <strong>90% off</strong>, but they can be interrupted by AWS with two minutes’ notice.</p><p><strong>✅ Practical Tip</strong>:<br> Use <strong>Spot Fleets</strong> or <strong>EC2 Auto Scaling groups</strong> with mixed instances and capacity pools for better availability.</p><p><strong>📌 Example</strong>:<br> If you’re <strong>processing large data sets in batches</strong>, Spot is ideal for drastically reducing costs.</p><h3>🧬 5. Consider AWS Graviton Instances</h3><p>Graviton2 and Graviton3 instances (based on ARM architecture) offer <strong>better performance per dollar</strong> than traditional x86-based instances — especially for containerized workloads.</p><p><strong>✅ Practical Tip</strong>:<br>Start with non-prod environments to verify compatibility. Use Graviton-ready AMIs and test performance before going all in.</p><p><strong>📌 Example</strong>:<br> If you’re running <strong>microservices on ECS/EKS</strong>, migrating to Graviton can yield up to 40% cost savings.</p><h3>📏 6. Rightsize Your Instances</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/1*ndVztjRWaYnuxSCE9mTDHA.png" /><figcaption><strong>AWS Compute Optimize</strong></figcaption></figure><p>It’s easy to over-provision “just in case” — but underutilized instances are silent cost eaters.</p><p><strong>✅ Practical Tip</strong>:<br> Use <strong>Amazon CloudWatch</strong> metrics and <strong>AWS Compute Optimizer</strong> to identify over-provisioned instances and downsize accordingly.</p><p><strong>📌 Example</strong>:<br> If you’re running at <strong>&lt;50% CPU/RAM usage</strong>, switching m5.large to t3.medium could cut costs in half.</p><h3>🕰️ 7. Use EC2 Instance Scheduler</h3><p>Development and QA environments often don’t need to run 24/7.</p><p><strong>✅ Practical Tip</strong>:<br> Use the AWS Instance Scheduler to start/stop EC2 instances based on a schedule — like weekdays 9 am–6 pm.</p><p><strong>📌 Example</strong>:<br> If you’re a <strong>dev team running tests only during business hours</strong>, scheduling EC2 uptime can reduce costs by ~60%.</p><h3>🌐 8. Elastic Load Balancing (ELB) for Efficient Traffic Distribution</h3><p>Use <strong>ALB or NLB</strong> to distribute incoming traffic across EC2 instances dynamically — improving performance and cost-effectiveness.</p><p><strong>✅ Practical Tip</strong>:<br> Set up <strong>target groups and health checks</strong> to ensure traffic only goes to healthy instances.</p><p><strong>📌 Example</strong>:<br> If you’re a <strong>retail site experiencing peak-hour traffic</strong>, ELB prevents overprovisioning while ensuring availability.</p><h3>📈 9. Auto Scaling for Dynamic Demand</h3><p>Don’t manually manage capacity. Use <strong>Auto Scaling</strong> to scale in/out based on actual demand.</p><p><strong>✅ Practical Tip</strong>:<br> Define scaling policies based on <strong>CloudWatch alarms</strong>, CPU utilization, or custom metrics.</p><p><strong>📌 Example</strong>:<br> If you’re running <strong>traffic-sensitive applications</strong>, Auto Scaling lets you run lean during off-peak hours and scale up only when needed.</p><h3>🔍 10. Monitoring and Optimization Tools</h3><p>Tools like <strong>AWS Cost Explorer</strong>, <strong>CloudWatch</strong>, <strong>Compute Optimizer</strong>, and <strong>Trusted Advisor</strong> are essential for cost insights.</p><p><strong>📌 Example</strong>:<br> If you’re a <strong>financial services company with multiple environments</strong>, Cost Explorer helps you identify underutilized or idle resources to clean up.</p><h3>🧹 11. Delete Stopped Instances</h3><p>Stopped instances still incur charges for <strong>EBS volumes</strong> and <strong>Elastic IPs</strong>.</p><p><strong>✅ Practical Tip</strong>:<br> Use Lambda + CloudWatch Events to detect and delete stopped instances older than a threshold (e.g., 7 days).</p><p><strong>📌 Example</strong>:<br> If you’re a <strong>dev team leaving instances stopped for weeks</strong>, you’re paying for storage you don’t need.</p><h3>💤 12. Enable Hibernation Mode (for Supported Instances)</h3><p>Preserve memory state and resume faster with hibernation — ideal for apps with long init times.</p><p><strong>📌 Example</strong>:<br> If you’re running <strong>analytics workloads with heavy boot logic</strong>, hibernation saves both time and money.</p><h3>🌎 13. Minimize Unnecessary Data Transfer</h3><p><strong>Cross-region data transfer</strong> and outbound traffic are often overlooked cost factors.</p><p><strong>✅ Practical Tip</strong>:<br> Use VPC endpoints, consolidate resources within a single region, and consider <strong>Direct Connect</strong> for hybrid networks.</p><p><strong>📌 Example</strong>:<br> If you’re a <strong>multinational org with a multi-region setup</strong>, moving resources closer together can save thousands in transfer fees.</p><h3>☁️ 14. Use Serverless Where Possible</h3><p>Not every workload needs a full EC2 instance. Services like <strong>AWS Lambda</strong>, <strong>API Gateway</strong>, and <strong>DynamoDB</strong> often offer better pricing and scalability.</p><p><strong>📌 Example</strong>:<br> If you’re running <strong>REST APIs or lightweight services</strong>, going serverless can eliminate idle EC2 costs.</p><h3>🔁 15. Continuously Review and Refine</h3><p>Cost optimization is not a one-time exercise. Make it part of your operational rhythm.</p><p><strong>✅ Suggested Review Cycle</strong>:</p><ul><li>Weekly: CloudWatch alerts &amp; dashboard reviews</li><li>Monthly: RI/Savings Plan evaluation</li><li>Quarterly: Instance family migration, tagging review, unused resource cleanup</li></ul><h3>🎯 Conclusion</h3><p>Amazon EC2 gives you immense power — but with great power comes a hefty bill (unless you manage it smartly). By adopting these <strong>15 cost optimization strategies</strong>, teams can achieve:</p><p>✅ Significant cost savings<br>✅ Improved infrastructure efficiency<br>✅ Better resource utilization<br>✅ Higher operational maturity</p><p>Whether you’re a cloud architect, DevOps engineer, or FinOps lead, these practices can help you get the most out of your EC2 investment.</p><p>📌 <em>Looking to implement these in your organization? Start with right-sizing, scheduling, and Reserved Instances — they offer the fastest returns.</em></p><p>🔖 If you found this helpful, consider bookmarking or clapping 👏 to help others find it.</p><h3>A message from our Founder</h3><p><strong>Hey, </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>Sunil</strong></a><strong> here.</strong> I wanted to take a moment to thank you for reading until the end and for being a part of this community.</p><p>Did you know that our team run these publications as a volunteer effort to over 200k supporters? <strong>We do not get paid by Medium</strong>!</p><p>If you want to show some love, please take a moment to <strong>follow me on </strong><a href="https://linkedin.com/in/sunilsandhu"><strong>LinkedIn</strong></a><strong>, </strong><a href="https://tiktok.com/@messyfounder"><strong>TikTok</strong></a><strong> and </strong><a href="https://instagram.com/sunilsandhu"><strong>Instagram</strong></a>. And before you go, don’t forget to <strong>clap</strong> and <strong>follow</strong> the writer️!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4a8f71340d33" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/15-strategies-to-optimize-amazon-ec2-costs-without-compromising-performance-4a8f71340d33">15 Strategies To Optimize Amazon EC2 Costs Without Compromising Performance</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Serverless Insurance Quote System with AWS — A Complete Tutorial]]></title>
            <link>https://towardsaws.com/serverless-insurance-quote-system-with-aws-a-complete-tutorial-d3e6c8e54863?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/d3e6c8e54863</guid>
            <category><![CDATA[lambda-function]]></category>
            <category><![CDATA[tech-tutorials]]></category>
            <category><![CDATA[api-gateway]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[scalable-applications]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Tue, 08 Jul 2025 05:31:09 GMT</pubDate>
            <atom:updated>2025-07-08T05:31:09.112Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uF9_FplBq0EbLPUy1bSl5w.png" /><figcaption>Serverless Insurance Quote System with AWS Service</figcaption></figure><h4>Building a Production-Ready</h4><h3>Serverless Insurance Quote System with AWS — A Complete Tutorial</h3><h4><em>Learn how to build an enterprise-grade serverless application using event-driven architecture, secure authentication, and modern cloud patterns</em></h4><p>Ever wondered how to build a <strong>real-world serverless application</strong> that handles complex business logic, user authentication, and scales automatically?</p><p>Today, we’ll build a POC for the <strong>Insurance Quote System</strong> that demonstrates enterprise-grade serverless patterns, which can be enhanced into a real system.</p><p><strong>🚀 Live Demo:</strong> <a href="https://d1fstpoqyc3odw.cloudfront.net/">Try it here</a> — For an end-to-end experience, please contact me <a href="https://www.linkedin.com/in/harpreetsiddhu/">here </a>for access code.</p><p><strong>📚 Full Code:</strong> <a href="https://github.com/hsiddhu2/Serverless-Insurance-Quote-System">GitHub Repository</a></p><p><strong>By the end of this tutorial, you’ll have built:</strong></p><ul><li>A <strong>multi-insurance quote system</strong> (Auto, Life, Home)</li><li><strong>Event-driven architecture</strong> with guaranteed message delivery</li><li><strong>Secure authentication</strong> with JWT tokens</li><li><strong>Session-based access control</strong> for anonymous users</li><li><strong>Real-time premium calculations support</strong></li><li><strong>Built-in support for the API Rate limiting</strong></li><li><strong>Duplicate requests prevention</strong> using smart database design</li></ul><h3>🏗 Architecture Overview</h3><p>This system uses a <strong>modern serverless architecture</strong> that’s inherently highly available, scalable, and cost-effective.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ffnSH5As8nJ1W3qhEwerbQ.png" /></figure><p><strong>Here is the high-level flow:</strong></p><p>User → CloudFront → S3 (Frontend)<br> ↓<br>Browser → API Gateway → Lambda Functions<br> ↓<br>SNS Topic → SQS Queues → Processing Lambdas → DynamoDB</p><p><strong>Why This Architecture Rocks:</strong></p><ul><li>✅ <strong>Auto-scaling:</strong> Handles 1 user or 10,000 users seamlessly</li><li>✅ <strong>Cost-effective:</strong> Pay only for what you use</li><li>✅ <strong>Reliable:</strong> Event-driven processing ensures no lost quotes</li><li>✅ <strong>Secure:</strong> Multiple layers of security protection</li></ul><blockquote><strong>In each AWS Region, and for each function, the lambda concurrency scaling rate is 1,000 execution environment instances every 10 seconds (or 10,000 requests per second every 10 seconds).</strong></blockquote><blockquote>In other words, every 10 seconds, Lambda can allocate at most 1,000 additional execution environment instances, or accommodate 10,000 additional requests per second, to each of your functions.</blockquote><h3>🛠️ Tech Stack</h3><p><strong>Frontend: </strong>HTML5/CSS3 + Vanilla JavaScript, Session-based access control, Responsive design</p><p><strong>Backend:</strong></p><ul><li><strong>AWS Lambda</strong> (Python 3.10)</li><li><strong>API Gateway</strong> (HTTP API)</li><li><strong>DynamoDB</strong> (NoSQL database)</li><li><strong>SNS/SQS</strong> (Message queuing)</li><li><strong>AWS Cognito</strong> (Authentication)</li><li><strong>CloudFront + S3</strong> (CDN + Hosting)</li><li><strong>Secrets Manager</strong> (Secure configuration)</li></ul><h3>📋 Prerequisites</h3><ul><li>AWS Account with administrative access</li><li>Basic understanding of AWS services</li><li>Text editor for code files</li></ul><p><strong>Estimated Time:</strong> 2–3 hours<br><strong>Estimated Cost:</strong> $5–10/month for moderate usage</p><h3>🚀 Step-by-Step Tutorial Using AWS Console</h3><p>Follow this step-by-step guide to build and deploy a production-ready serverless application using only the AWS Console</p><h4>Step 1: Create a DynamoDB Table</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bkn4mlu3NFDbf_rFGa8yiQ.png" /><figcaption>Create a Table in Dynamo DB</figcaption></figure><ol><li>Navigate to <strong>DynamoDB Console</strong> → <strong>Tables</strong> → <strong>Create table</strong></li><li>Configure table:</li><li><strong>Table name:</strong>InsuranceQuoteRequestsV2</li></ol><ul><li><strong>Partition key:</strong>compositeKey(String)</li><li><strong>Settings:</strong> Use default settings</li><li>Click <strong>Create table</strong></li></ul><p><strong>Why this design?</strong> The composite key <strong><em>email#insuranceType</em></strong> prevents duplicate quotes per user per insurance type.</p><h3>Step 2: Set Up SNS Topic and SQS Queues</h3><p><strong>2.1 Navigate to SNS Console → Topics → Create topic</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*-mTxHFKhnizJaJ3PvjnKcw.png" /><figcaption>Create SNS Topic</figcaption></figure><ul><li><strong>Type:</strong> Standard</li><li><strong>Name:</strong>Insurance-Quote-Requests</li><li>Click <strong>Create topic</strong></li></ul><p><strong>2.2 Create SQS Queues (repeat for each):</strong></p><p>Navigate to <strong>SQS Console</strong> → <strong>Create queue</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ayMXom8rsPqtS0qGmkgotA.png" /><figcaption>Create 3 SQS Queues for three line of business I.e. Life, Auto, Home</figcaption></figure><p><strong>Queue 1:</strong></p><ul><li><strong>Name:</strong>vehicle-insurance-quotes</li><li><strong>Type:</strong> Standard</li><li>Click <strong>Create queue</strong></li></ul><p><strong>Queue 2:</strong></p><ul><li><strong>Name:</strong>home-insurance-quotes</li><li><strong>Type:</strong> Standard</li><li>Click <strong>Create queue</strong></li></ul><p><strong>Queue 3:</strong></p><ul><li><strong>Name:</strong>life-insurance-quotes</li><li><strong>Type:</strong> Standard</li><li>Click <strong>Create queue</strong></li></ul><p><strong>2.3 Subscribe Queues to SNS Topic</strong></p><p>For each queue, go to the <strong>SNS Console</strong> → <strong>Topics</strong> → <strong>Insurance-Quote-Requests→</strong> <strong>Create a subscription</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kdTd3UIYZEbql0Cs3IRe_Q.png" /><figcaption>Subscribe SQS Queue to Topic ARN</figcaption></figure><p><strong>Vehicle Queue Subscription:</strong></p><ul><li><strong>Protocol:</strong> Amazon SQS</li><li><strong>Endpoint:</strong> Selectvehicle-insurance-quotesARN</li><li><strong>Subscription filter policy:</strong></li></ul><pre>{<br>  &quot;insuranceType&quot;: [&quot;auto&quot;, &quot;vehicle&quot;]<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CR0wpYI6aJGjuyAMbwyy4w.png" /><figcaption>After creating subscription and adding subscription filter policy</figcaption></figure><p><strong>Home Queue Subscription:</strong></p><ul><li><strong>Protocol:</strong> Amazon SQS</li><li><strong>Endpoint:</strong> Selecthome-insurance-quotesARN</li><li><strong>Subscription filter policy:</strong></li></ul><pre>{<br>  &quot;insuranceType&quot;: [&quot;home&quot;]<br>}</pre><p><strong>Life Queue Subscription:</strong></p><ul><li><strong>Protocol:</strong> Amazon SQS</li><li><strong>Endpoint:</strong> Select</li><li>life-insurance-quotesARN</li><li><strong>Subscription filter policy:</strong></li></ul><pre>{<br>  &quot;insuranceType&quot;: [&quot;life&quot;]<br>}</pre><h3>Step 3: Create AWS Cognito User Pool</h3><p><strong>3.1</strong> Navigate to <strong>Cognito Console</strong> → <strong>User pools</strong> → <strong>Create user pool</strong></p><p><strong>3.2 Configure sign-in experience:</strong></p><ul><li><strong>Authentication providers:</strong> Cognito user pool</li><li><strong>Cognito user pool sign-in options:</strong> Email</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9HaFcNkX5_r9SN2ugLo8eA.png" /><figcaption>Create a Cognito User Pool — Follow the arrows only</figcaption></figure><p><strong>3.6 Integrate your app:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yy3FNQMoMwphVSzbvcn_EQ.png" /><figcaption>Click on this link and go edit the Login pages</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H92yYapbZh7r1QJFMTXViQ.png" /><figcaption>Update the configuration of login callback URLs as mentioned below.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F5jHZ7Q0P4QU9uMCKdeUlw.png" /></figure><ul><li><strong>User pool name:</strong>InsuranceQuoteUsers</li><li><strong>Hosted authentication pages:</strong> Use Cognito Hosted UI</li><li><strong>Domain:</strong>insurance-quotes-[YOUR-ACCOUNT-ID]</li><li><strong>App client name:</strong>InsuranceQuoteWebClient</li><li><strong>Client secret:</strong> Don’t generate</li><li><strong>Callback URLs:</strong><a href="http://localhost:5500/">http://localhost:5500/</a></li><li><strong>Sign out URLs:</strong><a href="http://localhost:5500/">http://localhost:5500/</a></li><li>Click <strong>Create user pool</strong></li></ul><p><strong>3.7</strong> Note down:</p><ul><li>User Pool ID</li><li>App Client ID</li><li>Cognito Domain</li></ul><h3>Step 4: Create Lambda Functions</h3><p><strong>4.1</strong> Navigate to <strong>Lambda Console</strong> → <strong>Create function</strong></p><p>Create these 6 functions (repeat process for each):</p><h4>Function 1: Submit Quote Lambda</h4><ul><li><strong>Function name:</strong>SubmitQuoteLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/submitQuote.py</li><li><strong>Environment variables:</strong></li><li>SNS_TOPIC_ARN: [Your SNS Topic ARN]</li><li>DYNAMODB_TABLE: InsuranceQuoteRequests</li></ul><h4>Function 2: Calculate Premium Lambda</h4><ul><li><strong>Function name:</strong>CalculatePremiumLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/calculatePremium.py</li></ul><h4>Function 3: Get User Quotes Lambda</h4><ul><li><strong>Function name:</strong>GetUserQuotesLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/getUserQuotes.py</li><li><strong>Environment variables:</strong></li><li>DYNAMODB_TABLE:InsuranceQuoteRequest</li></ul><h4>Function 4: Auto Quote Lambda</h4><ul><li><strong>Function name:</strong>AutoQuoteLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/autoQuoteLambda.py</li><li><strong>Environment variables:</strong></li><li>DYNAMODB_TABLE:InsuranceQuoteRequestsV2</li></ul><h4>Function 5: Home Quote Lambda</h4><ul><li><strong>Function name:</strong>HomeQuoteLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/homeQuoteLambda.py</li><li><strong>Environment variables:</strong></li><li>DYNAMODB_TABLE:InsuranceQuoteRequestsV2</li></ul><h4>Function 6: Life Quote Lambda</h4><ul><li><strong>Function name:</strong>LifeQuoteLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/LifeQuoteLambda.py</li><li><strong>Environment variables:</strong></li><li>DYNAMODB_TABLE:InsuranceQuoteRequestsV2</li></ul><h3>Step 5: Configure Lambda Permissions</h3><p>For each Lambda function, add the necessary permissions:</p><p><strong>5.1</strong> Go to <strong>Lambda Console</strong> → Select function → <strong>Configuration</strong> → <strong>Permissions</strong></p><p><strong>5.2</strong> Click <strong>Execution role</strong> → <strong>Add permissions</strong> → <strong>Attach policies</strong></p><p><strong>CalculatePremiumLambda: </strong>Only need “AWSLambdaBasicExecutionRole”</p><p><strong>SubmitQuoteLambda:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hYGg7LcJ8p89dat3N5eE1A.png" /><figcaption>Add — Permission to publish messgae to SNS topic</figcaption></figure><p><strong>GetUserQuotesLambda, AutoQuoteLambda, HomeQuoteLambda, LifeQuoteLambda:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CtXtkIyRznYMpalLwmna2Q.png" /><figcaption>Add — Permission to read-write dynamo and Lambda SQS Execution Role</figcaption></figure><h3>Step 6: Set Up SQS Triggers for Processing Lambdas</h3><p><strong>6.1</strong> Navigate to <strong>Lambda Console</strong> → Select <strong>AutoQuoteLambda</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Foxk_wHxwUkghBw5PUNyvQ.png" /><figcaption>Adding Lambda Trigger in SQS</figcaption></figure><p><strong>6.2</strong> Click <strong>Add trigger</strong></p><ul><li><strong>Source:</strong> SQS</li><li><strong>SQS queue:</strong>vehicle-insurance-quotes</li><li><strong>Batch size:</strong> 10</li><li>Click <strong>Add</strong></li></ul><p><strong>6.3</strong> Repeat for other processing Lambdas:</p><ul><li>HomeQuoteLambda→home-insurance-quotes</li><li>LifeQuoteLambda→life-insurance-quotes</li></ul><h3>Step 7: Create API Gateway</h3><p><strong>7.1</strong> Navigate to <strong>API Gateway Console</strong> → <strong>Create API</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MDe0KaY3-16L5j40t_Fv_w.png" /><figcaption>Setup the API Gateway routes, configure integrations and CORS and deploy to a stage</figcaption></figure><p><strong>7.2</strong> Choose <strong>HTTP API</strong> → <strong>Build</strong></p><p><strong>7.3</strong> Configure API:</p><ul><li><strong>API name:</strong>InsuranceQuoteAPI</li><li><strong>Description:</strong>API for insurance quote system</li><li>Click <strong>Next</strong></li></ul><p><strong>7.4</strong> Configure routes (click <strong>Add route</strong> for each):</p><p><strong>Route 1:</strong></p><ul><li><strong>Method:</strong> POST</li><li><strong>Resource path:</strong>/submitQuote</li><li><strong>Integration:</strong> Lambda function →SubmitQuoteLambda</li></ul><p><strong>Route 2:</strong></p><ul><li><strong>Method:</strong> POST</li><li><strong>Resource path:</strong>/calculatePremium</li><li><strong>Integration:</strong> Lambda function →CalculatePremiumLambda</li></ul><p><strong>Route 3:</strong></p><ul><li><strong>Method:</strong> GET</li><li><strong>Resource path:</strong>/user/quotes</li><li><strong>Integration:</strong> Lambda function →GetUserQuotesLambda</li><li><strong>Authorization:</strong> JWT (configure with Cognito User Pool)</li></ul><p><strong>7.5</strong> Configure CORS:</p><ul><li><strong>Access-Control-Allow-Origin:</strong>*</li><li><strong>Access-Control-Allow-Headers:</strong>Content-Type, Authorization</li><li><strong>Access-Control-Allow-Methods:</strong>GET, POST, OPTIONS</li></ul><p><strong>7.6</strong> Deploy API and note the <strong>Invoke URL</strong></p><h3>Step 8: Create Secrets Manager Secret</h3><p><strong>8.1</strong> Navigate to <strong>Secrets Manager Console</strong> → <strong>Store a new secret</strong></p><p><strong>8.2</strong> Configure secret:</p><ul><li><strong>Secret type:</strong> Other type of secret</li><li><strong>Key:</strong>access-codes</li><li><strong>Value:</strong>DEMO2024,SECURE2024,PORTFOLIO2024</li><li><strong>Secret name:</strong>serverless-insurance-quote-app-access-codes</li><li>Click <strong>Store</strong></li></ul><h3>Step 9: Create a Validate Access Lambda</h3><p><strong>9.1</strong> Navigate to <strong>Lambda Console</strong> → <strong>Create function</strong></p><ul><li><strong>Function name:</strong>ValidateAccessLambda</li><li><strong>Runtime:</strong> Python 3.10</li><li><strong>Code:</strong> Copy frombackend/lambda/validateAccess.py</li><li><strong>Environment variables:</strong></li><li>ACCESS_CODES_SECRET_NAME:serverless-insurance-quote-app-access-codes</li></ul><p><strong>9.2</strong> Add permissions:</p><ul><li>Attach policy:</li><li>SecretsManagerReadWrite</li></ul><p><strong>9.3</strong> Add API Gateway route:</p><ul><li><strong>Method:</strong> POST</li><li><strong>Resource path:</strong>/validate-access</li><li><strong>Integration:</strong> Lambda function →ValidateAccessLambda</li></ul><h3>Step 10: Set Up S3 and CloudFront for Frontend</h3><p><strong>10.1</strong> Navigate to <strong>S3 Console</strong> → <strong>Create bucket</strong></p><ul><li><strong>Bucket name:</strong>insurance-quote-frontend-[random-string]</li><li><strong>Region:</strong> Same as your other resources</li><li><strong>Block all public access:</strong> Uncheck</li><li>Click <strong>Create bucket</strong></li></ul><p><strong>10.2</strong> Upload frontend files:</p><ul><li>Upload all files fromfrontend/folder</li><li>Make sure to updateconfig.jsWith your API Gateway URL and Cognito details and upload this to the S3 bucket.</li></ul><p><strong>10.4</strong> Create CloudFront distribution:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lWoGFjFOHKai5QzvdtGw6A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3uPji3ru7voAR5Inqan-nA.png" /></figure><ul><li>Navigate to <strong>CloudFront Console</strong> → <strong>Create distribution</strong></li><li><strong>Origin domain:</strong> Your S3 bucket website endpoint</li><li><strong>Default root object:</strong>index.html</li><li>Click <strong>Create distribution</strong></li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FBlZYVY8EXVI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DBlZYVY8EXVI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FBlZYVY8EXVI%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/3a4b4fe3a5337bb401a30638d81b691e/href">https://medium.com/media/3a4b4fe3a5337bb401a30638d81b691e/href</a></iframe><h3>Step 11: Configure Frontend</h3><p><strong>11.1</strong> Update frontend/config.js</p><pre>window.API_ENDPOINT = &quot;https://your-api-gateway-id.execute-api.region.amazonaws.com/Prod&quot;;<br>window.COGNITO_CONFIG = {<br>    userPoolId: &#39;your-user-pool-id&#39;,<br>    clientId: &#39;your-app-client-id&#39;,<br>    region: &#39;your-region&#39;,<br>    accountId: &#39;your-account-id&#39;<br>};</pre><p><strong>11.2</strong> Upload updated files to S3</p><p><strong>11.3</strong> Update Cognito callback URLs:</p><ul><li>Go to <strong>Cognito Console</strong> → Your User Pool → <strong>App integration</strong> → <strong>App clients</strong></li><li>Add your CloudFront URL to the <strong>Callback URLs</strong> and <strong>Sign out URLs</strong></li></ul><h3>Step 12: Test Your Application</h3><p><strong>12.1</strong> Access your CloudFront URL</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bP7erqod58a6-JE2TT8QKg.png" /></figure><p><strong>12.2 Test the flow:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RWGAYAMSbuboLcC8S_DMQg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KUW7nU-rQJHxIJu5rH5XBw.png" /></figure><ol><li>Enter access code (use one from your Secrets Manager)</li><li>Try calculating a premium</li><li>Submit a quote</li><li>Register/login with Cognito</li><li>View your quotes</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*upklTO1ttBNadRG0UCjcLQ.png" /></figure><h3>🔧 Troubleshooting Common Issues</h3><h3>Issue 1: CORS Errors</h3><p><strong>Solution:</strong> Ensure the API Gateway has proper CORS configuration for origins during development.</p><h3>Issue 2: Lambda Timeout</h3><p><strong>Solution:</strong> Increase timeout in Lambda Configuration → General configuration → Timeout.</p><h3>Issue 3: DynamoDB Access Denied</h3><p><strong>Solution:</strong> Check that the Lambda execution role has DynamoDB permissions.</p><h3>Issue 4: SNS/SQS Not Working</h3><p><strong>Solution:</strong> Verify SQS queue policies allow SNS to send messages.</p><h3>💡 Key Takeaways</h3><p>✅ <strong>Event-driven architecture</strong> enables scalable, decoupled systems<br>✅ <strong>Serverless</strong> reduces operational overhead and costs<br>✅ <strong>Proper security layers</strong> protect your application<br>✅ <strong>Infrastructure as Code</strong> makes deployments repeatable<br>✅ <strong>Composite keys</strong> solve complex business requirements</p><h3>🚀 What’s Next?</h3><p>Implement a three-tier architecture with a simple step-by-step tutorial.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FAnZ_xxbOSfI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DAnZ_xxbOSfI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FAnZ_xxbOSfI%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/eb0ecff4217fe5bf115570f69ba5e42b/href">https://medium.com/media/eb0ecff4217fe5bf115570f69ba5e42b/href</a></iframe><p>Found this helpful? Please share your implementation with me on my <a href="https://www.linkedin.com/in/harpreetsiddhu/">LinkedIn</a>, or need access to the app, request an access code from there.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3e6c8e54863" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/serverless-insurance-quote-system-with-aws-a-complete-tutorial-d3e6c8e54863">Serverless Insurance Quote System with AWS — A Complete Tutorial</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Real-Time Data Transformation with Amazon S3 Object Lambda]]></title>
            <link>https://towardsaws.com/real-time-data-transformation-with-amazon-s3-object-lambda-f9451c6a4c8a?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/f9451c6a4c8a</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[aws-lambda-functions]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[s3-bucket]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Wed, 28 May 2025 13:22:25 GMT</pubDate>
            <atom:updated>2025-05-28T13:22:25.853Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c1_LVVSlURFJEp4MDXysKQ.png" /></figure><h4>Topic: AWS Data Analytics (Level: Intermediate)</h4><h4>A Hands-On Guide for Developers</h4><p>Amazon S3 is the backbone of cloud object storage. But what if you could <strong>process and transform S3 data at the moment it’s retrieved</strong>, without altering the underlying object?</p><p><strong>Amazon S3 Object Lambda</strong> — a powerful feature that enables developers to inject custom code into S3 GET, HEAD, GetObjectTagging andLIST requests using AWS Lambda. 💡</p><p>So, instead of duplicating objects or preprocessing objects using a proxy layer in front of S3, Object Lambda allows you to run your code <em>as the object is retrieved</em>.</p><p>In this blog post, we’ll explore how S3 Object Lambda works, walk through real-world use cases, and build a <strong>hands-on lab</strong> with code that transforms object data on retrieval.</p><p>By the end, you’ll have a working prototype and a deep understanding of how to implement this in your solutions.</p><h3>🧠 What is Amazon S3 Object Lambda?</h3><p>Amazon S3 Object Lambda lets you <em>intercept</em> and <em>transform</em> S3 GetObject requests with your own AWS Lambda function. Think of it as middleware for S3 retrievals.</p><p>When a user or app requests an object, Object Lambda routes that request through your Lambda function before returning it to the requester. Your function can:</p><ul><li>🔒 <strong>Redact Confidential Data</strong> — Remove or mask sensitive data (e.g., PII) before it’s returned.</li><li>📄 <strong>Transform Data Format</strong> — Convert from JSON to CSV, XML to JSON, etc., on retrieval.</li><li>📁 <strong>Filter Rows/Columns</strong> — Deliver only relevant parts of large datasets.</li></ul><p>You can do all this in real time, and the original object in S3 stays untouched.</p><h3>🛠️ Architecture Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*PBMxnvmNxVo09fnfsvtkGQ.png" /><figcaption>Source — AWS Documentation</figcaption></figure><ul><li><strong>Client Request via Object Lambda Access Point</strong>: A user or application sends a GetObject request to an S3 Object Lambda Access Point, which acts as a layer over a standard S3 access point.</li><li><strong>Lambda Function Transforms Data</strong>: The Object Lambda Access Point invokes an AWS Lambda function, which retrieves the object from S3 via the standard access point and transforms it (e.g., redacts fields, filters records).</li><li><strong>Transformed Object Returned</strong>: The modified content is returned to the client as the final response, without altering the original object stored in the S3 bucket.</li></ul><p><strong>Bottom line</strong> — Instead of accessing data directly from a standard S3 bucket, the requester interacts through an <strong>Object Lambda Access Point</strong>, which invokes your <strong>Lambda function</strong> to modify or transform the object <strong>on-the-fly</strong> before delivering it.</p><h3>🧪 Hands-On Labs: <strong>Transforming data with S3 Object Lambda</strong></h3><p>In this tutorial, we’ll walk through the process of setting up an AWS Lambda function to transform objects stored in Amazon S3 using S3 Object Lambda. By the end of this guide, you’ll have a working example of redacting sensitive information from JSON files stored in S3.</p><h4>Prerequisites</h4><p>Before we begin, ensure you have the following:</p><ol><li><strong>AWS CLI</strong> installed and configured with appropriate permissions.</li><li><strong>Python 3.9 or later</strong> is installed on your local machine.</li><li>A <strong>GitHub repository</strong> to store your code (optional but recommended).</li><li>A JSON file named user-info.json containing sample data to test the transformation.</li></ol><h4>Github Repo:</h4><p><a href="https://github.com/hsiddhu2/s3-object-lambda.git"><em>https://github.com/hsiddhu2/s3-object-lambda.git</em></a></p><h4>Step 1: Create an S3 Bucket</h4><p>First, create a general-purpose S3 bucket to store your JSON file.</p><pre>aws s3api create-bucket - bucket demo-s3-object-lambda - region us-east-1 - create-bucket-configuration LocationConstraint=us-east-1</pre><p>Next, upload the below user-info.json file to the bucket:</p><pre>{<br>    &quot;userName&quot;: &quot;john_doe&quot;,<br>    &quot;ssn&quot;: &quot;123-456-6789&quot;,<br>    &quot;dob&quot;: &quot;1990-01-01&quot;,<br>    &quot;address&quot;: {<br>        &quot;street&quot;: &quot;123 Main St&quot;,<br>        &quot;city&quot;: &quot;Springfield&quot;,<br>        &quot;state&quot;: &quot;IL&quot;,<br>        &quot;zip&quot;: &quot;62701&quot;<br>    },<br>    &quot;phoneNumbers&quot;: [<br>        {<br>            &quot;type&quot;: &quot;home&quot;,<br>            &quot;number&quot;: &quot;555-1234&quot;<br>        },<br>        {<br>            &quot;type&quot;: &quot;work&quot;,<br>            &quot;number&quot;: &quot;555-5678&quot;<br>        }<br>    ]<br>}</pre><pre>aws s3 cp user-info.json s3://demo-s3-object-lambda/user-info.json</pre><h4>Step 2: Create an S3 Access Point</h4><p>Create an S3 Access Point for the bucket to allow controlled access.</p><pre>aws s3control create-access-point --account-id 123456789012 --name s3-object-lambda-ap --bucket demo-s3-object-lambda --region us-east-1</pre><p>Verify the access point:</p><pre>aws s3control get-access-point --account-id 123456789012 --name s3-object-lambda-ap</pre><h4>Step 3: Create a Lambda Execution Role</h4><p>Create an IAM role with the necessary permissions for the Lambda function. Follow the steps in the lambda-execution-role.md file in the <a href="https://github.com/hsiddhu2/s3-object-lambda.git">GitHub repository</a>.</p><pre>## Lambda Execution Role for S3 Object Lambda<br><br>This document provides the necessary IAM role and policies required for an AWS<br>Lambda function that processes and transforms JSON objects stored in Amazon S3<br>using S3 Object Lambda. The s3-object-lambda:WriteGetObjectResponse API is <br>used to return the transformed object.<br><br>### Steps:<br>1. Create an IAM role for the Lambda function. <br>Name is `S3-Object-Lambda-Execution-Role`.<br><br>2. Attach the following policies to the role:<br>   - `AmazonS3ObjectLambdaExecutionRolePolicy`<br>   - `AmazonS3ReadOnlyAccess`<br><br>3. Complete the role with the necessary trust relationship for Lambda.<br><br>### Add &quot;AmazonS3ObjectLambdaExecutionRolePolicy&quot; - AWS Managed Policy<br><br>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;logs:CreateLogGroup&quot;,<br>                &quot;logs:CreateLogStream&quot;,<br>                &quot;logs:PutLogEvents&quot;,<br>                &quot;s3-object-lambda:WriteGetObjectResponse&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}<br><br>### Add &quot;AmazonS3ReadOnlyAccess&quot; - AWS Managed Policy<br><br>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;s3:Get*&quot;,<br>                &quot;s3:List*&quot;,<br>                &quot;s3:Describe*&quot;,<br>                &quot;s3-object-lambda:Get*&quot;,<br>                &quot;s3-object-lambda:List*&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><h4>Step 4: Write the Lambda Function</h4><p>Write the transformation logic in a Python file named transform.py. You can find the complete code in the <a href="https://github.com/hsiddhu2/s3-object-lambda.git">GitHub repository.</a></p><pre>import boto3<br>import json<br>import urllib.request<br>from urllib.error import HTTPError, URLError<br>from botocore.config import Config<br><br>def lambda_handler(event, context):<br>    print(&#39;Event:&#39;, event)<br><br>    object_context = event.get(&quot;getObjectContext&quot;, {})<br>    s3_url = object_context.get(&quot;inputS3Url&quot;)<br>    request_route = object_context.get(&quot;outputRoute&quot;)<br>    request_token = object_context.get(&quot;outputToken&quot;)<br><br>    s3 = boto3.client(&#39;s3&#39;, config=Config(signature_version=&#39;s3v4&#39;))<br><br>    try:<br>        # Validate input<br>        if not s3_url or not request_route or not request_token:<br>            raise ValueError(&quot;Missing inputS3Url, outputRoute, or outputToken.&quot;)<br><br>        # Attempt to read from S3<br>        with urllib.request.urlopen(s3_url) as response:<br>            original_object = response.read().decode(&quot;utf-8&quot;)<br><br>        # Parse JSON<br>        data = json.loads(original_object)<br><br>        # Redact sensitive fields<br>        if &quot;ssn&quot; in data:<br>            ssn = data[&quot;ssn&quot;]<br>            data[&quot;ssn&quot;] = &quot;***-**-&quot; + ssn[-4:] if len(ssn) &gt;= 4 else &quot;****&quot;<br><br>        if &quot;dob&quot; in data:<br>            data[&quot;dob&quot;] = &quot;1990/**/**&quot;<br><br>        if &quot;address&quot; in data and isinstance(data[&quot;address&quot;], dict):<br>            addr = data[&quot;address&quot;]<br>            addr[&quot;street&quot;] = addr.get(&quot;street&quot;, &quot;&quot;).split()[0] + &quot; ****&quot;<br>            addr[&quot;city&quot;] = &quot;****&quot;<br>            addr[&quot;state&quot;] = &quot;**&quot;<br>            addr[&quot;zip&quot;] = addr.get(&quot;zip&quot;, &quot;&quot;)[:3] + &quot;**&quot;<br><br>        if &quot;phoneNumbers&quot; in data and isinstance(data[&quot;phoneNumbers&quot;], list):<br>            for phone in data[&quot;phoneNumbers&quot;]:<br>                if &quot;number&quot; in phone:<br>                    phone[&quot;number&quot;] = phone[&quot;number&quot;][:4] + &quot;****&quot;<br><br>        # Create uppercase version<br>        redacted_json_str = json.dumps(data, indent=2).upper()<br><br>        combined_output = (<br>            &quot;ORIGINAL OBJECT:\n&quot; + original_object + &quot;\n\n&quot; +<br>            &quot;TRANSFORMED OBJECT:\n&quot; + redacted_json_str<br>        )<br><br>        # Return transformed object<br>        s3.write_get_object_response(<br>            Body=combined_output,<br>            RequestRoute=request_route,<br>            RequestToken=request_token,<br>            StatusCode=200<br>        )<br>        return {&quot;status_code&quot;: 200}<br><br>    except (HTTPError, URLError) as e:<br>        error_msg = f&quot;Unauthorized or inaccessible object: {e.reason}&quot;<br>        print(&quot;HTTP error:&quot;, error_msg)<br><br>        s3.write_get_object_response(<br>            Body=error_msg,<br>            RequestRoute=request_route,<br>            RequestToken=request_token,<br>            StatusCode=e.code if hasattr(e, &#39;code&#39;) else 403<br>        )<br>        return {&quot;status_code&quot;: e.code if hasattr(e, &#39;code&#39;) else 403}<br><br>    except Exception as e:<br>        error_msg = f&quot;Internal error: {str(e)}&quot;<br>        print(&quot;Exception:&quot;, error_msg)<br><br>        s3.write_get_object_response(<br>            Body=error_msg,<br>            RequestRoute=request_route,<br>            RequestToken=request_token,<br>            StatusCode=500<br>        )<br>        return {&quot;status_code&quot;: 500}</pre><h4>Step 5: Deploy the Lambda Function</h4><p>Zip the transform.py file:</p><pre>zip transform.zip transform.py</pre><p>Create the Lambda function using the AWS CLI:</p><pre>aws lambda create-function \<br>    --function-name s3-object-lambda-transform \<br>    --runtime python3.9 \<br>    --handler transform.handler \<br>    --role arn:aws:iam::123456789012:role/S3-Object-Lambda-Execution-Role \<br>    --zip-file fileb://transform.zip</pre><h4>Step 6: Create an S3 Object Lambda Access Point</h4><p>Create an S3 Object Lambda Access Point and associate it with the Lambda function:</p><pre>aws s3control create-access-point-for-object-lambda \<br>    --account-id 123456789012 \<br>    --name s3-object-lambda-ap-ol \<br>    --supporting-access-point-arn arn:aws:s3:us-east-1:123456789012:accesspoint/s3-object-lambda-ap \<br>    --object-lambda-configuration &#39;{&quot;SupportingAccessPoint&quot;: &quot;arn:aws:s3:us-east-1:123456789012:accesspoint/s3-object-lambda-ap&quot;, &quot;TransformationConfigurations&quot;: [{&quot;Actions&quot;: [&quot;GetObject&quot;], &quot;ContentTransformation&quot;: {&quot;AwsLambda&quot;: {&quot;FunctionArn&quot;: &quot;arn:aws:lambda:us-east-1:123456789012:function:s3-object-lambda-transform&quot;}}}]}&#39; \<br>    --region us-east-1</pre><h4>Step 7: Test the S3 Object Lambda Access Point</h4><p>Retrieve the transformed object using the S3 Object Lambda Access Point:</p><pre>aws s3api get-object --bucket s3-object-lambda-ap-ol --key user-info.json output.json</pre><h4>Step 8: Verify the Output</h4><p>Check the transformed output:</p><pre>cat output.json</pre><pre>ORIGINAL OBJECT:<br>{<br>    &quot;userName&quot;: &quot;john_doe&quot;,<br>    &quot;ssn&quot;: &quot;123-456-6789&quot;,<br>    &quot;dob&quot;: &quot;1990-01-01&quot;,<br>    &quot;address&quot;: {<br>        &quot;street&quot;: &quot;123 Main St&quot;,<br>        &quot;city&quot;: &quot;Springfield&quot;,<br>        &quot;state&quot;: &quot;IL&quot;,<br>        &quot;zip&quot;: &quot;62701&quot;<br>    },<br>    &quot;phoneNumbers&quot;: [<br>        {<br>            &quot;type&quot;: &quot;home&quot;,<br>            &quot;number&quot;: &quot;555-1234&quot;<br>        },<br>        {<br>            &quot;type&quot;: &quot;work&quot;,<br>            &quot;number&quot;: &quot;555-5678&quot;<br>        }<br>    ]<br>}<br><br>TRANSFORMED OBJECT:<br>{<br>  &quot;USERNAME&quot;: &quot;JOHN_DOE&quot;,<br>  &quot;SSN&quot;: &quot;***-**-6789&quot;,<br>  &quot;DOB&quot;: &quot;1990/**/**&quot;,<br>  &quot;ADDRESS&quot;: {<br>    &quot;STREET&quot;: &quot;123 ****&quot;,<br>    &quot;CITY&quot;: &quot;****&quot;,<br>    &quot;STATE&quot;: &quot;**&quot;,<br>    &quot;ZIP&quot;: &quot;627**&quot;<br>  },<br>  &quot;PHONENUMBERS&quot;: [<br>    {<br>      &quot;TYPE&quot;: &quot;HOME&quot;,<br>      &quot;NUMBER&quot;: &quot;555-****&quot;<br>    },<br>    {<br>      &quot;TYPE&quot;: &quot;WORK&quot;,<br>      &quot;NUMBER&quot;: &quot;555-****&quot;<br>    }<br>  ]<br>}</pre><h4>Step 9: Clean Up</h4><p>To avoid incurring unnecessary costs, clean up the resources:</p><pre>aws s3 rm s3://s3-object-lambda-ap-ol --recursive<br>aws s3api delete-bucket --bucket demo-s3-object-lambda --region us-east-1<br>aws s3control delete-access-point --account-id 123456789012 --name s3-object-lambda-ap<br>aws s3control delete-access-point-for-object-lambda --account-id 123456789012 --name s3-object-lambda-ap-ol<br>aws lambda delete-function --function-name s3-object-lambda-transform<br>aws iam delete-role --role-name S3-Object-Lambda-Execution-Role</pre><h3>Conclusion</h3><p>Congratulations! We have successfully set up an AWS Lambda function to transform S3 objects using S3 Object Lambda.</p><p>This hands-on guide demonstrated how to redact sensitive information from JSON files and return the transformed data.</p><p>For the complete code and additional details, please check out the <a href="https://github.com/hsiddhu2/s3-object-lambda.git">GitHub repository</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f9451c6a4c8a" width="1" height="1" alt=""><hr><p><a href="https://towardsaws.com/real-time-data-transformation-with-amazon-s3-object-lambda-f9451c6a4c8a">Real-Time Data Transformation with Amazon S3 Object Lambda</a> was originally published in <a href="https://towardsaws.com/">Towards AWS</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy SpringBoot Java App on EC2 with CICD Automation ✌️]]></title>
            <link>https://aws.plainenglish.io/deploy-springboot-java-app-on-ec2-with-cicd-automation-%EF%B8%8F-2915eb737db7?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/2915eb737db7</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[cicd]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Fri, 31 Jan 2025 02:26:06 GMT</pubDate>
            <atom:updated>2026-01-09T15:31:30.033Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_BRjP6iefnwH0tymNKrL6Q.png" /><figcaption>Deploy SpringBoot Java App on EC2 with CICD Automation</figcaption></figure><h4>Topic: AWS DevOps (Level: Intermediate)</h4><h4>In this project, we will learn how to deploy a Springboot application on an EC2 instance in AWS using GitHub Actions.</h4><p>Continuous Integration and Continuous Deployment (CI/CD) is at the heart of modern DevOps workflows, enabling seamless delivery of applications with minimal human intervention.</p><p>In this blog, I’ll walk you through a practical CI/CD pipeline implementation using <strong>GitHub Actions</strong> to deploy a Spring Boot application on an AWS EC2 instance.</p><h3>Overview 🖼️</h3><p>The architecture leverages GitHub Actions for CI/CD and AWS for deployment. Here’s a breakdown of the process:</p><h4>1. GitHub Actions Workflow</h4><ul><li><strong>Build Job</strong>: <br>+ Checkout code <br>+ Set up JDK (Java 21) <br>+ Build the project using Maven <br>+ Clean and archive artifacts <br>+ Upload the build artifact 🚀</li><li><strong>Deploy Job</strong>:<br>+ Download the artifact 📥<br>+ Set up the target path <br>+ Configure SSH 🔑<br>+ Deploy the artifact to the EC2 instance <br>+ Restart the Spring Boot application 🔄</li></ul><h4>2. AWS EC2 Setup</h4><ul><li><strong>Security Group </strong>- <strong>Inbound Rules:</strong><br> + TCP port 8080 to access the application<br> + SSH port 22 for deployment</li><li><strong>Security Group- Outbound Rules:<br> + </strong>Allow all outbound traffic 🌍</li><li><strong>Web Server</strong>:<br>+ Install the JDK of your choice ( &gt;17 recommended) <br>+ Create a “systemd” service configured to run the spring boot app.</li></ul><h3>Step-by-Step CI/CD Implementation 🛠️</h3><h4>Step 1: Create a Spring Boot Project and Set Up a GitHub Repository ️</h4><p>Create a new Spring Boot project using your preferred method (Spring Initializr, IDE, or CLI).</p><p>Make sure your pom.xml system is configured so Maven can manage dependencies.</p><p>Initialize a GitHub repository and push your project code. This will serve as the foundation for your CI/CD pipeline.</p><p><a href="https://github.com/hsiddhu2/springboot-helloworld"><strong><em>Sample Springboot Application Github repo</em></strong></a></p><h3>Step 2: Launch the AWS EC2 Instance and Configure</h3><p>This step involves setting up the infrastructure where your application will be deployed.</p><h4><strong>Launch an EC2 Instance</strong>:</h4><ul><li>Log in to your AWS Management Console and navigate to the EC2 dashboard.</li><li>Launch a new instance using Amazon Linux 2 or Ubuntu AMI. Please create a new key pair and download it. Download both .ppk (putty) and .pem as you need to provide this in the GitHub action CICD yaml file to SSH to EC2 to upload the artifacts.</li><li>Assign a security group with appropriate inbound and outbound rules (described below).</li></ul><h4><strong>Update the System and Install JDK</strong>:</h4><ul><li>Connect to your EC2 instance via SSH:</li></ul><pre>ssh -i /path/to/your-key.pem ec2-user@&lt;EC2_PUBLIC_IP&gt;</pre><ul><li>Update the instance packages:</li></ul><pre>sudo yum update -y</pre><ul><li>Install Java JDK 21 (or the version of your choice):</li></ul><pre>sudo yum install java-21-amazon-corretto -y</pre><h4><strong>Configure Security Group</strong>:</h4><p>Open the EC2 Management Console and select the instance’s security group.</p><ul><li>Add inbound rules: <br>- <strong>TCP Port 8080</strong>: For application access <br>- <strong>SSH Port 22</strong>: For deployment access.</li><li>Outbound rules should allow all traffic to ensure connectivity.</li></ul><h4><strong>Set Up a Systemd Service for the Spring Boot Application</strong>:</h4><p>Create a systemd service file for your application. Run this command to create a service file —</p><pre>sudo nano /etc/systemd/system/springboot.service</pre><p>And, add the following content to this file-</p><pre>[Unit]<br>Description=Spring Boot Application<br>After=network.target<br><br>[Service]<br>User=ec2-user<br>ExecStart=/usr/bin/java -jar /home/ec2-user/deployment/target/springboot-app.war<br>Restart=always<br><br>[Install]<br>WantedBy=multi-user.target</pre><p>This systemd configuration file defines a service to manage and run your Spring Boot application on the EC2 instance.</p><p><strong>[Unit]Section:</strong></p><p>Provides a short description of the service, making it easier to identify.</p><ul><li>The After Specifies dependencies that must be initialized before starting the service. network.target ensures the service starts only after network-related services are up and running, which is crucial for web applications relying on networking.</li></ul><p><strong>[Service] Section</strong></p><p>Defines the user under which the service runs.</p><ul><li>ec2-user is the default user for Amazon Linux instances. Running the service under a specific user enhances security.</li><li><strong>ExecStart</strong>: Specifies the command to start the service. Here, the java -jar command runs the Spring Boot .jar file, located at /path/to/your-app.jar. Replace this with the actual path to your application&#39;s .jar or .warfile.</li><li><strong>Restart</strong>: Ensures the service automatically restarts if it crashes or stops unexpectedly. Options include always, on-failure, etc. always provides the highest level of resilience.</li></ul><p><strong>[install] Section</strong></p><p><strong>WantedBy</strong>: Defines the target at which the service should be started.</p><ul><li>multi-user.target is a common target for services that run after the system boots into a non-graphical (multi-user) mode, such as servers.</li></ul><h4>Create Deployment Directory</h4><p>On the EC2 instance, create a directory which will be the location where the deploy job will upload the artifacts like .jar or .war file.</p><p>Location: /home/ec2-user/deployment</p><h3>Step 3: Configure GitHub Actions Workflow</h3><p>Add a .github/workflows directory in your repository and define a YAML file for the CI/CD pipeline. Here is an example workflow configuration:</p><pre>name: CI/CD for Spring Boot App<br><br>on:<br>  push:<br>    branches:<br>      - master or main <br><br>jobs:<br>  build:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v3<br><br>      - name: Set up JDK<br>        uses: actions/setup-java@v3<br>        with:<br>          distribution: &#39;adopt&#39;<br>          java-version: &#39;21&#39;<br><br>      - name: Build with Maven<br>        run: mvn clean package<br><br>      - name: Archive production artifacts<br>        uses: actions/upload-artifact@v3<br>        with:<br>          name: packaged-application<br>          path: target/*.war<br><br>  deploy:<br>    needs: build<br>    runs-on: ubuntu-latest<br><br>    steps:<br>      - name: Download artifacts<br>        uses: actions/download-artifact@v3<br>        with:<br>          name: packaged-application<br>          path: target/<br><br>      - name: Setup key<br>        id: setup-key<br>        env:<br>          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}<br>        run: |<br>          echo &quot;$DEPLOY_KEY&quot; &gt; $HOME/key.pem<br>          chmod 400 $HOME/key.pem<br><br>      - name: Copy WAR to EC2<br>        uses: appleboy/scp-action@master<br>        with:<br>          host: ${{ secrets.DEPLOY_HOST }}<br>          username: ec2-user<br>          key: ${{ secrets.DEPLOY_KEY }}<br>          source: &quot;target/*.war&quot;<br>          target: &quot;/home/ec2-user/deployment/&quot;<br><br>      - name: Connect SSH and Restart Services<br>        run: |<br>          ssh -o StrictHostKeyChecking=no -i $HOME/key.pem ec2-user@${{ secrets.DEPLOY_HOST }} &#39;<br>            sudo systemctl daemon-reload<br>            sudo systemctl restart spring-boot-app<br>          &#39;</pre><h3>Step 4: Connect GitHub Actions to EC2</h3><p>To connect CICD to the EC2 instance, you must provide the SSH private key and the EC2 instance public DNS in the above .YML file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WiumeRYWIoNdJT0CXWAmgw.png" /></figure><ul><li>Generate an SSH key pair and configure your EC2 instance to use the public key for SSH access.</li><li>Add the private key as a GitHub secret (EC2_KEY).</li><li>Replace &lt;EC2_PUBLIC_IP&gt; in the workflow file with your instance&#39;s public IP address or DNS. You can add this parameter as GitHub Secret as well.</li></ul><h3>Testing the Workflow &amp; Application on Web</h3><p>Let’s test it and see what happens …</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SN0Tpyo1uQGQBlkZKMMSdg.png" /><figcaption>GitHub Actions — CICD Workflow Build logs</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q1RWIAFxp8wNduHFc3NlNA.png" /><figcaption>GitHub Actions — CICD Workflow Deploy logs</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*Tf3WdCE1I23nd8az4A6QnA.png" /><figcaption>Successfull acces to the application via HTTP:8080 over internet</figcaption></figure><h3>Final Architecture Diagram</h3><p>Here’s how it all comes together:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xfgc-zxCi90i1z06BGUCTg.jpeg" /></figure><h3>Benefits of This Setup 🌟</h3><ol><li><strong>Automation</strong>: No manual intervention to build or deploy code.</li><li><strong>Scalability</strong>: Easily scalable using AWS Auto Scaling Groups.</li><li><strong>Security</strong>: Secure deployments using GitHub Secrets and AWS Security Groups.</li><li><strong>Cost-Effective</strong>: Minimal infrastructure setup with maximum output.</li></ol><h3>Conclusion</h3><p>This CI/CD pipeline is a robust solution for deploying Spring Boot applications seamlessly to AWS.</p><p>With GitHub Actions handling the CI/CD workflows and AWS providing a scalable infrastructure, you can focus on writing great code while the pipeline does the heavy lifting. 💪</p><p>Let me know your thoughts in the comments below, and don’t forget to 👏 if you found this helpful! Thank you!</p><h3>Thank you for being a part of the community</h3><p><em>Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://x.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a> | <a href="https://open.spotify.com/show/7qxylRWKhvZwMz2WuEoua0"><strong>Podcast</strong></a></li><li><a href="https://cofeed.app/"><strong>Check out CoFeed, the smart way to stay up-to-date with the latest in tech</strong></a> <strong>🧪</strong></li><li><a href="https://differ.blog/"><strong>Start your own free AI-powered blog on Differ</strong></a> 🚀</li><li><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Join our content creators community on Discord</strong></a> 🧑🏻‍💻</li><li>For more content, visit <a href="https://plainenglish.io/"><strong>plainenglish.io</strong></a> + <a href="https://stackademic.com/"><strong>stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2915eb737db7" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/deploy-springboot-java-app-on-ec2-with-cicd-automation-%EF%B8%8F-2915eb737db7">Deploy SpringBoot Java App on EC2 with CICD Automation ✌️</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build A Serverless Email Marketing Applications Using AWS]]></title>
            <link>https://aws.plainenglish.io/build-a-serverless-email-marketing-applications-using-aws-252143efb5cf?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/252143efb5cf</guid>
            <category><![CDATA[email-marketing]]></category>
            <category><![CDATA[aws-beginner]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws-project]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Sun, 04 Aug 2024 23:53:19 GMT</pubDate>
            <atom:updated>2025-02-03T19:06:41.214Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fpI1GovMG6AYGkO2Oloq6A.png" /></figure><h4>AWS Project For Beginners (Level: Medium)</h4><h4>Learn S3, Lamba, SES, EventBridge and IAM</h4><h3>Introduction</h3><p>Hi Everyone, Welcome to <a href="https://www.youtube.com/@CloudWithHarry">the CloudWithHarry</a> Blog. Today, I have an exciting new project for you!</p><p>In this tutorial, we will build our very own email marketing campaign system. This system will allow you to send regular emails, such as a weekly newsletter or any other type of email, to your subscribers. While there are plenty of email marketing systems available, building one in AWS is a fantastic way to gain hands-on experience with multiple AWS services. That’s the main goal of this project.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6AmG7bDoIoA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6AmG7bDoIoA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F6AmG7bDoIoA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href">https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href</a></iframe><p><strong>AWS Services We Will Use</strong></p><p>We will be leveraging the following AWS services:</p><ol><li><strong>Simple Email Service (SES)</strong>: To send emails.</li><li><strong>Lambda</strong>: To handle email sending logic and personalize the emails.</li><li><strong>S3</strong>: To store the email templates and the list of contacts.</li><li><strong>EventBridge</strong>: To set up a schedule and trigger the Lambda function.</li><li><strong>Identity and Access Management (IAM)</strong>: To ensure our services have the necessary permissions to interact with each other.</li></ol><h4>Cost</h4><p>The good news is that most of what we will do here is eligible for the AWS free tier. However, there might be minimal costs associated with some services, such as EventBridge.</p><p><strong>Here’s a breakdown:</strong></p><p><strong>EventBridge</strong>: Costs $1 per million schedule invocations per month. Since we will only invoke it once or twice for testing, the cost will be practically zero.<br><strong>Overall</strong>: If you are inside the free tier, most of our activities will be free. Outside the free tier, the charges will be very minimal, around $0.03 to $0.08.</p><p>I’ll also show you how to delete everything at the end of the tutorial to ensure you don’t incur any unnecessary costs.</p><h4>Prerequisites</h4><p>To follow along with this project, you’ll need a few things:</p><ol><li><strong>An email address</strong>: This will be used to send and receive emails for verification.</li><li><strong>Basic knowledge of AWS</strong>: Understanding how to navigate the AWS Management Console and the basics of AWS services will be helpful.</li><li><strong>AWS Free Tier Account</strong>: Ensure you have an AWS account with free tier access.</li><li><strong>Text Editor</strong>: I recommend using VS Code for writing and modifying code due to its syntax highlighting and ease of use.</li></ol><h3>Getting Started</h3><blockquote>Please download the project code and files from here — <a href="https://github.com/hsiddhu2/Serverless-Email-Marketing-Application-.git">[<strong><em>GITHUB</em></strong>]</a></blockquote><p>If you like to follow along with a video, follow along with me. You can watch this YouTube video.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6AmG7bDoIoA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6AmG7bDoIoA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F6AmG7bDoIoA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href">https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href</a></iframe><h4>Creating the S3 Bucket</h4><p>First, we need a place to store our email template and contact list. We’ll use Simple Storage Service (S3) for this.</p><ol><li><strong>Navigate to S3 in the AWS Console</strong>: Search for S3 in the AWS Management Console and click on it.</li><li><strong>Create a Bucket</strong>: Click on “Create Bucket”. Name your bucket (e.g., `cwh-email-marketing-bucket`) and leave all other settings as default.</li><li><strong>Upload Files</strong>: Upload the `email_template.html` and `contacts.csv` files to your new bucket.</li></ol><h4>Setting Up SES</h4><p>Next, we’ll set up Amazon Simple Email Service (SES) to send emails.</p><ol><li><strong>Navigate to SES in the AWS Console</strong>: Open SES from the AWS Management Console.</li><li><strong>Verify Email Addresses</strong>: Verify the email addresses you will use for sending and receiving emails.<br> — Click on “Create Identity” and choose “Email Address”.<br> — Enter your email address and click “Create Identity”.<br> — Check your email for a verification link and click it to verify.</li><li><strong>Domain Verification</strong>: If you want to send emails from a domain you own, you’ll need to verify the domain. This step involves adding DNS records provided by AWS to your domain’s DNS settings.</li></ol><blockquote><em>💡 </em><strong><em>Note:</em></strong><em> Since this article focuses on building an email marketing app with AWS, we’ll use SES. However, while SES is a powerful tool, its setup process can be quite complex, especially if you’re not already familiar with AWS’s documentation and requirements. If you’d rather skip SES’s convoluted setup and documentation, something like </em><a href="https://notify.cx/?utm_source=medium&amp;utm_medium=blog_post&amp;utm_campaign=q1_2025_embed"><em>Notify</em></a><em> could be a good alternative — it works out of the box without the AWS headaches.</em></blockquote><p>Let’s move on.</p><h4>Creating the Lambda Function</h4><p>Now, let’s create a Lambda function to handle the email-sending logic.</p><ol><li><strong>Navigate to Lambda in the AWS Console</strong>: Open Lambda from the AWS Management Console.</li><li><strong>Create a Function</strong>: Click on “Create Function” and choose “Author from Scratch”.<br> — Name your function (e.g., `EmailMarketingFunction`).<br> — Choose Python as the runtime.</li><li>Write the Code: Use the following code for your Lambda function:</li></ol><pre>import boto3<br>import csv<br><br># Initialize the boto3 client<br>s3_client = boto3.client(&#39;s3&#39;)<br>ses_client = boto3.client(&#39;ses&#39;)<br><br>def lambda_handler(event, context):<br>    # Specify the S3 bucket name<br>    bucket_name = &#39;cwh-email-marketing-bucket&#39;  # Replace with your bucket name<br><br>    try:<br>        # Retrieve the CSV file from S3<br>        csv_file = s3_client.get_object(Bucket=bucket_name, Key=&#39;contacts.csv&#39;)<br>        lines = csv_file[&#39;Body&#39;].read().decode(&#39;utf-8&#39;).splitlines()<br>        <br>        # Retrieve the HTML email template from S3<br>        email_template = s3_client.get_object(Bucket=bucket_name, Key=&#39;email_template.html&#39;)<br>        email_html = email_template[&#39;Body&#39;].read().decode(&#39;utf-8&#39;)<br>        <br>        # Parse the CSV file<br>        contacts = csv.DictReader(lines)<br>        <br>        for contact in contacts:<br>            # Replace placeholders in the email template with contact information<br>            personalized_email = email_html.replace(&#39;{{FirstName}}&#39;, contact[&#39;FirstName&#39;])<br>            <br>            # Send the email using SES<br>            response = ses_client.send_email(<br>                Source=&#39;you@yourdomainname.com&#39;,  # Replace with your verified &quot;From&quot; address<br>                Destination={&#39;ToAddresses&#39;: [contact[&#39;Email&#39;]]},<br>                Message={<br>                    &#39;Subject&#39;: {&#39;Data&#39;: &#39;Your Weekly Mail!&#39;, &#39;Charset&#39;: &#39;UTF-8&#39;},<br>                    &#39;Body&#39;: {&#39;Html&#39;: {&#39;Data&#39;: personalized_email, &#39;Charset&#39;: &#39;UTF-8&#39;}}<br>                }<br>            )<br>            print(f&quot;Email sent to {contact[&#39;Email&#39;]}: Response {response}&quot;)<br>    except Exception as e:<br>        print(f&quot;An error occurred: {e}&quot;)</pre><h4>IAM Permissions</h4><p>Ensure your Lambda function has the necessary permissions to access S3 and SES.</p><ol><li><strong>Navigate to IAM in the AWS Console</strong>: Open IAM from the AWS Management Console.</li><li><strong>Create a Policy</strong>: Create a policy with the following permissions:</li></ol><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;s3:GetObject&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::cwh-email-marketing-bucket/*&quot;<br>        },<br>        {<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Action&quot;: [<br>                &quot;ses:SendEmail&quot;,<br>                &quot;ses:SendRawEmail&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;<br>        }<br>    ]<br>}</pre><p>3. <strong>Attach the Policy</strong>: Attach this policy to the role used by your Lambda function. You can locate it from the AWS Lambda function configuration.</p><h4>Setting Up EventBridge</h4><p>EventBridge will be used to trigger the Lambda function on a schedule.</p><ol><li><strong>Navigate to EventBridge in the AWS Console</strong>: Open EventBridge from the AWS Management Console.</li><li><strong>Create an EventBridge Schedule</strong>: Click on “Create EventBrider Schedule.<br> — Name your schedule(e.g., WeeklyEmailMarketingSchedule`).<br> — Set the rule to trigger on a schedule (e.g., every week). For testing keep it only for run once and set up the time. <br> — Add the Lambda function as the target</li></ol><h3>Testing</h3><p>Check email generating at the scheduled time. You can review the AWS Lamba CloudWatch logs also.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*C1WvVBmhjaFbVCdGp0OGSg.png" /></figure><h3>Conclusion</h3><p>Congratulations! You’ve now set up an email marketing campaign system using AWS. This project has provided you with hands-on experience with several AWS services, including SES, Lambda, S3, EventBridge, and IAM.</p><h4>Follow Along</h4><p>If you want to follow along with this project, I recommend using a tool called Eraser.io. This tool will help you diagram the architecture and keep track of the checklist as we progress. Watch this video to see Erasio.io in action.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6AmG7bDoIoA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6AmG7bDoIoA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F6AmG7bDoIoA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href">https://medium.com/media/5b35717f5f7ad85d910b8d73b16e997b/href</a></iframe><h3>Final Steps</h3><p>Don’t forget to delete the resources after testing to avoid any unwanted costs. Follow these steps to clean up:</p><ol><li><strong>Delete Lambda Function</strong>: Go to the Lambda console, select your function, and click “Delete”.</li><li><strong>Remove EventBridge Schedule</strong>: Navigate to EventBridge, find your rule, and delete it.</li><li><strong>Delete S3 Bucket</strong>: Go to the S3 console, select your bucket, and click “Delete”. Make sure to delete all objects in the bucket first.</li><li><strong>Remove SES Identities</strong>: In the SES console, go to “Email Addresses” and delete any identities you created for testing.</li><li><strong>Delete IAM Policies and Roles</strong>: Navigate to IAM, find the policies and roles used for this project, and delete them.</li></ol><p>Happy building! If you found this tutorial helpful, make sure to share it with others and subscribe for more tutorials like this. Thank you!</p><h3>In Plain English 🚀</h3><p><em>Thank you for being a part of the </em><a href="https://plainenglish.io/"><strong><em>In Plain English</em></strong></a><em> community! Before you go:</em></p><ul><li>Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></li><li>Follow us: <a href="https://twitter.com/inPlainEngHQ"><strong>X</strong></a> | <a href="https://www.linkedin.com/company/inplainenglish/"><strong>LinkedIn</strong></a> | <a href="https://www.youtube.com/channel/UCtipWUghju290NWcn8jhyAw"><strong>YouTube</strong></a> | <a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a> | <a href="https://newsletter.plainenglish.io/"><strong>Newsletter</strong></a></li><li>Visit our other platforms: <a href="https://cofeed.app/"><strong>CoFeed</strong></a> | <a href="https://differ.blog/"><strong>Differ</strong></a></li><li>More content at <a href="https://plainenglish.io/"><strong>PlainEnglish.io</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=252143efb5cf" width="1" height="1" alt=""><hr><p><a href="https://aws.plainenglish.io/build-a-serverless-email-marketing-applications-using-aws-252143efb5cf">Build A Serverless Email Marketing Applications Using AWS</a> was originally published in <a href="https://aws.plainenglish.io/">AWS in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Break into Cloud Engineering WITHOUT a Degree!]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://aws.plainenglish.io/break-into-cloud-engineering-without-a-degree-3e218fe24d01?source=rss-96fcaaa6e15------2"><img src="https://cdn-images-1.medium.com/max/1200/1*y2kZis-sxcfoR-Z6qAa8fw.png" width="1200"></a></p><p class="medium-feed-snippet">Is it Possible? HELL YES!</p><p class="medium-feed-link"><a href="https://aws.plainenglish.io/break-into-cloud-engineering-without-a-degree-3e218fe24d01?source=rss-96fcaaa6e15------2">Continue reading on AWS in Plain English »</a></p></div>]]></description>
            <link>https://aws.plainenglish.io/break-into-cloud-engineering-without-a-degree-3e218fe24d01?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e218fe24d01</guid>
            <category><![CDATA[career-development]]></category>
            <category><![CDATA[awscommunitybuilder]]></category>
            <category><![CDATA[career-change]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws-certification]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Fri, 24 May 2024 07:56:50 GMT</pubDate>
            <atom:updated>2025-01-24T14:55:10.546Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Crafting a Secure AWS Environment: A Guide to IAM Essentials]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://aws.plainenglish.io/crafting-a-secure-aws-environment-a-guide-to-iam-essentials-2e86f27b202b?source=rss-96fcaaa6e15------2"><img src="https://cdn-images-1.medium.com/max/1200/1*TJc3EAmNBL7VH1TAB5OhFA.png" width="1200"></a></p><p class="medium-feed-snippet">A Deep Dive Into AWS Identify &amp; Access Management</p><p class="medium-feed-link"><a href="https://aws.plainenglish.io/crafting-a-secure-aws-environment-a-guide-to-iam-essentials-2e86f27b202b?source=rss-96fcaaa6e15------2">Continue reading on AWS in Plain English »</a></p></div>]]></description>
            <link>https://aws.plainenglish.io/crafting-a-secure-aws-environment-a-guide-to-iam-essentials-2e86f27b202b?source=rss-96fcaaa6e15------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e86f27b202b</guid>
            <category><![CDATA[iam-roles]]></category>
            <category><![CDATA[access-management]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[identity-management]]></category>
            <category><![CDATA[iam-policy]]></category>
            <dc:creator><![CDATA[Harpreet Siddhu]]></dc:creator>
            <pubDate>Fri, 20 Oct 2023 21:42:50 GMT</pubDate>
            <atom:updated>2024-07-03T18:06:22.618Z</atom:updated>
        </item>
    </channel>
</rss>