<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amrut Pagidipally on Medium]]></title>
        <description><![CDATA[Stories by Amrut Pagidipally on Medium]]></description>
        <link>https://medium.com/@amruteng?source=rss-ef35bc517478------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*VOxwrKF5Eb2pkOPObgLPUg@2x.jpeg</url>
            <title>Stories by Amrut Pagidipally on Medium</title>
            <link>https://medium.com/@amruteng?source=rss-ef35bc517478------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 03:46:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@amruteng/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I wrote a linter for a bug that can’t be unit tested]]></title>
            <link>https://amruteng.medium.com/i-wrote-a-linter-for-a-bug-that-cant-be-unit-tested-e296245eb39d?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/e296245eb39d</guid>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Fri, 04 Sep 2026 06:52:50 GMT</pubDate>
            <atom:updated>2026-09-04T06:52:50.411Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><h3>AWS Lambda functions can now pause for up to a year. When one wakes up, your handler runs again from line one. That breaks a lot of ordinary code.</h3><p>AWS durable functions let you write a Lambda that waits. Not polls. Waits, for up to 366 days.</p><p>Say you need a payment, then a human approval, then a shipment. The approval might take four minutes or four months. Normally that’s a state machine, or a queue plus a database table plus three Lambdas, and most of your code ends up being plumbing.</p><p>With durable functions it’s one function that stops in the middle.</p><p>Which raises the obvious question, because nothing keeps a process alive for a year.</p><h3>How it actually works</h3><p>It doesn’t stay alive. When the execution resumes, Lambda runs your handler again from the top.</p><p>Anything you wrapped in a step() doesn&#39;t re-execute. Those got checkpointed the first time, so the SDK hands back the recorded result and your function fast-forwards to where it stopped. From inside it looks like resuming. From outside, your handler ran twice.</p><p>Or six times, if it suspended six times.</p><p>It’s a good design. It’s also got a consequence I didn’t fully absorb at first: everything outside a step runs again, every time.</p><p>Hence this line in the docs:</p><blockquote>Any code that is not inside a durable operation must be a pure function of the handler inputs and the results of completed operations.</blockquote><p>No clocks, no randomness, no I/O, no shared state. Unless it’s in a step.</p><p>I went looking for the thing that checks this. A rule that strict usually comes with a compiler error, or a warning, or something.</p><p>There isn’t one. You just have to not do it.</p><h3>What it looks like when you get it wrong</h3><p>Here’s a saga. Do some work, track what succeeded, roll it back if a later step fails.</p><pre>const handler = async (event, context) =&gt; {<br>  const completedSteps = [];<br>  await context.step(&#39;reserve-inventory&#39;, async () =&gt; {<br>    await reserveInventory(event.orderId);<br>    completedSteps.push(&#39;inventory&#39;);<br>  });<br>  await context.step(&#39;charge-payment&#39;, async () =&gt; {<br>    await chargeCard(event.orderId);<br>    completedSteps.push(&#39;payment&#39;);<br>  });<br>  if (somethingWentWrong) {<br>    for (const step of completedSteps.reverse()) {<br>      await compensate(step);<br>    }<br>  }<br>};</pre><p>Read it top to bottom and it’s fine. I’d have written this.</p><p>Now resume it.</p><p>The handler restarts. completedSteps is a new empty array. Both steps are checkpointed already, so their bodies don&#39;t run, so neither push happens.</p><p>The compensation loop iterates over nothing and rolls back nothing. Then it returns like it worked.</p><p>Inventory stays reserved. Nothing throws. Nothing logs an error. On a dashboard you see a successful execution, which is worse than seeing a failed one.</p><p>I didn’t make that example up. I found it in a public repo, in the file presented as the reference saga implementation.</p><h3>Your tests won’t catch it</h3><p>This is what made me build something.</p><p>Your test calls the handler. It runs top to bottom. Everything passes.</p><p>It passes because the first run is always correct. That’s not luck, it’s the whole shape of the bug. You’d need a test that suspends an execution and resumes it, and I’ve never seen anyone write one.</p><p>So it ships green. Months later something waits long enough to actually suspend, and now you have a production bug on a workflow holding real money. AWS’s docs list double-charging as a possible outcome.</p><p>A linter should catch this. There wasn’t one, so I wrote one.</p><h3>replayguard</h3><p>Two halves, and the split matters more than I expected.</p><p>The <strong>static half</strong> parses handlers and looks for what’s visible in source: a clock outside a step, I/O outside a step, a step body writing to something it doesn’t own, a branch on a nondeterministic value, a step name built from something unstable.</p><p>That last one caught me out. Checkpoints match by name and order. Name a step with a template literal containing Date.now() and the name is different on replay. The SDK can&#39;t match it, so the step runs again, side effects included. I found that exact pattern in an AWS sample building a task id.</p><p>This needs real scope resolution, not pattern matching. To know if completedSteps.push() is a bug you need to know whether that variable was declared inside the step body or outside it. Grep can&#39;t tell you. So it&#39;s AST analysis, four languages, all lowered to one shared representation so the rules get written once.</p><p>The <strong>dynamic half</strong> is the one I’d pick if you only take one. It runs your handler twice, once normally and once with the clock moved thirteen hours and the random seed changed, then diffs the operations each run asked for.</p><pre>from replayguard.dynamic import assert_deterministic<br>def test_handler_is_deterministic():<br>    assert_deterministic(handler, {&quot;orderId&quot;: &quot;A1&quot;})</pre><p>It earns its place because static analysis can’t see some causes. If the nondeterminism is inside a library you didn’t write, or the value got tainted six calls upstream, or you’re iterating a set, no rule catches it. The harness doesn’t need the cause. It just notices the execution changed shape.</p><p>A handler whose step order comes from random.sample diverges immediately. I can&#39;t write a static rule for that.</p><h3>Then I ran it on other people’s code</h3><p>This is the part worth passing on, and it isn’t really about Lambda.</p><p>I ran it over 1,547 files written by about a dozen other people. AWS samples, all three official SDKs, community repos.</p><p>It found bugs. It also found thirteen bugs in my own tool.</p><p>None of those were catchable by my own fixtures. I wrote the fixtures and I wrote the analyser, so the same blind spots went into both. My fixtures tested the code I already knew how to handle.</p><p>Other people write differently. They pass step bodies as keyword arguments. They write n++ instead of n += 1, which my mutation detection missed, and that one was hiding a real bug in a file that was also throwing four false positives. They use decorators that defer execution, so the body runs somewhere other than where it&#39;s written.</p><p>If you build a dev tool and only test it on code you wrote, you’ve tested your own assumptions.</p><p>Same lesson, different route: early on I wrote in three separate documents that Rust had no durable execution SDK. I’d searched GitHub. A crates.io search would have turned one up in seconds, mature, 245 commits, claiming parity with the official Python SDK. One search surface isn’t a prior art check, and I’d already written the claim down three times.</p><h3>The unflattering bit</h3><p>Three of the six rules have never found a real bug in anyone else’s code. Not once, across all 1,547 files.</p><p>They’re not broken. I checked that rather than assuming. Each one fires on its own violation in all four languages, and that runs as a test on every commit, so “no findings” can’t quietly mean “no detector”.</p><p>They’re quiet because published example code doesn’t have those mistakes in it. I went through all 837 handlers by hand. 85 branch conditions in the replayed region, every one testing stable data. 38 computed step names, none built from a clock.</p><p>Which tracks, when you think about who writes example code. Nobody demonstrates a pattern by building a step name out of Date.now() at 6pm on a Friday. Those are deadline mistakes and they live in private repos.</p><p>So those three work and stay unproven, and that’s in the README above the install instructions. A checker that overstates what it’s demonstrated isn’t worth installing.</p><h3>If you want to try it</h3><pre>pip install &#39;replayguard[all]&#39;<br>replayguard check src/</pre><p>It emits SARIF, so in CI findings show up as annotations on the pull request that introduced them. For a bug nobody notices for six months that’s about the only delivery mechanism that works.</p><p>Two limits before you trust a clean run. It doesn’t follow calls across files yet, so a handler that hands its I/O to another module comes back green. And the dynamic side can’t prove determinism, only fail to disprove it under one perturbation. The report says so.</p><p>If you’re running durable functions on something real, I’d like to know what happens when you point this at it. A bug it caught, or noise it raised that it shouldn’t have. Either is useful. One run against a codebase written under deadline beats another thousand files of samples, and that’s the evidence I can’t generate myself.</p><p><em>Source and full validation record: </em><a href="https://github.com/amrutp24/replayguard"><em>github.com/amrutp24/replayguard</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e296245eb39d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I put an embedding model in a Lambda container. A year later, here’s what I got wrong.]]></title>
            <link>https://amruteng.medium.com/i-put-an-embedding-model-in-a-lambda-container-a-year-later-heres-what-i-got-wrong-bcc1dc6995f4?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/bcc1dc6995f4</guid>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 02:03:53 GMT</pubDate>
            <atom:updated>2026-08-17T02:23:23.889Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p>In May 2025 I built <a href="https://github.com/amrutp24/llm-lambda">llm-lambda</a>: a bge-small-en sentence-transformer packaged as a container image and deployed to AWS Lambda with Terraform. No inference API, no endpoint to keep warm, no GPU. Send it text, get back a vector, pay only for what you use.</p><p>It worked. I wrote it up, pushed it, moved on.</p><p>Going back a year later, three things turned out to be true. The image no longer builds at all. The two factors most responsible for its cold-start behaviour were both in plain sight on the day I shipped: one a hard limit in the docs, the other 2.6 GB of GPU drivers I was shipping into a function that has never seen a GPU, which had quietly pushed the image to within 1.6 GB of a service ceiling I didn’t know existed. And the feature everyone would tell you to reach for was never available to this design in the first place.</p><p>None of that is AWS’s fault. It’s what happens when you ship something that works and stop asking why.</p><h3>What the thing actually is</h3><p>The whole function is fifteen lines:</p><pre>from sentence_transformers import SentenceTransformer<br>import json<br>model = SentenceTransformer(&quot;/var/task/model/bge-small-en&quot;)<br>def lambda_handler(event, context):<br>    text = event.get(&quot;text&quot;, &quot;&quot;)<br>    if not text:<br>        return {&quot;statusCode&quot;: 400, &quot;body&quot;: &quot;Text missing&quot;}<br>    embedding = model.encode(text).tolist()<br>    return {<br>        &quot;statusCode&quot;: 200,<br>        &quot;body&quot;: json.dumps({&quot;embedding&quot;: embedding[:5]})<br>    }</pre><p>The model is baked into the image rather than pulled at runtime, which is the right call. Downloading weights from Hugging Face on every cold start would be worse in every way. The Terraform gives it 2 GB of memory and a 60-second timeout:</p><pre>resource &quot;aws_lambda_function&quot; &quot;llm_lambda&quot; {<br>  package_type  = &quot;Image&quot;<br>  image_uri     = var.image_uri<br>  timeout       = 60<br>  memory_size   = 2048<br>}</pre><p>Note line 4 of the handler. SentenceTransformer(...) runs at module scope, outside the handler, which is the standard advice: do expensive setup once per execution environment so warm invocations skip it. That advice is correct, and that line is where most of this post ends up. Hold onto it.</p><h3>It doesn’t build any more</h3><p>Before any of the analysis, the first thing I did was rebuild the image. It failed.</p><p>Not because I changed anything. The repo has three commits, all from May 2025, and I hadn’t touched it. It failed because requirements.txt pins five packages and leaves every transitive dependency to float:</p><pre>sentence-transformers==2.4.0<br>transformers==4.35.2<br>huggingface-hub==0.17.3<br>torch==2.0.1<br>numpy==1.26.4</pre><p>sentence-transformers depends on scikit-learn without a version bound. Today pip resolves that to scikit-learn 1.9.0, which no longer publishes a cp311 wheel. Python 3.11 has aged out of its support matrix. So pip falls back to the source tarball, which needs numpy 2.x as a <em>build</em> dependency, which also has no cp311 wheel, which means building numpy from source too, inside a Lambda base image that ships no C compiler:</p><pre>Collecting scikit-learn (from sentence-transformers==2.4.0)<br>  Downloading scikit_learn-1.9.0.tar.gz (7.8 MB)<br>  Installing build dependencies: finished with status &#39;error&#39;<br>      Collecting numpy&lt;2.5.0,&gt;=2<br>        Downloading numpy-2.4.6.tar.gz (20.7 MB)<br>        ../meson.build:1:0: ERROR: Unknown compiler(s): [[&#39;cc&#39;], [&#39;gcc&#39;], [&#39;clang&#39;], ...]</pre><p>I pinned scikit-learn to get past it. The next build got further and then died the same death on Pillow: another unpinned transitive dependency, another version with no cp311 wheel, another source build reaching for a compiler that isn’t there:</p><pre>Building wheel for Pillow (pyproject.toml): finished with status &#39;error&#39;<br>    error: [Errno 2] No such file or directory: &#39;gcc&#39;<br>ERROR: Failed building wheel for Pillow</pre><p>That’s the part worth sitting with. It isn’t one unlucky package. It’s every package I didn’t name, drifting independently, and I can’t tell you how many more are queued up behind Pillow because I stopped digging. Repairing this properly means resolving the tree as it stood in May 2025, and that information doesn’t exist anywhere. I never wrote it down.</p><p>Five pins felt like enough at the time. They pinned the packages I had heard of. The dependencies that broke the build are ones I never typed, pulled in by packages I did, and they broke not because anyone made a mistake but because a year passed.</p><p>If you take one operational lesson from this post, it’s that a lockfile is not pedantry for a container you intend to still build later. pip freeze the whole resolved tree, or use a tool that produces a real lock. The five-line requirements file is how a working image becomes an unbuildable one with no commits in between.</p><h3>The 10-second wall I didn’t know existed</h3><p>Lambda’s Init phase — extensions, runtime bootstrap, and your module-level code is <a href="https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html">limited to 10 seconds</a>. Not 60. Not whatever you set as your timeout. Ten. From the docs:</p><blockquote>The Init phase is limited to 10 seconds. If all three tasks do not complete within 10 seconds, Lambda retries the Init phase at the time of the first function invocation with the configured function timeout.</blockquote><p>So if importing torch, importing transformers, and deserializing a model takes longer than ten seconds, Lambda doesn’t fail. It throws the init away and does it <em>again</em>, this time inside your invocation, billed against your 60-second timeout. Lambda does tell you, in a log line called INIT_REPORT. The catch is that you have almost certainly never seen it, for reasons I&#39;ll come back to.</p><p>I wrote this in my own README a year ago, under Troubleshooting:</p><blockquote>❌ “Task timed out after 30.00 seconds”: Increase timeout and memory_size in Terraform.</blockquote><p>That’s me documenting the symptom and prescribing a workaround without understanding the mechanism. Raising the timeout gives the retried init more room to finish; it doesn’t stop the double initialization. Raising memory genuinely does help, because Lambda scales CPU with memory and this init is CPU-bound. But I arrived at 2048 MB by trying numbers until the errors stopped, and I never wrote down which change actually moved it</p><p>The fix for a slow module-level model load is not a bigger timeout. It’s getting init under ten seconds.</p><h3>What it actually costs</h3><p>I deployed the rebuilt image to Lambda to find out. 2048 MB, 60-second timeout, the same config as 2025. This is the entire cold start, verbatim:</p><pre>INIT_REPORT Init Duration: 9996.19 ms   Phase: init   Status: timeout<br>START RequestId: d5691e39-9024-414d-bfca-c0285fca49c7 Version: $LATEST<br>END RequestId: d5691e39-9024-414d-bfca-c0285fca49c7<br>REPORT  Duration: 43023.84 ms   Billed Duration: 43024 ms   Memory Size: 2048 MB   Max Memory Used: 505 MB</pre><p>There it is, in AWS’s own words. Status: timeout on the init phase at 9996 ms, four milliseconds under the cap. Lambda threw that initialization away, started again inside the invocation, and charged me for all of it: <strong>43 seconds billed</strong> on a function whose actual work is 40 milliseconds.</p><p>The request succeeded. It returned a correct 384-dimension vector. Nothing errored, nothing alarmed, and if you were watching a dashboard you’d see one slow invocation. The only thing that tells you what happened is that INIT_REPORT line, and Lambda emits it only when init fails, or when SnapStart or provisioned concurrency is enabled. A healthy on-demand function never logs it. So the one line that explains the problem is a line you have never seen in your own logs and won&#39;t go looking for unless you already suspect it.</p><p>Timing the phases individually, this is where the ten seconds go:</p><pre>import torch            8.50s<br>import sentence_tf      3.84s<br>load model              5.19s<br>-- total init          17.53s<br>first encode            0.51s<br>second encode           0.04s</pre><p>Note Max Memory Used: 505 MB against 2048 MB configured. The function needs a quarter of the memory it&#39;s paying for. But on Lambda, memory is how you buy CPU. Allocation is proportional, one vCPU arrives at 1,769 MB, so that 2 GB was never about RAM. It was me, a year ago, discovering that a bigger number made the errors go away, and writing &quot;increase timeout and memory_size&quot; in the README without understanding that I&#39;d been buying CPU to outrun an init budget I didn&#39;t know existed.</p><p>At 43 seconds and 2 GB, each cold start costs about $0.0014. That’s not the problem. The problem is 43 seconds of latency on a function advertised as an on-demand embedding endpoint.</p><h3>Moving the load out of module scope</h3><p>So I moved it. Both the model load and the sentence_transformers import go inside the handler behind a module-level cache. The import alone pulls torch, which costs 8.5 seconds on its own:</p><pre>_model = None<br>def _get_model():<br>    global _model<br>    if _model is None:<br>        from sentence_transformers import SentenceTransformer<br>        _model = SentenceTransformer(&quot;/var/task/model/bge-small-en&quot;)<br>    return _model</pre><p>Init went from timing out at 9996 ms to <strong>146–514 ms</strong>, across every run I measured. No more INIT_REPORT, no more discarded initialization, no more paying for the same work twice. That part is unambiguous.</p><p>The cold-start latency is a messier story, and it took a while to see honestly. Five consecutive cold starts of the identical image, forced by updating function config:</p><pre>5,269 ms   16,490 ms   6,829 ms   5,160 ms   7,809 ms</pre><p>Median about 6.8 seconds, spread 5.2 to 16.5. A three-fold spread on an unchanged image and unchanged code.</p><p>And the first two cold starts after deploying were 56.7 s and 49.9 s, worse than the version I was trying to fix. I read that as a regression at the time and was wrong. Lambda downloads and optimizes a container image on first use, and those numbers are dominated by work that happens exactly once. It took five more runs before the steady state showed up.</p><p>Which means the clean before/after I wanted doesn’t exist. Every number I have for the original module-scope version is a first-invocation measurement, taken before I understood that distinction; I never measured it warm-cached. I can tell you init stopped timing out, because that is deterministic and reproduced every time. I can tell you a warm-cached cold start is now roughly 5 to 16 seconds. I can’t hand you “43 seconds became 7” as a measured result, because the two figures aren’t the same experiment.</p><p>The uncomfortable part is what the steady-state number means. Even at its best, this is ~5 seconds before the first request returns, and the actual embedding is 21 ms. Moving the load bought correctness and visibility. You stop paying twice, and the cost finally shows up in Duration where you can see it. But it did not make the model load faster. Roughly five seconds of importing torch and deserializing weights is simply what this design costs on a 2 GB execution environment. If that&#39;s unacceptable, no amount of moving code around fixes it, and the answer is provisioned concurrency, or something that isn&#39;t Lambda, or not self-hosting the model at all.</p><h3>The 2.6 GB of GPU drivers</h3><p>This is the part I’m least proud of, because the evidence scrolled past me in the build output every single time and I never read it.</p><p>torch==2.0.1 on x86_64 Linux is a 620 MB wheel. I could live with that. What I never looked at was everything it drags in behind it, which pip prints, verbatim, on every build:</p><pre>nvidia_cudnn_cu11-8.5.0.96      557.1 MB<br>nvidia_cublas_cu11-11.10.3.66   317.1 MB<br>nvidia_nccl_cu11-2.14.3         177.1 MB<br>nvidia_cusparse_cu11-11.7.4.91  173.2 MB<br>nvidia_cufft_cu11-10.9.0.58     168.4 MB<br>nvidia_cusolver_cu11-11.4.0.1   102.6 MB<br>triton-2.0.0                     63.3 MB<br>nvidia_curand_cu11-10.2.10.91    54.6 MB<br>nvidia_cuda_nvrtc_cu11-11.7.99   21.0 MB<br>nvidia_cuda_cupti_cu11-11.7.101  11.8 MB</pre><p>That’s 1,647 MB of compressed CUDA runtime on the wire, which unpacks to 2,586 MB inside the image, on top of the 620 MB torch wheel itself. My function has no GPU. It will never have a GPU. Lambda does not offer one.</p><p>For scale: the model I was actually trying to deploy, model.safetensors for bge-small-en, is 133.5 MB. I was shipping roughly twelve times the model&#39;s weight in GPU libraries that cannot execute in the environment they were being sent to, and paying to pull and unpack them on every cold start.</p><p>Two ways out, and they compose.</p><p>Install the CPU-only build from PyTorch’s own index, which drops the CUDA packages entirely rather than shipping them unused:</p><pre>--extra-index-url https://download.pytorch.org/whl/cpu<br>torch==2.0.1+cpu</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jePjZGAMIcInXmywA6938A.png" /></figure><p>Changing one line removes 5.77 GB.</p><p>Now put that first number next to the one constraint I should have known: Lambda’s container image limit is <a href="https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html">10 GB uncompressed</a>.</p><p>My image was at 8.4 GB. I was inside 1.6 GB of a hard service ceiling, on a function that turns a sentence into 384 floats, and I had no idea, because I never once ran docker images and looked. Had I picked a slightly larger model, or pinned a torch version with a fatter CUDA bundle, the deploy would simply have stopped working and I&#39;d have had no idea why.</p><p>Two caveats on these figures. They exclude the model, since the weights aren’t committed to the repo, so add 133.5 MB for a real deployment. And this is a <em>repaired</em> build, with scikit-learn, scipy and Pillow pinned to make it compile at all, so it approximates the 2025 image rather than reproducing it exactly. The 5.77 GB delta between the two columns is exact, though, because only the torch line differs between them.</p><h3>Then arm64, for less than you’d think</h3><p>Switch to arm64 too, though for less than the headline suggests. The aarch64 torch wheel is 74 MB against x86’s 620 MB, which sounds enormous, but that gap is mostly compression. I built both and measured the same way:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CqrZ2ccdKeaRAPQxA6JYcw.png" /></figure><p>About 400 MB, call it 18%, essentially all of it torch. I’d have guessed higher from the wheel sizes alone, which is the lesson: compare unpacked bytes, not download sizes.</p><p>I deployed both to Lambda at 2048 MB and took six or seven cold starts each, discarding the first two while the image cache warms:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yrJelMGFEha5mqXGp2ahqA.png" /></figure><p>arm64 comes out roughly 15% faster at the median, which is about what you’d expect with 400 MB less to page in, and it is meaningfully cheaper: $0.0000133334 per GB-second against $0.0000166667. Combined, a median cold start goes from about $0.00023 to $0.00015, a third off.</p><p>I want to be careful with that 15%, though. The two ranges overlap heavily and I have five and six samples. What I’m confident saying is that arm64 is not slower and appears somewhat faster; I would not defend the specific percentage. Worth taking for the price alone, and the image size and latency come along as a bonus.</p><p>One thing that did surprise me: the very first invocation after deploy was 28.3 s on arm64 against 56.7 s on x86. A smaller image is faster for Lambda to pull and optimize, which matters more than you’d think if you deploy often. Every deployment pays it once.</p><p>There’s a catch worth knowing before you try it. torch==2.0.1+cpu does not exist for aarch64, since that index is x86-only, and it doesn&#39;t need to, because there&#39;s no CUDA on ARM to strip, so the stock PyPI aarch64 wheel is already CPU-only. If your requirements are properly pinned, the pin itself is now architecture-specific and needs markers:</p><pre>torch==2.0.1+cpu ; platform_machine == &quot;x86_64&quot;<br>torch==2.0.1     ; platform_machine == &quot;aarch64&quot;</pre><p>So “just switch to arm64” is not a one-line change once you’ve locked your dependencies, which is its own small joke given how this post started. The build script also hardcodes --platform linux/amd64 for no reason I can defend. It was the default and I never questioned it.</p><h3>The standard fix never applied</h3><p>The usual answer to Lambda cold starts is SnapStart: initialize once when you publish a version, snapshot the memory and disk state, restore from the snapshot instead of running init again. For a function whose entire cold-start cost is “load a model into memory,” that is exactly the right shape of solution.</p><p>SnapStart does not support container images.</p><p>That’s the whole answer. It was true in May 2025, it’s true now, and it applies to every function packaged the way this one is. Python SnapStart also requires 3.12 or later, and this image is built on public.ecr.aws/lambda/python:3.11. Two disqualifications deep, neither of which I noticed.</p><p>The remaining lever was provisioned concurrency, which pre-initializes environments and bills you for them whether or not anyone calls the function. It works. It also deletes the reason I chose Lambda in the first place, which was scale-to-zero.</p><h3>The pricing asymmetry that quietly disappeared</h3><p>There used to be a real cost argument against container images that nobody mentioned: the Init phase was billed for container images and custom runtimes, but <em>free</em> for ZIP functions on managed runtimes. Same cold start, different bill, purely because of packaging.</p><p>That ended on 1 August 2025, when AWS <a href="https://aws.amazon.com/blogs/compute/aws-lambda-standardizes-billing-for-init-phase/">standardized INIT billing</a> across all packaging types and runtimes.</p><p>Two months after I shipped this, one of the arguments against the design stopped being an argument, not because containers got cheaper but because everything else got more expensive. I’ll take it.</p><h3>What I’d build today</h3><p>Three things exist now that didn’t in May 2025.</p><p><strong>Bedrock</strong> has made the build/buy math hard to argue with. Titan Text Embeddings V2 is $0.02 per million tokens, handles 8K of context, and emits 256, 512, or 1024 dimensions. Run it against this function: at 2 GB on x86, a 100 ms warm invocation costs about $0.0000035 including the request charge, so roughly $3.50 per million calls, <em>if every one of them is warm</em>. A million documents at 100 tokens each through Bedrock is $2.00, with no image to build, no torch version to pin, and no init phase to lose sleep over.</p><p>Self-hosting still wins on three counts, and they’re the only three that matter: the text never leaves your function, you control exactly which model version runs, and you aren’t subject to Bedrock’s per-region on-demand throughput limits. If none of those apply to you, you are doing infrastructure work you could have bought for two dollars.</p><p><strong>Lambda Managed Instances</strong>, announced at re:Invent 2025, targets this project’s actual problem. Functions run on EC2 instances that Lambda provisions and patches, with no cold starts, multi-concurrent invocations per environment, and EC2 pricing including Savings Plans. Initialization gets up to 15 minutes instead of 10 seconds. The wall I spent this whole post describing simply isn’t there.</p><p>I could not determine whether it accepts container images. The API reference treats CapacityProviderConfig and PackageType as independent parameters and documents no interaction between them, which suggests Image is allowed. But the user guide&#39;s runtime page lists managed runtimes only, starting at Python 3.13, and every example in the getting- started guide creates functions with --package-type Zip. I&#39;m not going to assert either way from documentation that doesn&#39;t say. If you&#39;re considering it for a containerized function, test that before you plan around it.</p><p>Either way it comes with a capacity provider, a VPC, and scaling to your configured minimum rather than to zero, three instances by default for AZ resiliency. It solves the cold start by removing the cold, which is a good trade for steady traffic and a bad one for a function called eleven times a day.</p><p><strong>Lambda MicroVMs</strong> went GA on 22 June 2026: Firecracker-backed, VM-level isolation, up to eight hours of runtime, built from a Dockerfile you upload to S3 and snapshot. Eight hours is not what an embedding endpoint needs. But it’s the first Lambda-family primitive where “my container takes a while to warm up and then I want to use it for a long time” is the intended shape rather than something you fight.</p><h3>The version I’d write now</h3><p>Keep the container, keep the model in the image, change four things.</p><p>Lock the dependency tree. Not five pins, the whole resolved graph, so the image that builds today still builds in 2027. This is the one that would have cost me nothing at the time and is now the difference between a working artifact and a broken one.</p><p>Install CPU-only torch. That single line takes the image from 8.43 GB to 2.66 GB and is by a wide margin the highest-leverage change available.</p><p>Then build for arm64: 400 MB smaller, 20% cheaper per GB-second, and measurably no slower, with a median cold start of 5.78 s against 6.83 s on x86. Useful and compounding, but much smaller than the first change; don’t let the 74 MB wheel figure fool you into expecting more.</p><p>Move the model load and its imports into the handler behind a module-level cache. This is the change I’d argue hardest for and the one whose benefit is most easily oversold: it takes init from a 10-second timeout to ~150 ms and stops you paying for the same work twice, but it does not make loading faster. Steady-state cold starts still landed between 5 and 16 seconds. It buys correctness and visibility, not speed.</p><p>Then measure Init Duration in the REPORT line, and measure it more than once, after the image has been pulled at least twice. The first two cold starts of a fresh image include one-time optimization work and will lie to you about your steady state.</p><p>Drop AmazonEC2ContainerRegistryReadOnly from the execution role. Lambda pulls the image itself before your code runs; the execution role governs what the <em>function</em> does, and this function talks to nothing. Two managed policies where one would do, and the extra one grants read access to every ECR repository in the account.</p><h3>The honest summary</h3><p>The project did what it said, on the day it said it. An embedding model does run in a Lambda container, the Terraform does provision it, and the cost at low volume is genuinely close to zero.</p><p>But I shipped it without understanding the constraint that governs whether the design works, without checking the size of my largest dependency, and without locking the tree that holds it together. Then I wrote up the symptom of hitting that constraint as a troubleshooting tip. The README told readers to raise the timeout, which is treating a fever with a thicker blanket.</p><p>The unbuildable image is the one that stings, because it’s the failure with the longest fuse and the cheapest fix. Everything else here is a judgment call I’d defend as reasonable for the time. That one was just laziness with a one-year delay on the invoice.</p><p>On the numbers: everything here is measured. The image sizes come from real builds, the init timings from the function deployed to Lambda at 2048 MB. The one caveat is that the dependency tree had to be repaired before it would build at all, so this is the 2025 design on 2026 pins rather than a byte-exact resurrection. The original CloudWatch data is long gone and I haven’t tried to reconstruct it.</p><p>If you’re building this today, use Bedrock unless you have a specific reason not to. If you have that reason: lock your dependency tree, read what pip is actually downloading, and measure Init Duration before you tune anything else.</p><p><em>Code: </em><a href="https://github.com/amrutp24/llm-lambda"><em>github.com/amrutp24/llm-lambda</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bcc1dc6995f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I blogged about an AWS docs gap. Eight days later, AWS closed it.]]></title>
            <link>https://amruteng.medium.com/i-blogged-about-an-aws-docs-gap-eight-days-later-aws-closed-it-f335d81d8a7e?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/f335d81d8a7e</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[durable-functions]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Sun, 02 Aug 2026 00:38:46 GMT</pubDate>
            <atom:updated>2026-08-02T00:38:46.759Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p><em>A follow-up to </em><a href="https://amruteng.medium.com/lambda-functions-can-pause-for-free-now-i-built-an-ai-agent-pipeline-to-test-that-66c2623fb7a7"><em>Lambda functions can pause for free now — I built an AI agent pipeline to test that</em></a><em>.</em></p><p>That post ended with a list of sharp edges I hit while building on Lambda durable functions. A developer on the Lambda product team read it, commented, and linked a merged PR and a new docs section that fix the two biggest ones. I’m writing down what happened because it’s the best outcome a blog post of mine has ever had, and because I think I can explain why it worked when I never even filed an issue.</p><h3>The two problems</h3><p>For anyone who skipped the first post: I built a three-agent content pipeline (researcher → writer → editor) on Lambda durable functions, the checkpoint/replay feature AWS shipped at re:Invent 2025. The feature mostly delivered. My pipeline suspended overnight waiting for human approval and billed nothing, which is the whole point. But two things cost me real time.</p><p>The first was IAM. My approval endpoint calls send_durable_execution_callback_success to wake the suspended function. I granted that action on the function ARN and got AccessDeniedException anyway. It took about an hour to figure out that callbacks are authorized against the durable execution&#39;s ARN, which is a sub-resource of a function <em>version</em>, so a policy on the unqualified function ARN never matches. The grant has to be:</p><pre>&quot;Resource&quot;: &quot;arn:aws:lambda:us-east-1:123456789012:function:my-function:*&quot;</pre><p>That trailing :* wasn&#39;t documented anywhere I could find at the time. I worked it out from the shape of the error and put it in my Terraform module&#39;s design notes.</p><p>The second was a mismatch between the docs and the SDK. The examples I was working from showed a one-argument submitter function for wait_for_callback and a plain timeout=86400. The SDK I actually installed invokes the submitter with two positional arguments and wants a WaitForCallbackConfig. If you copy the example, you get a runtime error.</p><h3>What happened next</h3><p>I published on July 6. On July 11 I shared the post in an AWS community Slack channel and asked for honest feedback. On July 14 a PR merged in the SDK repo, and the same day someone from AWS replied in that Slack thread to say the team had fixed the issue, with the PR linked. The next day, a developer on the Lambda product team left a longer comment on the post itself:</p><blockquote>“…your detailed write-up helped us to find and close the docs gaps to save the next person the hour you lost :-)”</blockquote><p>Two concrete things came out of it.</p><p>First, the Lambda developer guide now has a <a href="https://docs.aws.amazon.com/lambda/latest/dg/durable-security.html#durable-callback-permissions">security and permissions page for durable functions</a> that covers exactly the thing I lost an hour to. It explains that a durable execution ARN extends the function ARN with a qualifier, that an unqualified ARN in a policy will never match one, and it shows the :* pattern in a worked example for the callback actions.</p><p>Second, the stale examples were tracked down and removed in <a href="https://github.com/aws/aws-durable-execution-sdk-python/pull/538">aws-durable-execution-sdk-python#538</a>. The PR confirms the submitter bug (it really is invoked with two positional arguments, callback_id and a WaitForCallbackContext) and turned up a third problem I hadn&#39;t caught: the old examples imported WaitForCallbackConfig from the wrong module path.</p><h3>The part I didn’t expect</h3><p>The team traced the stale examples not to the developer guide but to AGENTS.md in the SDK repo, the file that tells AI coding agents how to use the SDK. It had grown to 929 lines of duplicated documentation, drifted out of sync with the actual code, and the fix was to cut it down to 69 lines that defer to the developer guide.</p><p>I built an AI agent pipeline, with an AI coding agent, and the wrong examples we kept hitting were in the file written specifically to teach AI agents the SDK. The agent was doing exactly what it was told. The instructions were just wrong.</p><p>I suspect this will become a common failure mode. AGENTS.md-style files are turning into a second documentation surface, and they rot the same way READMEs do, except that when a README rots a human squints at it, and when an agent-instructions file rots the bad example gets typed into your codebase at 60 tokens a second. What AWS did here is the right fix: keep the agent file thin and point it at a single source of truth.</p><h3>The docs, before and after</h3><p>Before, the callback permissions story was an AccessDeniedException, an hour of head-scratching, and an educated guess about ARN sub-resources. Now the docs have a worked policy:</p><pre>{<br>    &quot;Sid&quot;: &quot;SendCallbacks&quot;,<br>    &quot;Effect&quot;: &quot;Allow&quot;,<br>    &quot;Action&quot;: [<br>        &quot;lambda:SendDurableExecutionCallbackSuccess&quot;,<br>        &quot;lambda:SendDurableExecutionCallbackFailure&quot;,<br>        &quot;lambda:SendDurableExecutionCallbackHeartbeat&quot;<br>    ],<br>    &quot;Resource&quot;: &quot;arn:aws:lambda:us-east-1:123456789012:function:myDurableFunction:*&quot;<br>}</pre><p>The next person building a human-in-the-loop approval flow reads one paragraph instead of debugging IAM by vibes.</p><h3>Two things they taught me back</h3><p>The comment also had implementation tips worth passing on.</p><p>You can reject with an exception instead of a payload. I modeled rejection as send_durable_execution_callback_success with {&quot;approved&quot;: false}. There&#39;s also send_durable_execution_callback_failure, which surfaces the rejection to the orchestrator as an exception. Which one fits depends on whether &quot;human said no&quot; is a normal outcome or an error in your domain. For my pipeline, the payload approach still makes sense; for a compliance gate that should halt everything, the failure callback is cleaner.</p><p>And if you don’t like inline lambdas for steps, there’s a <a href="https://docs.aws.amazon.com/durable-execution/sdk-reference/operations/step/#the-steps-function">@durable_step decorator</a> for top-level functions.</p><h3>Why a blog post worked</h3><p>I never filed a GitHub issue, and I’m not recommending that. An issue would have been the proper channel, and PR #538 shows the team acts on them. But a few things made the post work anyway.</p><p>It had a reproduction. The exact error, the exact fix, the exact wrong example, all tested against a live deployment. Underneath the prose, it was a bug report with better distribution.</p><p>It was fair. The first post was mostly positive because the feature is mostly good, and criticism inside an honest review travels further than a takedown. People who build the thing will actually read it to the end.</p><p>It was early. Durable functions were a few months old, so a detailed real-world account was rare enough to get passed around by people who could act on it.</p><p>And I didn’t just publish it and wait. Sharing it in the Slack channel is visibly what worked here, since the reply about the fix came back through that same thread three days later. If the post had only lived on Medium, I doubt any of this would have happened.</p><p>If you hit a docs gap, write it up with the reproduction. Worst case, you save the next person an hour. Best case, the docs change and someone from the product team shows up in your comments with the merge links</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f335d81d8a7e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lambda functions can pause for free now. I built an AI agent pipeline to test that.]]></title>
            <link>https://amruteng.medium.com/lambda-functions-can-pause-for-free-now-i-built-an-ai-agent-pipeline-to-test-that-66c2623fb7a7?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/66c2623fb7a7</guid>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[terraform-modules]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 07:11:31 GMT</pubDate>
            <atom:updated>2026-07-06T07:15:04.318Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mNZVbfRA9FEzupF-I3KqHw.png" /></figure><p>Every serverless AI workflow I’ve built has the same problem. At some point a human needs to review something, and now you have to keep state alive while you wait. Step Functions can do it, but you’re paying for the state machine, wiring up a callback pattern, and hoping your payload fits under the size limit. I’ve also done the Lambda + SQS + polling worker version. It works. It’s ugly.</p><p>At re:Invent 2025 AWS shipped Lambda durable functions. A durable function can stop in the middle of its code, wait (for ten seconds or ten months), and continue from that exact line later. You don’t pay for the waiting time at all. AWS’s launch material pointed at multi-agent AI workflows and human approvals as the main use case, so I built one to see if it holds up.</p><p><strong>What I built</strong></p><p>A content pipeline with three agents:</p><p>1. A researcher that turns a topic into an outline</p><p>2. A writer that drafts a post from the outline</p><p>3. An editor that scores the draft 1 to 10. If the score is too low, the draft goes back to the writer with feedback and they loop.</p><p>When the editor is satisfied, the function suspends and waits for a human to approve or reject the draft. Nothing runs and nothing is billed during the wait. When someone hits the approve endpoint, Lambda resumes the function mid-execution and it publishes the draft to S3.</p><p>All three agents are the same model (Claude Haiku 4.5 on Bedrock) with different system prompts. The model isn’t the interesting part here. The orchestration is.</p><p><strong>Steps and waits</strong></p><p>The durable execution SDK hands your function a DurableContext instead of the normal Lambda context. Two methods do the work:</p><ul><li>context.step(…) runs some logic and checkpoints the result. If the function gets interrupted later, completed steps don’t re-run. Lambda replays their stored results.</li><li>context.wait_for_callback(…) gives you a callback ID, suspends the function, and resumes it when something outside calls back with that ID.</li></ul><p>Here’s the orchestrator, trimmed down:</p><pre>from aws_durable_execution_sdk_python import DurableContext, durable_execution<br>from aws_durable_execution_sdk_python.config import Duration, WaitForCallbackConfig<br><br>@durable_execution<br>def lambda_handler(event, context: DurableContext):<br>    topic = event[&quot;topic&quot;]<br><br>    outline = context.step(lambda _: call_agent(RESEARCH_PROMPT, topic), name=&quot;research&quot;)<br>    draft = context.step(lambda _: call_agent(WRITE_PROMPT, f&quot;{topic}\n{outline}&quot;), name=&quot;write-0&quot;)<br><br>    for revision in range(MAX_REVISIONS):<br>        critique = json.loads(<br>            context.step(lambda _: call_agent(EDIT_PROMPT, draft), name=f&quot;critique-{revision}&quot;)<br>        )<br>        if critique[&quot;score&quot;] &gt;= APPROVAL_SCORE_THRESHOLD:<br>            break<br>        draft = context.step(<br>            lambda _: call_agent(REVISE_PROMPT, f&quot;{draft}\n{critique[&#39;feedback&#39;]}&quot;),<br>            name=f&quot;write-{revision + 1}&quot;,<br>        )<br><br>    def request_approval(callback_id, _ctx):<br>        # store callback_id + draft in DynamoDB so an API route can find it later<br>        set_status(execution_id, callback_id=callback_id, draft=draft, status=&quot;AWAITING_APPROVAL&quot;)<br><br>    # The callback result arrives as a raw string, decode it yourself<br>    approval_raw = context.wait_for_callback(<br>        request_approval,<br>        name=&quot;human-approval&quot;,<br>        config=WaitForCallbackConfig(timeout=Duration.from_seconds(86400)),<br>    )<br>    approval = json.loads(approval_raw) if approval_raw else None<br><br>    if not approval or not approval.get(&quot;approved&quot;):<br>        return {&quot;status&quot;: &quot;REJECTED&quot;}<br><br>    final_url = context.step(lambda _: publish_to_s3(draft), name=&quot;publish&quot;)<br>    return {&quot;status&quot;: &quot;PUBLISHED&quot;, &quot;url&quot;: final_url}</pre><p>Notice the revision loop is a normal for loop. No state machine JSON, no Choice states. The editor’s decision is a Python if statement, and branching on it is safe because the score came out of a checkpointed step. On replay the stored value is reused instead of calling Bedrock again.</p><p>The zero-cost wait is real. I left a run sitting at <em>AWAITING_APPROVAL</em> overnight and the bill for those hours was zero. No polling worker burning invocations, no state machine charging per transition.</p><p><strong>Things that bit me</strong></p><p>A few problems I hit that you’ll probably hit too.</p><p><strong>**The callback IAM permission.**</strong> My first approval attempt failed with <em>AccessDeniedException</em> even though I had granted <em>lambda:SendDurableExecutionCallbackSuccess</em> on the function’s ARN. Turns out the callback resource is a sub-resource of the versioned function ARN, something like `<em>function:my-fn:2/durable-execution/&lt;execution-id&gt;/&lt;callback-id&gt;</em>`, so a policy on the bare function ARN never matches. You need a trailing wildcard: `<em>”${function_arn}:*”</em>`. This one cost me an hour.</p><p><strong>**The docs and the installed SDK disagree.**</strong> The doc examples show a one-argument submitter function and `timeout=86400`. The SDK version pip actually installs wants a two-argument submitter and a WaitForCallbackConfig with a Duration object. The feature is a few months old and AWS tells you to pin the SDK version, and they’re right. When something this new misbehaves, read the installed source, not the docs.</p><p><strong>**LLMs wrap JSON in markdown fences.**</strong> My first live run died at the critique step because the editor returned its JSON inside json fences even though the prompt said not to. Five lines of fence stripping before json.loads fixed it. One nice side effect of the versioning model: in-flight executions stay pinned to the code version that started them, so deploying the fix didn’t disturb anything already running.</p><p><strong>Getting the human in</strong></p><p>The approval side is a small HTTP API in front of a regular, non-durable Lambda:</p><pre>- POST /posts starts a run and returns an execution_id<br>- GET /posts/{id} shows the status and the current draft<br>- POST /posts/{id}/approve approves or rejects</pre><p>The approve route is the glue. It looks up the callback_id that the orchestrator stashed in DynamoDB before suspending, then tells Lambda the callback succeeded:</p><pre>lambda_client.send_durable_execution_callback_success(<br>  CallbackId=item[&quot;callback_id&quot;],<br>  Result=json.dumps({&quot;approved&quot;: approved}),<br>  )</pre><p>That one call wakes the orchestrator, it replays its checkpoint log, and execution continues into the publish step.</p><p><strong>Running it</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7TTmLUCGa4i6plGnmeSsRQ.png" /></figure><p>The Durable executions tab in the Lambda console shows every operation with timestamps. You can see each step, the WaitForCallback where it suspended, and the resume after the approval:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mwxCa-80fmmfkW8QPUy_cA.png" /></figure><p>The durable configuration on the function is two values: how long the whole execution is allowed to live, and how long Lambda keeps the checkpoint history after it finishes. Note this is separate from the normal Lambda timeout, which still applies to each individual invocation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*twE5_Y2heZ7AZmd8Imsh7A.png" /></figure><p><strong>What it cost</strong></p><p>Building all of this, including the failed runs, several end-to-end tests, an overnight suspension, and a test suite that deploys and tears down a full copy of the stack:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A2lxXrMV1pCxh8wyNqsKpw.png" /></figure><p>Twenty one cents, almost all of it Bedrock inference. Each run makes 5 to 7 Haiku calls. The waiting, which is where these workflows spend most of their wall-clock time, costs nothing.</p><p><strong>Code</strong></p><p>Everything is on GitHub: <a href="https://github.com/amrutp24/durable-ai-agent-pipeline">https://github.com/amrutp24/durable-ai-agent-pipeline</a>. It deploys with terraform apply and the README has a troubleshooting table for the problems above.</p><p>I also extracted the infrastructure into a Terraform module, published on the registry as amrutp24/durable-agent-pipeline/aws (<a href="https://registry.terraform.io/modules/amrutp24/durable-agent-pipeline/aws/latest">https://registry.terraform.io/modules/amrutp24/durable-agent-pipeline/aws)</a>.</p><p>If you want to get hands-on with agents on AWS, this is a good place to start. The judgment part of an agent, the should-I-retry and should-I-wait decisions, ends up as ordinary Python instead of an orchestration DSL.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66c2623fb7a7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Streaming Events into BigQuery with Pub/Sub, Cloud Run, and Terraform]]></title>
            <link>https://amruteng.medium.com/streaming-events-into-bigquery-with-pub-sub-cloud-run-and-terraform-c1df98417e5b?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1df98417e5b</guid>
            <category><![CDATA[bigquery]]></category>
            <category><![CDATA[google-cloud-run]]></category>
            <category><![CDATA[google-pubsub]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Sun, 28 Jun 2026 02:06:36 GMT</pubDate>
            <atom:updated>2026-06-28T02:06:36.969Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BFCBKWm2DL4vQUQF2UQLtw.png" /></figure><p>Ingesting real-time events into BigQuery is a problem most backend teams hit eventually. The naive solution is a long-running VM or a Kubernetes pod that polls a queue — but that means managing servers, handling restarts, and paying for idle compute. There is a cleaner path: let GCP’s managed services do the heavy lifting, wire them together with Terraform, and ship something production-ready in an afternoon.</p><p>This post walks through a fully automated pipeline: a Pub/Sub topic receives JSON events, a push subscription delivers them to a Cloud Run service over HTTPS, and a Python Flask app streams each message directly into a BigQuery table. No servers to manage, no polling loops, no API keys sitting in environment variables.</p><p><strong>— -</strong></p><p><strong>## The Architecture</strong></p><p>The flow is straightforward, which is part of what makes it maintainable.</p><pre>Publisher<br>    |<br>    v<br>[ Pub/Sub Topic ]<br>    |<br>    | (push subscription, OIDC token)<br>    v<br>[ Cloud Run Service ] ← Flask + Gunicorn, Dockerized<br>    |<br>    | (streaming insert)<br>    v<br>[ BigQuery Table ]<br>    |<br>    v (on repeated failure)<br>[ Dead-Letter Topic ]</pre><p>Every component is serverless. Cloud Run scales to zero when there is no traffic and scales out automatically under load. Pub/Sub handles durability and retry logic. BigQuery handles the storage and query layer. The only thing you operate is the application code inside the container.</p><p><strong>— -</strong></p><p><strong>## Why Push Instead of Pull?</strong></p><p>Pub/Sub supports two subscription modes: pull (your consumer calls the API to fetch messages) and push (Pub/Sub calls your endpoint with each message). For a Cloud Run workload, push is the right choice for a few reasons.</p><p>First, it fits the request/response model Cloud Run is built around. The service only runs when there is work to do. A pull-based consumer would need to run continuously, defeating the purpose of a serverless runtime.</p><p>Second, push subscriptions handle backpressure automatically. If Cloud Run returns a non-2xx status, Pub/Sub retries with exponential backoff. You get retry semantics for free without writing any retry logic in your application.</p><p>Third — and this is the security highlight — push subscriptions support OIDC token authentication. Pub/Sub attaches a short-lived, Google-signed JWT to each request. Your Cloud Run service validates that token before processing anything. No API keys, no static secrets, no shared passwords. The identity is cryptographic and managed entirely by Google.</p><p><strong>— -</strong></p><p><strong>## The Flask Application</strong></p><p>The app is intentionally minimal. Its only jobs are: receive the Pub/Sub push payload, decode it, and write a row to BigQuery.</p><pre>import base64<br>import json<br>import os<br>from flask import Flask, request, jsonify<br>from google.cloud import bigquery<br>app = Flask(__name__)<br>BQ_PROJECT = os.environ[&quot;BQ_PROJECT&quot;]<br>BQ_DATASET = os.environ[&quot;BQ_DATASET&quot;]<br>BQ_TABLE = os.environ[&quot;BQ_TABLE&quot;]<br>bq_client = bigquery.Client(project=BQ_PROJECT)<br>table_ref = f&quot;{BQ_PROJECT}.{BQ_DATASET}.{BQ_TABLE}&quot;<br>@app.route(&quot;/&quot;, methods=[&quot;POST&quot;])<br>def handle_pubsub():<br>envelope = request.get_json(silent=True)<br>if not envelope or &quot;message&quot; not in envelope:<br>return jsonify({&quot;error&quot;: &quot;invalid payload&quot;}), 400<br>message = envelope[&quot;message&quot;]<br>data_raw = base64.b64decode(message.get(&quot;data&quot;, &quot;&quot;)).decode(&quot;utf-8&quot;)<br>attrs = message.get(&quot;attributes&quot;, {})<br>row = {<br>&quot;message_id&quot;: message.get(&quot;messageId&quot;),<br>&quot;publish_time&quot;: message.get(&quot;publishTime&quot;),<br>&quot;data&quot;: data_raw,<br>&quot;attributes&quot;: json.dumps(attrs),<br>}<br>errors = bq_client.insert_rows_json(table_ref, [row])<br>if errors:<br>return jsonify({&quot;error&quot;: errors}), 500<br>return jsonify({&quot;status&quot;: &quot;ok&quot;}), 200<br>@app.route(&quot;/health&quot;, methods=[&quot;GET&quot;])<br>def health():<br>return jsonify({&quot;status&quot;: &quot;healthy&quot;}), 200<br></pre><p>A few things worth noting here. The BigQuery client is initialized once at module load time, not per request — this avoids re-authenticating on every message. The `insert_rows_json` call uses the streaming insert API, which makes data available for queries within seconds. The `attributes` dict is serialized to a JSON string so it maps cleanly to a BigQuery STRING column without needing a RECORD type.</p><p>The app runs under Gunicorn with one worker and eight threads. One worker keeps the BigQuery client truly shared across requests. Eight threads give reasonable concurrency for I/O-bound work without the overhead of multiple processes.</p><pre>gunicorn - bind 0.0.0.0:8080 - workers 1 - threads 8 main:app</pre><p>The BigQuery table schema is kept flat and generic on purpose:</p><pre>| Column       | Type       | Description                        |<br>|--------------|------------|------------------------------------|<br>| message_id   | STRING     | Pub/Sub message ID                 |<br>| publish_time | TIMESTAMP  | When the message was published     |<br>| data         | STRING     | Raw decoded payload                |<br>| attributes   | STRING     | JSON-serialized message attributes |</pre><p>Storing `data` as a raw string keeps the ingestion layer decoupled from the event schema. If your downstream team wants structured columns, they define a view or a downstream table transformation. The pipeline itself does not need to know anything about the shape of your events.</p><p><strong>— -</strong></p><p><strong>## The Terraform Module</strong></p><p>The entire infrastructure is defined as a reusable Terraform module published to the Terraform Registry at:</p><p><a href="https://registry.terraform.io/modules/amrutp24/pubsub-bq-pipeline/google/latest">https://registry.terraform.io/modules/amrutp24/pubsub-bq-pipeline/google/latest</a></p><p>Calling it in your own configuration looks like this:</p><pre>module &quot;pubsub_bq_pipeline&quot; {<br>    source = &quot;amrutp24/pubsub-bq-pipeline/google&quot;<br>    version = &quot;~&gt; 1.0&quot;<br>    project = var.project<br>    region = &quot;us-central1&quot;<br>    topic_name = &quot;my-events&quot;<br>    bq_dataset_id = &quot;events&quot;<br>    bq_table_id = &quot;raw_events&quot;<br>    image = &quot;us-central1-docker.pkg.dev/${var.project}/pipeline/app:latest&quot;<br>    service_name = &quot;pubsub-to-bq&quot;<br>    labels = { environment = &quot;dev&quot; }<br>}</pre><p>The module provisions everything the pipeline needs:</p><p>- A Pub/Sub topic and a push subscription pointed at the Cloud Run service URL</p><p>- An Artifact Registry repository for your container images</p><p>- A Cloud Run service with the correct environment variables set</p><p>- A BigQuery dataset and table with the schema described above</p><p>- Two service accounts: one for the Cloud Run service (with `bigquery.dataEditor` and `bigquery.jobUser` roles), one for the Pub/Sub subscription (with `run.invoker` on the Cloud Run service)</p><p>- IAM bindings wiring everything together</p><p>The multi-environment layout (dev, test, prod) uses partial backend configuration so the module is safe to publish publicly — no backend bucket names or project IDs are hardcoded.</p><p><strong>— -</strong></p><p><strong>## Security: OIDC Auth Between Pub/Sub and Cloud Run</strong></p><p>This is worth dwelling on because it is one of the more elegant parts of the design.</p><p>When the push subscription delivers a message, it attaches an `Authorization: Bearer &lt;token&gt;` header. That token is a Google-signed OIDC JWT issued to the Pub/Sub service account. Cloud Run, by default, validates incoming OIDC tokens when the service is not publicly accessible. The Pub/Sub service account has the `run.invoker` IAM role on the Cloud Run service, and that is the only credential needed.</p><p>What this means in practice: the Cloud Run service has no public URL that anonymous traffic can reach. If someone sends an unauthenticated POST to the endpoint, Cloud Run rejects it at the infrastructure level before your application code even runs. There are no API keys to rotate, no secrets to store, and no custom authentication middleware to write.</p><p>This is the correct way to handle service-to-service authentication on GCP. Workload identity, not static secrets.</p><p><strong>— -</strong></p><p><strong>## Dead-Letter Topic for Production Reliability</strong></p><p>Retry semantics are important. If your Cloud Run service returns a 5xx — say, BigQuery is briefly unavailable or the container is cold-starting — Pub/Sub will retry the message with exponential backoff. That is good. But you need a cap on retries, otherwise a persistently malformed message can block delivery indefinitely.</p><p>The module configures a dead-letter topic with a maximum of five delivery attempts. After five failures, the message is forwarded to the dead-letter topic rather than being retried. You can then subscribe to the dead-letter topic separately for alerting or manual inspection.</p><pre>resource &quot;google_pubsub_subscription&quot; &quot;push_sub&quot; {<br>  # …<br>  dead_letter_policy {<br>  dead_letter_topic = google_pubsub_topic.dead_letter.id<br>  max_delivery_attempts = 5<br>  }<br>}</pre><p>Five attempts gives the pipeline enough room to recover from transient errors without letting poison-pill messages pile up indefinitely. In a production system, you would add a Cloud Monitoring alert on the dead-letter topic’s message count so you are notified when messages start failing.</p><p><strong>— -</strong></p><p><strong>## CI/CD and Quality Gates</strong></p><p>The module repository runs a GitHub Actions pipeline on every pull request:</p><p>- `terraform fmt` — enforces consistent formatting</p><p>- `terraform validate` — catches configuration errors before any infrastructure is touched</p><p>- Checkov — static security scan of the Terraform configuration, flagging things like overly permissive IAM bindings or missing audit logging</p><p>- Snyk — scans IaC and Go dependencies for known vulnerabilities</p><p>The integration test is a Terratest suite written in Go. It deploys the actual infrastructure into a real GCP project, publishes a test message to the Pub/Sub topic, waits for it to propagate through Cloud Run, and then queries BigQuery to assert the row exists. Only when that assertion passes does the test clean up and destroy the resources.</p><p>Releases follow semver and are cut automatically by a GitHub Actions workflow that tags the repository and publishes the new version to the Terraform Registry.</p><p><strong>— -</strong></p><p><strong>## Putting It Together</strong></p><p>If you want to use this pipeline in your own GCP project, the steps are:</p><p>1. Build and push your container image to Artifact Registry (or any registry your Cloud Run service can pull from).</p><p>2. Add the module call to your Terraform configuration with your project ID, region, and image URL.</p><p>3. Run `terraform init` and `terraform apply`.</p><p>4. Publish a test message to the Pub/Sub topic and query BigQuery to verify the row appeared.</p><p>The module handles all the IAM wiring, so you do not need to manually configure service accounts or grant permissions. The push subscription URL is derived from the Cloud Run service URL automatically.</p><p>The full module is available at:</p><p><a href="https://registry.terraform.io/modules/amrutp24/pubsub-bq-pipeline/google/latest">registry.terraform.io/modules/amrutp24/pubsub-bq-pipeline/google</a></p><p>The source is on GitHub and accepts contributions. If your use case requires schema evolution, multi-table routing, or Dataflow transformations between Pub/Sub and BigQuery, those are natural extensions to build on top of this foundation — the ingestion layer stays the same.</p><p><strong>— -</strong></p><p><strong>## Wrapping Up</strong></p><p>The pipeline described here — Pub/Sub push to Cloud Run to BigQuery — is one of the most cost-effective ways to ingest streaming events on GCP. You pay for message delivery, container execution time, and BigQuery storage. When there is no traffic, you pay almost nothing. The OIDC authentication between Pub/Sub and Cloud Run eliminates an entire class of credential management problems. And the Terraform module means you can stand up a new environment in minutes rather than hours.</p><p>The architectural pattern is simple enough to understand in one sitting and solid enough to run in production.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1df98417e5b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Serverless API with API Gateway, Lambda, and DynamoDB (Step by Step)]]></title>
            <link>https://amruteng.medium.com/deploying-a-serverless-api-with-api-gateway-lambda-and-dynamodb-step-by-step-469d47ea3411?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/469d47ea3411</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[api-gateway]]></category>
            <category><![CDATA[dynamodb]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Tue, 16 Sep 2025 06:02:46 GMT</pubDate>
            <atom:updated>2025-09-16T06:02:46.046Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p>Serverless architectures let you build applications without managing servers — and in AWS, one of the most common patterns is <strong>API Gateway + Lambda + DynamoDB</strong>.</p><p>In this post, I’ll walk through deploying a simple CRUD-style API that stores todo items in DynamoDB, powered by API Gateway (v1 REST API) and a Python Lambda function — all provisioned with Terraform.</p><h3>🔹 Architecture Overview</h3><pre>Client (HTTP request)<br>   → API Gateway (REST API)<br>      → Lambda function (Python)<br>         → DynamoDB (NoSQL table)</pre><ul><li><strong>API Gateway</strong> exposes REST endpoints like /items and /items/{id}.</li><li><strong>Lambda</strong> runs Python code to process requests.</li><li><strong>DynamoDB</strong> stores todo items (id, title, done).</li></ul><h3>🔹 Terraform Setup</h3><p>We’ll use a consistent naming convention:</p><pre>&lt;app&gt;-&lt;component&gt;-&lt;env&gt;-&lt;region&gt;</pre><p>For example:</p><ul><li>serverless-api-items-dev-use1 (DynamoDB table)</li><li>serverless-api-handler-dev-use1 (Lambda)</li><li>serverless-api-dev-use1 (API Gateway)</li></ul><h3>DynamoDB Table</h3><pre>resource &quot;aws_dynamodb_table&quot; &quot;items&quot; {<br>  name         = &quot;serverless-api-items-dev-use1&quot;<br>  billing_mode = &quot;PAY_PER_REQUEST&quot;<br>  hash_key     = &quot;id&quot;<br>  attribute {<br>    name = &quot;id&quot;<br>    type = &quot;S&quot;<br>  }<br>}</pre><h3>Lambda Function</h3><pre>resource &quot;aws_lambda_function&quot; &quot;handler&quot; {<br>  function_name = &quot;serverless-api-handler-dev-use1&quot;<br>  role          = aws_iam_role.lambda_exec.arn<br>  runtime       = &quot;python3.12&quot;<br>  handler       = &quot;index.handler&quot;<br>  filename         = &quot;lambda.zip&quot;<br>  source_code_hash = filebase64sha256(&quot;lambda.zip&quot;)<br>  environment {<br>    variables = {<br>      TABLE_NAME = aws_dynamodb_table.items.name<br>    }<br>  }<br>}</pre><h3>API Gateway (REST API v1)</h3><pre>resource &quot;aws_api_gateway_rest_api&quot; &quot;main&quot; {<br>  name = &quot;serverless-api-dev-use1&quot;<br>}<br># /items<br>resource &quot;aws_api_gateway_resource&quot; &quot;items&quot; {<br>  rest_api_id = aws_api_gateway_rest_api.main.id<br>  parent_id   = aws_api_gateway_rest_api.main.root_resource_id<br>  path_part   = &quot;items&quot;<br>}<br># /items/{id}<br>resource &quot;aws_api_gateway_resource&quot; &quot;item&quot; {<br>  rest_api_id = aws_api_gateway_rest_api.main.id<br>  parent_id   = aws_api_gateway_resource.items.id<br>  path_part   = &quot;{id}&quot;<br>}<br># POST /items<br>resource &quot;aws_api_gateway_method&quot; &quot;post_items&quot; {<br>  rest_api_id   = aws_api_gateway_rest_api.main.id<br>  resource_id   = aws_api_gateway_resource.items.id<br>  http_method   = &quot;POST&quot;<br>  authorization = &quot;NONE&quot;<br>}<br>resource &quot;aws_api_gateway_integration&quot; &quot;post_items_lambda&quot; {<br>  rest_api_id             = aws_api_gateway_rest_api.main.id<br>  resource_id             = aws_api_gateway_resource.items.id<br>  http_method             = aws_api_gateway_method.post_items.http_method<br>  integration_http_method = &quot;POST&quot;<br>  type                    = &quot;AWS_PROXY&quot;<br>  uri                     = aws_lambda_function.handler.invoke_arn<br>}<br># GET /items/{id}<br>resource &quot;aws_api_gateway_method&quot; &quot;get_item&quot; {<br>  rest_api_id   = aws_api_gateway_rest_api.main.id<br>  resource_id   = aws_api_gateway_resource.item.id<br>  http_method   = &quot;GET&quot;<br>  authorization = &quot;NONE&quot;<br>}<br>resource &quot;aws_api_gateway_integration&quot; &quot;get_item_lambda&quot; {<br>  rest_api_id             = aws_api_gateway_rest_api.main.id<br>  resource_id             = aws_api_gateway_resource.item.id<br>  http_method             = aws_api_gateway_method.get_item.http_method<br>  integration_http_method = &quot;POST&quot;<br>  type                    = &quot;AWS_PROXY&quot;<br>  uri                     = aws_lambda_function.handler.invoke_arn<br>}</pre><p>Don’t forget the <strong>deployment + stage</strong> and a lambda_permission so API Gateway can invoke Lambda.</p><h3>🔹 Python Lambda Function</h3><p>Save this as lambda/index.py:</p><pre>import json, os, boto3<br>from decimal import Decimal<br>from botocore.exceptions import ClientError<br>TABLE_NAME = os.environ[&quot;TABLE_NAME&quot;]<br>ddb = boto3.resource(&quot;dynamodb&quot;).Table(TABLE_NAME)<br>def resp(status, body):<br>    return {&quot;statusCode&quot;: status,<br>            &quot;headers&quot;: {&quot;Content-Type&quot;: &quot;application/json&quot;},<br>            &quot;body&quot;: json.dumps(body, default=str)}<br>def handler(event, context):<br>    method = event.get(&quot;httpMethod&quot;)<br>    path   = event.get(&quot;path&quot;)<br>    body   = json.loads(event.get(&quot;body&quot;) or &quot;{}&quot;)<br>      if method == &quot;POST&quot; and path == &quot;/items&quot;:<br>        if &quot;id&quot; not in body or &quot;title&quot; not in body:<br>            return resp(400, {&quot;error&quot;: &quot;id and title required&quot;})<br>        try:<br>            ddb.put_item(Item={&quot;id&quot;: body[&quot;id&quot;], &quot;title&quot;: body[&quot;title&quot;], &quot;done&quot;: False})<br>            return resp(201, {&quot;ok&quot;: True, &quot;id&quot;: body[&quot;id&quot;]})<br>        except ClientError:<br>            return resp(500, {&quot;error&quot;: &quot;dynamodb error&quot;})<br>        if method == &quot;GET&quot; and path.startswith(&quot;/items/&quot;):<br>        item_id = path.split(&quot;/items/&quot;)[-1]<br>        res = ddb.get_item(Key={&quot;id&quot;: item_id})<br>        if &quot;Item&quot; not in res:<br>            return resp(404, {&quot;error&quot;: &quot;not found&quot;})<br>        return resp(200, res[&quot;Item&quot;])<br>    return resp(404, {&quot;error&quot;: &quot;route not found&quot;})</pre><p>Package and deploy:</p><pre>zip -j lambda.zip lambda/index.py<br>terraform apply -auto-approve</pre><h3>🔹 Test the API</h3><p>Grab the URL (from Terraform output):</p><pre>API_URL=$(terraform output -raw api_url)</pre><p><strong>Create an item</strong></p><pre>curl -X POST &quot;$API_URL/items&quot; \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;id&quot;:&quot;t1&quot;,&quot;title&quot;:&quot;Buy milk&quot;}&#39;<br># {&quot;ok&quot;: true, &quot;id&quot;: &quot;t1&quot;</pre><p><strong>Fetch the item</strong></p><pre>curl &quot;$API_URL/items/t1&quot;<br># {&quot;id&quot;:&quot;t1&quot;,&quot;title&quot;:&quot;Buy milk&quot;,&quot;done&quot;:false}</pre><h3>🔹 Lessons Learned</h3><ul><li><strong>Naming conventions matter</strong> — consistent names like serverless-api-items-dev-use1 make debugging easier.</li><li><strong>API Gateway v1 requires deployments</strong> — remember to redeploy after adding routes.</li><li><strong>Start small</strong> — this simple pattern (API Gateway → Lambda → DynamoDB) covers 80% of lightweight CRUD use cases.</li><li><strong>Logging is critical</strong> — use aws logs tail to watch Lambda run in real time.</li></ul><h3>🔹 When to Use This Pattern</h3><p>✅ <strong>Great for:</strong></p><ul><li>Small CRUD services</li><li>Prototypes or MVPs</li><li>Event-driven APIs</li></ul><p>❌ <strong>Less ideal for:</strong></p><ul><li>High-throughput apps with constant traffic (consider ECS/EKS)</li><li>Complex relational queries (consider RDS/Aurora)</li></ul><h3>🔹 Closing Thoughts</h3><p>This project shows how easy it is to stand up a <strong>serverless backend</strong> on AWS with just Terraform and a simple Lambda.</p><p>Next steps you could add:</p><ul><li>PUT /items/{id} → update items</li><li>DELETE /items/{id} → delete items</li><li><strong>CORS headers</strong> → frontend integration</li></ul><p>With this foundation, you can scale features without touching servers. That’s the real power of serverless. 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=469d47ea3411" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building RAG on a Budget Using AWS Lambda, Bedrock & OpenSearch]]></title>
            <link>https://amruteng.medium.com/building-rag-on-a-budget-using-aws-lambda-bedrock-opensearch-1f17d46b7b17?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f17d46b7b17</guid>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[opensearch]]></category>
            <category><![CDATA[lambda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Wed, 04 Jun 2025 06:50:08 GMT</pubDate>
            <atom:updated>2025-06-04T06:50:08.758Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p>Retrieval-Augmented Generation (RAG) lets language models fetch relevant context from external data sources before answering queries, improving the quality of responses without requiring expensive model retraining.</p><p>But hosting RAG setups is usually costly and complex — LLMs running on GPUs, vector DBs with managed clusters, API orchestration, etc. This post walks you through a <strong>cost-efficient RAG stack</strong> built entirely using <strong>AWS Lambda</strong>, <strong>Amazon Bedrock</strong>, and <strong>OpenSearch</strong>. Perfect for personal projects and low-traffic use cases.</p><h3>🧱 Stack Overview</h3><ul><li><strong>Document Ingestion + Embedding</strong>: Python script calling Titan Embeddings via Bedrock</li><li><strong>Vector Store</strong>: Amazon OpenSearch with KNN vector index</li><li><strong>Query + Completion</strong>: AWS Lambda function invoked through API Gateway</li><li><strong>Answer Generation</strong>: Claude v2 via Bedrock</li></ul><h3>🔧 Setup Steps</h3><h3>1. Deploy Infrastructure with Terraform</h3><p>Includes:</p><ul><li>OpenSearch domain with KNN vector index</li><li>Lambda function and IAM role</li><li>API Gateway for querying</li><li>S3 bucket for uploading zipped Lambda</li></ul><h3>2. Create KNN Index on OpenSearch</h3><pre>bash<br>python scripts/create_index.py</pre><p>Uses Titan embedding dimension 1024. The script creates the documents index.</p><h3>3. Embed and Upload Docs</h3><pre>bash<br>python embed_documents.py</pre><p>Takes hardcoded docs, calls Titan Embeddings via Bedrock, and stores them in OpenSearch.</p><h3>4. Deploy Lambda</h3><p>Terraform will upload it to S3 and create the Lambda function.</p><h3>5. Query Your RAG App</h3><p>Once deployed:</p><pre>bash<br>curl -X POST https://&lt;api-url&gt;/query \<br>  -H &quot;Content-Type: application/json&quot; \<br>  -d &#39;{&quot;query&quot;: &quot;What is OpenSearch?&quot;}&#39;</pre><h3>🐞 Issues I Faced</h3><ul><li><strong>AccessDeniedException</strong> when calling Bedrock — fixed by attaching inline policy with correct model ARNs.</li><li><strong>Mismatch in embedding dimensions (1024 vs 1536)</strong> — Titan V2 supports variable sizes, but your index and generation must match.</li><li><strong>Missing dependencies (</strong><strong>requests) in Lambda</strong> – fix by packaging them using PowerShell or bash.</li><li><strong>OpenSearch role access errors</strong> — ensure the Lambda IAM role is added to the domain access policy.</li></ul><h3>🤔 Why Use Both Titan &amp; Claude?</h3><ul><li><strong>Titan</strong>: Great for fast, cheap, multilingual vector embeddings.</li><li><strong>Claude</strong>: Higher-quality generation, better reasoning.</li></ul><p>This dual setup helps minimize cost while keeping answer quality high.</p><h3>✅ Conclusion</h3><p>You now have a low-cost, serverless RAG system that’s good enough to power search, FAQs, or assistant-style apps. No GPU instances. No custom model training. All AWS-native.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f17d46b7b17" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying an LLM to AWS Lambda — What Worked and What Didn’t]]></title>
            <link>https://amruteng.medium.com/deploying-an-llm-to-aws-lambda-what-worked-and-what-didnt-ad82952ed86d?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad82952ed86d</guid>
            <category><![CDATA[large-language-models]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Fri, 23 May 2025 03:26:57 GMT</pubDate>
            <atom:updated>2025-05-23T03:26:57.075Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploying an LLM to AWS Lambda — What Worked and What Didn’t</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p>I’ve seen a lot of posts about using large language models (LLMs), but most assume you’re using powerful GPUs or paid APIs. I wanted to try something different:</p><p><strong>Can you deploy an open-source LLM entirely inside AWS Lambda — using Docker — and run it serverless on-demand?</strong></p><p>Turns out, you can.</p><p>In this post, I’ll show you what worked, what didn’t, and how I got a lightweight LLM running inside a Lambda container — no GPU, no API calls, no servers.</p><h3>🔧 What I Wanted to Build</h3><p>The goal was to create a Lambda function that:</p><ul><li>Accepts a POST request like:</li></ul><pre>{&quot;text&quot;: &quot;Deploying LLMs to Lambda&quot;}</pre><ul><li>Returns a semantic embedding vector using <a href="https://huggingface.co/BAAI/bge-small-en">bge-small-en</a> via sentence-transformers</li></ul><p>Why? Because embeddings are like coordinates for meaning — you can use them for search, intent detection, or clustering later. But for this project, I just wanted the plumbing to work: model in, vector out.</p><h3>⚙️ What I Used</h3><ul><li><strong>Model</strong>: <a href="https://huggingface.co/BAAI/bge-small-en">BAAI/bge-small-en</a></li><li><strong>Packaging</strong>: AWS Lambda + Docker + ECR</li><li><strong>Languages/Libs</strong>: Python 3.11, sentence-transformers, transformers, torch, numpy</li><li><strong>Infra-as-Code</strong>: Terraform (for provisioning ECR, Lambda, and IAM roles)</li></ul><h3>📆 Folder Structure</h3><p>Here’s what the project structure looked like:</p><pre>llm-lambda/<br>├── Dockerfile<br>├── requirements.txt<br>├── build_model.py       # downloads the model locally<br>└── app/<br>    ├── lambda_function.py<br>    └── model/<br>        └── bge-small-en/</pre><h3>📁 Terraform for Infra Automation</h3><p>To make this more repeatable, I used Terraform to automate the infrastructure setup. This included:</p><ul><li>Creating the ECR repository</li><li>Defining the IAM role for Lambda with required permissions</li><li>Deploying the Lambda function with the container image</li></ul><p>Using Terraform helped me avoid manual steps and made testing changes much faster.</p><p>If you’re using Terraform, make sure to include:</p><ul><li>aws_lambda_function resource with image_uri</li><li>aws_iam_role with AWSLambdaBasicExecutionRole and ecr:GetAuthorizationToken</li><li>aws_ecr_repository for storing your Docker image</li></ul><h3>🐳 Dockerfile</h3><p>This is the final Dockerfile that worked:</p><pre>FROM public.ecr.aws/lambda/python:3.11<br>WORKDIR /var/task<br>COPY app/ .<br>COPY requirements.txt .<br>RUN python3 -m pip install --no-cache-dir -r requirements.txt -t /var/task<br>CMD [&quot;lambda_function.lambda_handler&quot;]</pre><h3>📄 requirements.txt</h3><pre>sentence-transformers==2.4.0<br>transformers==4.35.2<br>huggingface-hub==0.17.3<br>torch==2.0.1<br>numpy==1.26.4</pre><h3>🚀 Deploying the Image</h3><p>After building and tagging the image:</p><pre>docker build --platform linux/amd64 -t llm-lambda .<br>docker tag llm-lambda:latest $IMAGE_URI<br>docker push $IMAGE_URI</pre><p>Then update your Lambda:</p><pre>aws lambda update-function-code \<br>  --function-name llm-lambda \<br>  --image-uri $IMAGE_URI \<br>  --region us-east-1 \</pre><h3>🧰 Invoking the Function</h3><pre>aws lambda invoke \<br>  --function-name llm-lambda \<br>  --payload &#39;{&quot;text&quot;: &quot;Deploying LLMs to Lambda&quot;}&#39; \<br>  --cli-binary-format raw-in-base64-out \<br>  --region us-east-1 \<br>  response.json<br>cat response.json</pre><p>Response:</p><pre>{<br>  &quot;statusCode&quot;: 200,<br>  &quot;body&quot;: &quot;{\&quot;embedding\&quot;: [0.123, -0.045, 0.87, ...]}&quot;<br>}</pre><h3>✅ Final Thoughts</h3><p>You don’t need a GPU or hosted inference API to run lightweight LLMs — you can do it serverless with Lambda and a bit of Docker work.</p><p>This setup gives you:</p><ul><li>Zero idle cost</li><li>On-demand embeddings</li><li>Total control over the model</li></ul><p>It’s not meant for real-time ML at scale (cold starts exist), but it’s perfect for internal tools, one-off jobs, or even fun experiments like this.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad82952ed86d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Amazon SageMaker: A Hands-On Journey for AWS Practitioners]]></title>
            <link>https://amruteng.medium.com/exploring-amazon-sagemaker-a-hands-on-journey-for-aws-practitioners-2992d1ba4716?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/2992d1ba4716</guid>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Sun, 11 May 2025 23:11:00 GMT</pubDate>
            <atom:updated>2025-05-11T23:11:00.639Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><h3>Why I Gave SageMaker a Shot</h3><p>As someone who’s comfortable with AWS infrastructure and DevOps workflows, I’d skimmed through SageMaker docs before — but never seriously tried it in a real workflow. Recently, I decided to spend time understanding it beyond the surface. Not from an ML expert’s lens, but from the perspective of someone who understands AWS well and wanted to evaluate SageMaker’s real-world usability and developer experience.</p><p>This post walks through what I did, what confused me, what surprised me, and what I’d recommend to other AWS practitioners trying to add ML deployment to their toolbox.</p><h3>Step-by-Step: My First End-to-End Try with SageMaker</h3><h3>1. Setting Things Up with Terraform</h3><p>Instead of manually provisioning SageMaker Studio via the AWS Console, I used Terraform to automate the setup. This gave me repeatability and tighter control over permissions.</p><pre>resource &quot;aws_sagemaker_domain&quot; &quot;studio&quot; {<br>  domain_name = &quot;my-sagemaker-domain&quot;<br>  auth_mode   = &quot;IAM&quot;<br>  default_user_settings {<br>    execution_role = aws_iam_role.sagemaker_execution_role.arn<br>  }<br>}<br><br>resource &quot;aws_iam_role&quot; &quot;sagemaker_execution_role&quot; {<br>  name = &quot;sagemaker-execution-role&quot;<br>  assume_role_policy = jsonencode({<br>    Version = &quot;2012-10-17&quot;,<br>    Statement = [<br>      {<br>        Action = &quot;sts:AssumeRole&quot;,<br>        Principal = {<br>          Service = &quot;sagemaker.amazonaws.com&quot;<br>        },<br>        Effect = &quot;Allow&quot;,<br>        Sid    = &quot;&quot;<br>      }<br>    ]<br>  })<br>}<br><br>resource &quot;aws_iam_role_policy_attachment&quot; &quot;sagemaker_policy_attach&quot; {<br>  role       = aws_iam_role.sagemaker_execution_role.name<br>  policy_arn = &quot;arn:aws:iam::aws:policy/AmazonSageMakerFullAccess&quot;<br>}</pre><p>Once applied, this provisioned a SageMaker Studio domain with the required IAM role. In a production setup, you should avoid using broad policies like AmazonSageMakerFullAccess and instead assign only the specific permissions needed for your use case, following the principle of least privilege.</p><h3>2. Choosing a Sample Project</h3><ul><li>I picked the <strong>Iris dataset</strong> using one of SageMaker’s built-in examples.</li><li>The goal: train a model to classify flower types.</li></ul><h3>3. Training a Model</h3><ul><li>I used a <strong>built-in XGBoost algorithm</strong> to avoid writing custom ML code.</li><li>You provide a CSV, point it to an S3 bucket, and use the SageMaker SDK in a Jupyter notebook to train.</li></ul><pre>from sagemaker import XGBoost<br>from sagemaker.session import Session<br>xgb = XGBoost(entry_point=&#39;train.py&#39;, role=role, instance_type=&#39;ml.m5.xlarge&#39;)<br>xgb.fit({&#39;train&#39;: s3_input_train})</pre><ul><li>AWS recommends ml.m5.xlarge or ml.c5.xlarge for typical XGBoost jobs. Avoid GPU instances unless you&#39;re doing deep learning.</li></ul><h3>4. Deploying the Model</h3><ul><li>After training, I deployed the model to an endpoint:</li></ul><pre>predictor = xgb.deploy(initial_instance_count=1, instance_type=&#39;ml.m5.large&#39;)</pre><ul><li>I then sent test data using predictor.predict().</li></ul><h3>5. Cleaning Up</h3><ul><li>This part’s important: if you don’t delete the endpoint, you’ll keep getting charged.</li></ul><pre>predictor.delete_endpoint()</pre><h3>Challenges I Faced</h3><ul><li><strong>IAM Confusion</strong>: Even though I set it up via Terraform, understanding the minimum permissions needed was tricky.</li><li><strong>Cost Anxiety</strong>: Even with the free tier, it’s not super clear what costs what. I recommend using AWS Budgets and enabling Cost Explorer.</li><li><strong>Interface Overload</strong>: Studio has a lot going on — many features I didn’t need right away.</li></ul><h3>Cool Discoveries</h3><ul><li><strong>Built-in Algorithms</strong> are a blessing. You don’t need to write ML code to get started.</li><li><strong>Notebook Auto-Save</strong>: The Jupyter notebooks auto-save and version automatically.</li><li><strong>S3 Integration</strong> is seamless — datasets, models, and logs all land there by default.</li><li><strong>Custom Docker Support</strong>: You can bring your own container image for complete control.</li><li><strong>SageMaker JumpStart</strong>: Pre-built models and solutions are available for quick experimentation.</li></ul><h3>Would I Use It Again?</h3><p>Yes — with clearer goals. Now that I’ve used SageMaker end-to-end, I’d probably:</p><ul><li>Use it to host trained models without managing EC2 manually</li><li>Use Studio only for experimentation</li><li>Set budget alerts before deeper usage</li><li>Build Terraform modules to scale training pipelines</li><li>Configure encryption with KMS for security best practices</li></ul><h3>Key Takeaways</h3><ul><li>SageMaker is approachable for AWS engineers but has a learning curve</li><li>Built-in algorithms simplify early experimentation</li><li>Terraform helps automate SageMaker domain provisioning</li><li>Always clean up endpoints to avoid unnecessary charges</li><li>Use least-privilege IAM roles and monitor usage with AWS Budgets</li><li>A good tool for bridging infra skills with ML deployment</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2992d1ba4716" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI-Powered Cloud Cost Optimization: How AWS Forecast Can Reduce Your Cloud Bills]]></title>
            <link>https://awstip.com/ai-powered-cloud-cost-optimization-how-aws-forecast-can-reduce-your-cloud-bills-fc9e09ca1f88?source=rss-ef35bc517478------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc9e09ca1f88</guid>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Amrut Pagidipally]]></dc:creator>
            <pubDate>Tue, 11 Mar 2025 04:37:59 GMT</pubDate>
            <atom:updated>2025-03-11T19:26:15.493Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*YhJJoMQvdBQh9jHnMmcQkg.png" /></figure><p><strong>Introduction</strong> Cloud costs are one of the biggest challenges for businesses today. Over-provisioning, unused resources, and unexpected cost spikes lead to bloated AWS bills. While traditional cost optimization methods rely on manual intervention, AI-powered solutions like AWS Forecast offer a smarter way to predict and optimize cloud spending.</p><p>This article explores how AWS Forecast can be used to reduce cloud costs, with a practical example and implementation using Terraform and Python.</p><h3>Why Cloud Costs Are Difficult to Control</h3><h3>Common Cloud Cost Challenges</h3><ol><li><strong>Over-Provisioning:</strong> Many companies allocate more resources than needed to prevent performance issues, leading to waste.</li><li><strong>Idle Resources:</strong> Developers and engineers often forget to shut down unused instances or services.</li><li><strong>Unexpected Cost Spikes:</strong> A misconfigured service or unexpected traffic surge can cause unpredictable cost increases.</li><li><strong>Complex Pricing Models:</strong> Cloud providers offer a variety of pricing options, making it difficult to select the most cost-effective strategy.</li></ol><h3>Why Traditional Cost Optimization Isn’t Enough</h3><p>Traditional cost optimization strategies include:</p><ul><li><strong>Manual cost monitoring</strong> with AWS Cost Explorer</li><li><strong>Basic auto-scaling</strong> that reacts after demand spikes</li><li><strong>Scheduled shutdowns</strong> for unused resources</li></ul><p>However, these approaches <strong>lack intelligence</strong> — they react to inefficiencies rather than proactively preventing them.</p><h3>How AWS Forecast Uses AI for Cost Optimization</h3><h3>What is AWS Forecast?</h3><p>AWS Forecast is an AI-powered service that uses machine learning to analyze historical data and make accurate future predictions. Unlike traditional machine learning models, which often require manual tuning and data preprocessing, AWS Forecast automates feature engineering, model selection, and hyperparameter optimization. It leverages deep learning algorithms, such as DeepAR+, to handle time-series forecasting more effectively, even with sparse or irregular data patterns. This allows businesses to generate highly accurate cost predictions with minimal ML expertise. Unlike basic monitoring tools, it helps businesses:</p><ul><li>Predict demand for cloud resources.</li><li>Automate cost optimization decisions.</li><li>Prevent over-provisioning by scaling resources dynamically.</li></ul><h3>Step-by-Step Guide: Using AWS Forecast to Optimize Cloud Costs</h3><h3>Step 1: Enable AWS Cost and Usage Reports (CUR)</h3><p>To forecast cloud costs, we need historical usage data. AWS CUR provides a detailed breakdown of cloud spending.</p><p><strong>Terraform to Enable CUR:</strong></p><pre>resource &quot;aws_cur_report_definition&quot; &quot;cost_report&quot; {<br>  report_name            = &quot;AICloudCostReport&quot;<br>  s3_bucket              = &quot;my-cost-report-bucket&quot;<br>  s3_region              = &quot;us-east-1&quot;<br>  time_unit              = &quot;HOURLY&quot;<br>  format                 = &quot;Parquet&quot;<br>  compression            = &quot;GZIP&quot;<br>  additional_schema_elements = [&quot;RESOURCES&quot;]<br>}</pre><p>This ensures <strong>hourly cost data</strong> is stored in S3, which AWS Forecast can use.</p><h3>Step 2: Create an AWS Forecast Dataset</h3><p>Now, we create a <strong>time-series dataset</strong> to train AWS Forecast on cloud cost trends.</p><p><strong>Terraform to Create AWS Forecast Dataset:</strong></p><pre>resource &quot;aws_forecast_dataset&quot; &quot;cloud_cost_forecast&quot; {<br>  dataset_name = &quot;CloudCostForecastDataset&quot;<br>  dataset_type = &quot;TARGET_TIME_SERIES&quot;<br>  domain       = &quot;CUSTOM&quot;<br>  data_frequency = &quot;D&quot;<br>  schema = jsonencode({<br>    Attributes = [<br>      { AttributeName = &quot;timestamp&quot;, AttributeType = &quot;timestamp&quot; },<br>      { AttributeName = &quot;cost&quot;, AttributeType = &quot;float&quot; }<br>    ]<br>  })<br>}</pre><h3>Step 3: Train AI to Predict Cloud Costs</h3><p>Once the dataset is ready, we train AWS Forecast to analyze spending patterns.</p><p><strong>Terraform to Train AWS Forecast:</strong></p><pre>resource &quot;aws_forecast_predictor&quot; &quot;cost_predictor&quot; {<br>  predictor_name = &quot;CloudCostPredictor&quot;<br>  forecast_horizon = 30  # Predict costs for the next 30 days<br>  algorithm = &quot;Deep_AR_Plus&quot;<br>  perform_auto_ml = true<br>  dataset_group_arn = aws_forecast_dataset.cloud_cost_forecast.arn<br>}</pre><p>This <strong>trains an AI model</strong> to predict AWS costs for the next 30 days using DeepAR+, an advanced forecasting algorithm.</p><h3>Step 4: Generating AI-Based Cost Forecasts</h3><p>Once trained, the AI model can generate cost forecasts:</p><p><strong>Python Script to Query AWS Forecast:</strong></p><pre>import boto3<br>forecast = boto3.client(&#39;forecastquery&#39;)<br>response = forecast.query_forecast(<br>    ForecastArn=&#39;your-forecast-arn&#39;,<br>    Filters={&quot;item_id&quot;: &quot;AWS_Cloud_Cost&quot;}<br>)<br>print(response)</pre><p>This retrieves <strong>predicted AWS spending trends</strong> for the next month.</p><h3>Step 5: Automating Cost Optimization Actions</h3><p>AWS Forecast predictions can be used to <strong>automate cost-saving decisions</strong>. For example, More Retail, an Indian grocery retailer, used AWS Forecast to improve demand forecasting, reducing waste by 20% in fresh produce. By integrating AI-driven predictions into their ordering system, they optimized inventory management, leading to significant cost savings. (<a href="https://aws.amazon.com/forecast/customers/">Source</a>).</p><p><strong>Terraform for AI-Based Auto-Scaling:</strong></p><pre>resource &quot;aws_appautoscaling_policy&quot; &quot;ai_scaling&quot; {<br>  name               = &quot;AIScalingPolicy&quot;<br>  service_namespace  = &quot;ec2&quot;<br>  scalable_dimension = &quot;ec2:spot-fleet-request:TargetCapacity&quot;<br>  policy_type        = &quot;TargetTrackingScaling&quot;target_tracking_scaling_policy_configuration {<br>    predefined_metric_specification {<br>      predefined_metric_type = &quot;ASGAverageCPUUtilization&quot;<br>    }<br>    target_value = aws_forecast_predictor.cost_predictor.forecast_horizon<br>  }<br>}</pre><p>This <strong>scales EC2 instances proactively</strong> based on AI predictions, preventing over-provisioning and reducing costs.</p><h3>Real-World Cost Savings with AWS Forecast</h3><pre>| **Before AI (Reactive Scaling)**                      | **After AI (Predictive Scaling)**                       | **Savings (%)** | **Source**                                                              |<br>|-------------------------------------------------------|---------------------------------------------------------|-------------------------------------------------------------------------------------------|<br>| Over-provisioned EC2 instances ($10,000/month)        | AI rightsized EC2 instances ($7,000/month)              | 30%             | [Meesho Case Study](https://aws.amazon.com/forecast/customers/)         |<br>| Auto-Scaling adds 50 instances at peak ($5,000/month) | AI predicts peak, adds only 30 instances ($3,000/month) | 40%             | [More Retail Case Study](https://aws.amazon.com/forecast/customers/)    |<br>| Unused services running ($2,000/month)                | AI anomaly detection shuts them down ($0/month)         | 100%            | [AWS Forecast Documentation](https://aws.amazon.com/forecast/)          |<br>| **Total Monthly Cost: $17,000**                       | **Total Monthly Cost: $10,000**                         | **41% Savings** | [AWS Forecast Case Studies](https://aws.amazon.com/forecast/customers/) |</pre><h3>Conclusion: AI-Driven Cloud Cost Optimization is the Future</h3><p>✅ AWS Forecast provides <strong>AI-driven cost prediction and optimization</strong>. ✅ AI-based scaling eliminates <strong>wasteful over-provisioning</strong>. ✅ Businesses can save <strong>20–40%</strong> on cloud bills using predictive analytics.</p><p>Integrate <strong>AWS Forecast + Terraform</strong> today and <strong>reduce your AWS costs with AI</strong>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc9e09ca1f88" width="1" height="1" alt=""><hr><p><a href="https://awstip.com/ai-powered-cloud-cost-optimization-how-aws-forecast-can-reduce-your-cloud-bills-fc9e09ca1f88">AI-Powered Cloud Cost Optimization: How AWS Forecast Can Reduce Your Cloud Bills</a> was originally published in <a href="https://awstip.com/">AWS Tip</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>