<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bhuvaneswari Subramani on Medium]]></title>
        <description><![CDATA[Stories by Bhuvaneswari Subramani on Medium]]></description>
        <link>https://medium.com/@bhuvaneswari.subramani?source=rss-cbbf9d3173c6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*kWxi8GKhcogdLB_7ioY5ZQ.jpeg</url>
            <title>Stories by Bhuvaneswari Subramani on Medium</title>
            <link>https://medium.com/@bhuvaneswari.subramani?source=rss-cbbf9d3173c6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 01:33:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bhuvaneswari.subramani/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why RAG Loses the Half of Your Sentence That Mattered]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/why-rag-loses-the-half-of-your-sentence-that-mattered-ba50fd39f44c?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba50fd39f44c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Mon, 07 Sep 2026 12:16:00 GMT</pubDate>
            <atom:updated>2026-09-07T12:16:00.843Z</atom:updated>
            <content:encoded><![CDATA[<h3>Chunking protects sentences. Nobody ever claimed it protects meaning.</h3><p><em>The Retrieval Problem · Part 2 of 6</em></p><p>By Bhuvaneswari Subramani · Executive Technology Leader</p><p><a href="https://medium.com/@bhuvaneswari.subramani/rag-doesnt-retrieve-what-you-think-it-retrieves-22c380a2334b"><em>← Part 1: Why Your RAG System Confidently Gives You the Wrong Answer</em></a></p><p>In <a href="https://medium.com/@bhuvaneswari.subramani/rag-doesnt-retrieve-what-you-think-it-retrieves-22c380a2334b">Part 1</a> the bread came out flat, because the system searched for chunks about <em>bread</em> while the answer lived in a chunk about <em>yeast</em>.</p><p>This time the bread comes out wrong for a different reason, and it’s one you can’t fix by asking a better question.</p><p>I asked what temperature to bake the white loaf at. The system told me 220°C. It was quoting the cookbook exactly. The loaf came out dark on the outside and underdone in the middle, because I have a fan oven — and the sentence immediately after the one it quoted said to reduce every temperature in the book by 20°C.</p><p>That sentence was in the cookbook. It was one line further down the same page. And it may as well have been in a different building.</p><h3>What chunking is, and why it has to happen</h3><p>Part 1 introduced the word: before anything can be searched, documents are cut into fragments of a few hundred words called <strong>chunks</strong>.</p><p>This isn’t laziness. There are three real reasons for it. Embedding models have a size limit on what you can feed them. Smaller pieces make retrieval more precise — a whole book compressed into one vector means nothing. And you have to fit what comes back into the model’s reading window.</p><p>So chunking is unavoidable. The question is only <em>where the cuts fall.</em></p><p>The default answer is: roughly every N tokens. Amazon Bedrock Knowledge Bases splits into chunks of about 300 tokens by default, and it does honour sentence boundaries — complete sentences stay intact. Most other tools behave similarly.</p><p>Read that again, because it’s the whole problem. <strong>The splitter protects sentences.</strong> Nobody ever claimed it protects meaning.</p><h3>The cut</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Av_Un7S8-92yckvAY21tcQ.png" /></figure><p>Chunk 47 is a complete sentence. It’s grammatical, it’s quoted accurately, and it’s the correct temperature for a conventional oven.</p><p>Chunk 48 holds the condition that makes chunk 47 wrong for me.</p><p>And here’s the part that connects back to Part 1: chunk 48 was never going to be retrieved. Read it on its own and it’s about <em>fan ovens</em>. It doesn’t mention the white loaf, or baking bread, or the temperature I asked about. It’s a different subject — the same failure we saw with the yeast page, except this time we <em>created</em> the different subject by cutting the page in half.</p><p>The chunking didn’t just hide the information. It manufactured a new orphan chunk that no sensible question will ever match.</p><h3>Three relationships that don’t survive the cut</h3><p>The fan oven case is one instance of a general pattern. Technical writing is built out of relationships between sentences, and chunking severs all three kinds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kajdeRGIcwG5U7RRybKpCA.png" /></figure><p><strong>The condition.</strong> A rule and its exception are usually adjacent, not combined. “Bake at 220°C” and “unless using a fan oven” are two sentences. Retrieve one without the other and you have a confident, complete, wrong answer.</p><p><strong>The referent.</strong> Technical prose is full of pronouns. <em>“It must not exceed 5 A.”</em> What is “it”? Named three paragraphs earlier, in a chunk you didn’t retrieve. On its own this sentence is unusable, but it will still embed cleanly and still be returned as a result.</p><p><strong>The cross-reference.</strong> <em>“Follow the method described in Section 4.2.”</em> Section 4.2 is forty chunks away and shares no vocabulary with your question. This is the one that makes standards documents, API references and compliance manuals so unreliable under RAG — those documents are <em>mostly</em> cross-reference. Cut the references and you have a pile of fragments that individually say very little.</p><p>Prose survives chunking reasonably well, because prose repeats itself and carries its context along. Reference documentation doesn’t repeat itself — that’s what makes it good documentation, and it’s exactly what makes it fragile here.</p><h3>Why overlap doesn’t rescue you</h3><p>The standard answer is overlap: let each chunk include the last 20% of the one before it, so nothing falls through the crack. Bedrock’s fixed-size option defaults to 20%.</p><p>It helps at the seams, and it’s worth having. But look at what it actually buys you. Twenty percent of 300 tokens is about sixty tokens — two or three sentences of margin. That catches a condition sitting immediately next to its rule. It does nothing at all for a cross-reference forty chunks away, and nothing for a referent introduced at the top of a section.</p><p>Overlap widens the crack you can survive. It doesn’t change the fact that you’re cutting.</p><h3>What this costs, measured</h3><p>Anthropic ran an evaluation on exactly this problem, across codebases, research papers and fiction, measuring how often the chunk containing the answer failed to appear in the top twenty results.</p><p>The baseline number is 5.7%. That’s roughly one question in eighteen where the right chunk wasn’t in the top <strong>twenty</strong> — not the top five most systems actually use.</p><p>Then they tried something almost embarrassingly simple: before embedding each chunk, prepend a line of context explaining where it sits in the document.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eyw26v9RjZNriyEvpWA0EQ.png" /></figure><p>Adding that context cut retrieval failures by 35%. Adding keyword search alongside took it to 49%. Adding a reranking pass reached 67% — from 5.7% down to 1.9%.</p><p>Two-thirds fewer missed retrievals, and the embedding model never changed. What changed was that each chunk now says what it’s about.</p><p>Apply that to our cookbook and chunk 48 stops being an orphan. Instead of <em>“reduce all temperatures by 20°C”</em> floating free, it reads <em>“From the Fan Oven Adjustments section of Baking Basics; applies to every bread recipe in this book — reduce all temperatures by 20°C.”</em> Now it has something to match against.</p><h3>What I’d do if this were my system</h3><p><strong>Stop using the default splitter on structured documents.</strong> Both major clouds offer better options now — Bedrock has hierarchical chunking, which retrieves a small precise chunk and then hands the model its larger parent, and semantic chunking, which cuts at topic shifts rather than token counts. Neither is the default. Both are one configuration change away.</p><p><strong>Add context to your chunks before you tune anything else.</strong> The 35% figure comes from a technique you can implement in an afternoon, and it doesn’t require replacing your embedding model, your vector store, or your prompt. In a list of possible interventions, this has the best ratio of benefit to disruption that I’m aware of.</p><p><strong>Then accept the tradeoff honestly.</strong> Contextual chunks cost more to build — you’re running a model over every chunk at ingest — and hierarchical chunking returns more text per query. You’re paying tokens to buy accuracy. That’s a reasonable trade for a compliance system and a poor one for a low-stakes internal search, and the decision belongs to whoever owns the budget, not to whoever owns the pipeline.</p><p><strong>And test at the seams.</strong> Write questions whose answers straddle a chunk boundary — a rule and its exception, a value and its unit, a step and its precondition. Standard evaluation sets never contain these, because whoever wrote them was reading the document whole.</p><h3>Coming next</h3><p><a href="https://medium.com/@bhuvaneswari.subramani/rag-doesnt-retrieve-what-you-think-it-retrieves-22c380a2334b">Part 1</a> was about the search not understanding your question. Part 2 was about the documents being broken before the search ever ran.</p><p><strong>Part 3 — Your Demo Lied to You.</strong> The pilot worked on fifty documents. Production has twelve thousand. Nothing broke — you’re just further along a curve that was always there, and there’s a benchmark that shows exactly how steep it is.</p><p><em>If you’ve hit the chunk-boundary problem in your own systems, I’d like to hear how it surfaced. It rarely announces itself.</em></p><p><strong>Sources:</strong> Anthropic, Contextual Retrieval (2024) · Barnett et al., “Seven Failure Points When Engineering a Retrieval Augmented Generation System,” CAIN 2024, arXiv:2401.05856 · Amazon Bedrock Knowledge Bases documentation</p><p><strong>#RAG #AIEngineering #VectorSearch #MachineLearning #LLM</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba50fd39f44c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RAG Doesn’t Retrieve What You Think It Retrieves]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/rag-doesnt-retrieve-what-you-think-it-retrieves-22c380a2334b?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/22c380a2334b</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Mon, 07 Sep 2026 10:03:38 GMT</pubDate>
            <atom:updated>2026-09-07T12:52:44.241Z</atom:updated>
            <content:encoded><![CDATA[<h4>What cosine similarity actually measures — and why that quietly breaks on the questions that matter most</h4><p><em>The Retrieval Problem · Part 1 of 6<br>One cookbook. Six reasons RAG fails on questions that matter.</em></p><p>By Bhuvaneswari Subramani · Executive Technology Leader</p><h4>Imagine you are baking bread. It comes out flat and dense. So you ask an AI assistant that has read your entire cookbook:</h4><blockquote>”Why did my bread come out flat?”</blockquote><p>It answers immediately and confidently. It talks about kneading technique. It suggests checking your oven temperature. Everything it says is about bread, and none of it is right.</p><p>The cookbook did have the answer. It was on a page titled <em>”Activating yeast”</em>, and it said: <em>water hotter than about 50°C kills yeast.</em> My water was too hot. Dead yeast, no rise, flat loaf.</p><p>That page was never even looked at. Here is why — and why the same thing is happening inside a great many company AI systems right now.</p><h4>First, how these systems actually work</h4><p>The technique is called <strong>RAG — Retrieval-Augmented Generation</strong>. The idea is sensible: an AI can’t memorise your documents, so instead you fetch the relevant bits and let it read them before answering.</p><p>One detail matters later, so I’ll name it now. The system doesn’t store whole documents. It cuts them into fragments of a few hundred words each — these are called <strong><em>chunks</em></strong> — and searches those. In our cookbook, think of a chunk as roughly a page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OL9yACvWp0T_FHpychF00A.png" /></figure><p>Think of it as a librarian and an expert. You hand your question to a librarian. The librarian skims the shelves, grabs a handful of pages that look related, and slides them under the door to an expert locked in a room. The expert must answer using only those pages, and is not allowed to ask for anything else.</p><p>How many pages? Five, if you never change the setting. That’s the default in Amazon Bedrock Knowledge Bases and also in Azure OpenAI On Your Data; LangChain uses four. You can raise it. Hold that thought, because it’s the first thing everyone reaches for.</p><p>The setting has a different name everywhere. It’s numberOfResults in Bedrock, topNDocuments in Azure OpenAI On Your Data, k in LangChain and in Azure AI Search directly. Same idea in all of them: how many fragments to hand the model.</p><p>When the librarian picks well, the expert looks brilliant.</p><p>When the librarian picks badly, the expert doesn’t say <em>”these are the wrong pages.”</em> The expert answers anyway — fluently, confidently, and wrongly.</p><h4>How the librarian chooses</h4><p>This is the part worth understanding, because everything else follows from it.</p><p>The system converts each chunk into a list of numbers representing <em>what that chunk is about</em>. It does the same to your question. Then it returns the chunks whose numbers sit closest to your question’s numbers.</p><p>Both of those have names you’ll meet constantly, so worth having them: the list of numbers is an <strong>embedding</strong>, and the measure of closeness is <strong>cosine similarity</strong>. You don’t need the maths behind either. The whole idea is “how close in meaning are these two things?”</p><h4>What that actually looks like</h4><p>It’s worth seeing the mechanism, because the flaw is visible in it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ARtO-dviZkMiSLJKTcm7DA.png" /></figure><p>Every chunk becomes roughly a <strong>direction</strong>. Similarity is the angle between two directions — a small angle means similar, a wide angle means unrelated. The formula divides out the length of both, which sounds like a technical detail and isn’t. It means the <em>only</em> thing that survives is which way a piece of text points. How strongly, how specifically, or how emphatically something is stated is thrown away.</p><p>That has a consequence you can demonstrate in about a minute:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_6YPN_azzgs5hM_AX0EK3g.png" /></figure><p>Each pair differs by a single word, and in each pair that word is the entire instruction. To the search, the two sentences are nearly identical. If it retrieves the wrong one, your answer isn’t vague — it’s confidently backwards.</p><p>Run this on your own documents before you trust the system. Take two sentences that differ only by a negation, a version number, or a threshold. Embed both. Look at the score. It is a more convincing test than any benchmark, because it’s your corpus.</p><p>It’s important to be clear about what this does well. It is <strong>not</strong> keyword matching. It compares meanings, so it handles synonyms comfortably — ask about a “car” and it will happily find text about an “automobile.” People often assume the fix here is better wording. It isn’t.</p><p>What it measures is: <strong>is this chunk about the same thing as the question?</strong></p><p>What it does not measure is: <strong>does this chunk contain the answer?</strong></p><p>Most of the time those overlap, so nobody notices. Ask <em>“when was the Eiffel Tower built”</em> and the answer sits in a chunk about the Eiffel Tower. Easy.</p><p>Now look at the bread.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*whQdAdXIwepTdwycw6KQgw.png" /></figure><p>Your question is about <strong>bread</strong>. The page that answers it is about <strong>yeast</strong>. Those are genuinely different subjects — not different words for the same subject. No amount of clever phrasing bridges them, because the connection between them isn’t a matter of vocabulary. It’s a chain of cause and effect:</p><p><strong>water too hot → yeast dies → no gas → no rise → flat loaf</strong></p><p>A baker crosses that chain instantly. The search cannot cross it at all, because crossing it requires reasoning, and the search is only measuring closeness.</p><p>And here’s what makes this hard to escape: <strong>you asked because you didn’t know the cause.</strong> If you had known to look up yeast temperatures, you wouldn’t have needed to ask.</p><p>The system works beautifully for questions you could have answered yourself.</p><h4>So just fetch more pages</h4><p>This is the obvious fix, and it’s worth understanding why it only half works.</p><p>Raise the setting from five to fifty and yes, the yeast page might now scrape in. But you haven’t changed the ranking — that page is still buried, because it’s still about a different subject. You’ve widened the net and hoped.</p><p>You also pay for it: more text means higher cost, slower answers, and more nearly-relevant material cluttering the AI’s reading. Which, as it turns out, actively makes things worse.</p><h4>And that’s before we get to tables</h4><p>There’s a second loss happening earlier, at the point where documents go in.</p><p>Before anything can be searched, the document is converted to plain text. Prose survives that. Structure often does not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iMzCaOWgsdX1y7PuRriMGQ.png" /></figure><p>Ask “how long does rye take?” and there is nothing in the stored text tying 45 minutes to rye rather than to sourdough. The columns carried that relationship, and the columns are gone.</p><p>A cookbook shrugs this off. A specification sheet, a pricing table, a compatibility matrix or a configuration reference does not — those are documents where the meaning lives in the layout rather than in the sentences.</p><p>Both major clouds now offer better parsing for this — Bedrock has a foundation-model parser, Azure has Document Intelligence layout extraction. They genuinely help. They do not make the problem disappear, and they are not what you get by default.</p><h4>This has been measured</h4><p>Researchers built a test called BRIGHT for questions where you have to reason your way to the right document rather than simply match subject matter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9dvRqsjuuQTMTNYWZhNJzg.png" /></figure><p>The same retrieval models score 59 out of 100 on ordinary lookup questions and 18 on questions that need reasoning. Nothing changed but the type of question.</p><p>Nor does spending more help much. Across the thirteen models tested, the best score anyone managed on the hard questions was 24.</p><h4>The part that makes it dangerous</h4><blockquote>If the system responded with ”I couldn’t find this,” we’d be discussing a mild annoyance.</blockquote><p>It doesn’t. Search <strong>ranks</strong> results; it doesn’t reject them. If the answer isn’t in your documents at all, you still get results back, looking exactly like a successful search. There’s no score at which the system announces it has found nothing useful.</p><p>And work presented at ICLR 2025 measured what happens next. When one model was given related material that didn’t actually contain the answer, its wrong answers rose from around 10% to 66% — worse than giving it nothing at all. The models tested also became markedly less willing to say they didn’t know.</p><blockquote>Half-relevant information is worse than no information, because it manufactures confidence.</blockquote><p>A flat loaf is a small price. The same failure inside a compliance check, a medical query, or an architecture decision is not.</p><h4>What to do about it</h4><p>Three things, roughly in order of how much they help:</p><p><strong>Test with real questions, not tidy ones.</strong> Most teams test using questions written by the people who wrote the documents, in the documents’ own vocabulary. Ask a few colleagues who’ve never read the material to write the questions instead. That’s when the cracks appear.</p><p><strong>Measure retrieval separately.</strong> Before asking “was the answer good?”, ask “did the right chunk even come back?” If it didn’t, no amount of tweaking the AI will help — you’d be fixing the wrong end of the system.</p><p><strong>Let the system search more than once.</strong> The real fix is an AI that searches, reads what came back, recognises it isn’t enough, and searches again with better terms — the way a person would. That’s the difference between a fixed pipeline and an agent, and it’s where this field is heading.</p><h4>Coming next</h4><p>This is one of six reasons conventional RAG struggles with deep technical questions. The others: how cutting documents into chunks destroys meaning, why systems that worked in testing fail in production, and what actually fixes it.</p><p><a href="https://medium.com/@bhuvaneswari.subramani/why-rag-loses-the-half-of-your-sentence-that-mattered-ba50fd39f44c"><strong>Part 2 — </strong>Why RAG Loses the Half of Your Sentence That Mattered</a><strong>.</strong> The recipe said 220°C. The next page said reduce by 20°C for fan ovens. The system retrieved only one of them.</p><p><strong><em>If you’ve hit this in your own work, I’d like to hear about it — it’s shaping a talk I’m preparing.</em></strong></p><p><strong>Sources:</strong> BRIGHT benchmark (arXiv:2407.12883) · Sufficient Context, ICLR 2025 (arXiv:2411.06037) · Amazon Bedrock Knowledge Bases documentation</p><p><strong>#AI #RAG #MachineLearning #ArtificialIntelligence #AWS #Azure</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22c380a2334b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MongoDB for Enterprise AI — A Practitioner’s Deep Dive from Data to Production]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/mongodb-for-enterprise-ai-a-practitioners-deep-dive-from-data-to-production-52fcd5f73664?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/52fcd5f73664</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[finance]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 15:21:01 GMT</pubDate>
            <atom:updated>2026-07-19T13:49:00.897Z</atom:updated>
            <content:encoded><![CDATA[<p>Schema design, aggregation pipelines, Vector Search, cluster sizing, and sharding — built on real BFSI data, not theory.</p><p><em>By Bhuvaneswari Subramani — AWS Hero · Author, The AI-DLC Handbook</em></p><p><em>This is Part 2 of a hands-on MongoDB series. </em><a href="https://medium.com/p/5d2606ccf647"><em>Part 1</em></a><em> — Diagnosing Failed AI Pilots with MongoDB Atlas Vector Search and Amazon Bedrock Nova Pro. GitHub — </em><a href="https://github.com/bhuvana-s/financial-intelligence-platform-"><em>https://github.com/bhuvana-s/financial-intelligence-platform-</em></a></p><h3>Section 1 — Why This Series Exists</h3><p>Most MongoDB tutorials stop at CRUD.</p><p>They show you how to insert a document, run a find query, and call it a day. What they don’t show you is what happens when you’re sitting across from a BFSI enterprise architect who wants to know how you’d shard a 10-million-customer dataset — or a healthcare CTO asking why your aggregation pipeline is running slow at scale — or a developer lead asking whether to embed or reference a transaction history that could grow to millions of records per customer.</p><p>Those are the conversations that matter in production. And the only way to be ready for them is to have built through the hard parts yourself — not read about them.</p><p>This series is built for two audiences simultaneously:</p><ul><li><strong>For developers </strong>— you’ll get working code, real aggregation pipelines, explain() outputs, Vector Search indexes, and schema decisions you can take directly into your own projects. Every concept is grounded in a running application, not a contrived textbook example.</li><li><strong>For enterprise architect</strong>s — you’ll get the reasoning layer: why each schema decision was made, what the trade-offs are at scale, how cluster sizing math works in practice, and when sharding becomes necessary versus premature. The kind of thinking that belongs in an architecture review, not a getting-started guide.</li></ul><p>What we’re building in this series:</p><p><strong>Part 1</strong> built the Enterprise AI Pilot Rescue Kit — a RAG application that diagnoses why enterprise AI pilots fail, using MongoDB Atlas Vector Search and Amazon Bedrock Nova Pro as the reasoning engine. It covered document modeling, Vector Search indexes, and the $vectorSearch aggregation pipeline.</p><p><strong>Part 2</strong> builds the Enterprise Financial Intelligence Platform — a customer financial health scoring system on real BFSI-shaped data, designed to go deep on MongoDB’s most advanced capabilities: multi-stage aggregation pipelines with $setWindowFields, compound indexing strategy, Atlas Search for full-text retrieval, and the cluster sizing and sharding decisions that separate a proof-of-concept from a production architecture.</p><p>The GitHub repositories:</p><p>Part 1 — github.com/bhuvana-s/pilot-2-rescue <br>Part 2 — github.com/bhuvana-s/financial-intelligence-platform-</p><p>Both are open source. Fork them, break them, rebuild them. That’s the point.</p><p>One principle that runs through everything here:</p><blockquote>The fastest way to understand a platform’s architecture is to make real decisions on real data under real constraints. Not read the documentation. Not watch a tutorial. Build something that would embarrass you if it broke in a demo — and then make sure it doesn’t.</blockquote><p>That’s the standard I hold myself to. It’s the standard I think enterprise builders deserve.</p><h3>Section 2 — Loading Real Data into MongoDB Atlas</h3><h4>Choosing the right dataset</h4><p>The first decision in any hands-on deep dive is the data. Generic sample data produces generic insights. I wanted BFSI-shaped data — customers, accounts, transactions — because that’s the domain where MongoDB’s document model creates the most interesting architectural decisions, and where cluster sizing and sharding choices have the most real-world consequence.</p><p>MongoDB’s own sample_analytics dataset fits exactly. It contains:</p><ul><li>customers — 500 customer profiles with account references and tier details</li><li>accounts — 1,746 account documents with product arrays and credit limits</li><li>transactions — 1,746 transaction bucket documents with nested transaction arrays</li></ul><p>The transaction bucket pattern — multiple transactions stored per document rather than one document per transaction — is itself a deliberate MongoDB schema decision worth understanding. We’ll go deep on it in the schema section. First, let’s talk about how to actually get this data into Atlas.</p><h4>The M0 limitation…</h4><p>Before attempting any data loading, I tried the obvious path: Atlas UI → cluster → Load Sample Dataset.</p><p>On my M0 cluster, the Load Sample Dataset option wasn’t available in the UI — which led me to the command line route documented below. If you’re in the same situation, here’s what actually worked.</p><p>Everything that follows in this series runs on M0 free tier, which matters: the schema design, aggregation pipelines, Atlas Search, Atlas Vector Search, and indexing concepts are identical regardless of cluster tier. Tier affects performance and scale ceiling — not the MongoDB concepts themselves.</p><h4>Attempt 1 — mongorestore from the Atlas S3 archive</h4><p>The officially documented path:</p><pre>curl -O https://atlas-education.s3.amazonaws.com/sampledata.archive<br>mongorestore \<br>  --uri=&quot;mongodb+srv://username:password@cluster.mongodb.net&quot; \<br>  --archive=sampledata.archive \<br>  --gzip</pre><p>In practice: the archive downloaded as a corrupt file. Every restore attempt returned gzip: invalid header.</p><p>Rather than continuing to debug the archive restore path, I switched to a direct Python import from a public GitHub mirror of the same dataset — which worked cleanly. If you hit the same gzip error, the Python import path below is the reliable alternative.</p><p>One more gotcha with mongorestore on Mac: The tool is not installed by default. You need MongoDB Database Tools separately:</p><pre>brew tap mongodb/brew<br>brew trust mongodb/brew<br>brew install mongodb-database-tools<br># Verify<br>mongorestore --version</pre><h4>Attempt 2 — Python import from GitHub mirror (what actually worked)</h4><p>A reliable public mirror of MongoDB’s sample datasets exists at github.com/neelabalan/mongodb-sample-dataset. Three direct JSON files, downloadable without any archive extraction. No mongorestore required.</p><p>Install the only dependency needed:</p><pre>pip3 install pymongo --break-system-packages</pre><p>The first attempt at a straightforward JSON load fails immediately:</p><pre>json.decoder.JSONDecodeError: Extra data: line 2 column 1</pre><p>Why: The files use NDJSON format (one JSON object per line, not a JSON array). The standard json.loads() call expects an array and fails on the second line.</p><p>Fix: Read line by line:</p><pre>def load_ndjson(url):<br>    with urllib.request.urlopen(url) as r:<br>        content = r.read().decode().strip()<br>    if content.startswith(&#39;[&#39;):<br>        return json.loads(content)<br>    return [json.loads(line) for line in content.splitlines() if line.strip()]</pre><p>The second failure is more interesting:</p><pre>pymongo.errors.BulkWriteError: _id fields may not contain &#39;$&#39;-prefixed fields:<br>$oid is not valid for storage.</pre><p>Why: MongoDB’s sample datasets use Extended JSON format — every field is wrapped in a type descriptor:</p><pre>{<br>  &quot;_id&quot;:       { &quot;$oid&quot;: &quot;5ca4bbcea2dd94ee58162a68&quot; },<br>  &quot;birthdate&quot;: { &quot;$date&quot;: { &quot;$numberLong&quot;: &quot;226117231000&quot; } },<br>  &quot;accounts&quot;:  [{ &quot;$numberInt&quot;: &quot;371138&quot; }]<br>}</pre><p>Extended JSON is MongoDB’s wire format for representing BSON types in JSON. PyMongo’s insert_many expects native Python types, not Extended JSON descriptors. Passing Extended JSON directly causes the $oid is not valid for storage error.</p><p>Fix — a recursive type converter:</p><pre>from bson import ObjectId, Decimal128<br>from datetime import datetime, timezone</pre><pre>def convert(obj):<br>    &quot;&quot;&quot;Recursively convert Extended JSON to native Python/BSON types.&quot;&quot;&quot;<br>    if isinstance(obj, dict):<br>        if &quot;$oid&quot; in obj:<br>            return ObjectId(obj[&quot;$oid&quot;])<br>        if &quot;$date&quot; in obj:<br>            val = obj[&quot;$date&quot;]<br>            ms = val[&quot;$numberLong&quot;] if isinstance(val, dict) else val<br>            return datetime.fromtimestamp(int(ms)/1000, tz=timezone.utc)<br>        if &quot;$numberInt&quot; in obj:<br>            return int(obj[&quot;$numberInt&quot;])<br>        if &quot;$numberLong&quot; in obj:<br>            return int(obj[&quot;$numberLong&quot;])<br>        if &quot;$numberDouble&quot; in obj:<br>            return float(obj[&quot;$numberDouble&quot;])<br>        if &quot;$numberDecimal&quot; in obj:<br>            return Decimal128(obj[&quot;$numberDecimal&quot;])<br>        return {k: convert(v) for k, v in obj.items()}<br>    if isinstance(obj, list):<br>        return [convert(i) for i in obj]<br>    return obj</pre><p>This walks every document recursively and converts Extended JSON types to their native Python equivalents before insertion. Apply it to every document before calling insert_many.</p><h4>The complete working import script</h4><pre>import json<br>import urllib.request<br>from pymongo import MongoClient<br>from bson import ObjectId, Decimal128<br>from datetime import datetime, timezone</pre><pre>MONGO_URI = &quot;mongodb+srv://username:password@cluster.mongodb.net/sample_analytics?appName=your-app&quot;</pre><pre>DATASETS = {<br>    &quot;customers&quot;:    &quot;https://raw.githubusercontent.com/neelabalan/mongodb-sample-dataset/main/sample_analytics/customers.json&quot;,<br>    &quot;accounts&quot;:     &quot;https://raw.githubusercontent.com/neelabalan/mongodb-sample-dataset/main/sample_analytics/accounts.json&quot;,<br>    &quot;transactions&quot;: &quot;https://raw.githubusercontent.com/neelabalan/mongodb-sample-dataset/main/sample_analytics/transactions.json&quot;<br>}</pre><pre>def convert(obj):<br>    if isinstance(obj, dict):<br>        if &quot;$oid&quot; in obj:       return ObjectId(obj[&quot;$oid&quot;])<br>        if &quot;$date&quot; in obj:<br>            val = obj[&quot;$date&quot;]<br>            ms = val[&quot;$numberLong&quot;] if isinstance(val, dict) else val<br>            return datetime.fromtimestamp(int(ms)/1000, tz=timezone.utc)<br>        if &quot;$numberInt&quot; in obj:     return int(obj[&quot;$numberInt&quot;])<br>        if &quot;$numberLong&quot; in obj:    return int(obj[&quot;$numberLong&quot;])<br>        if &quot;$numberDouble&quot; in obj:  return float(obj[&quot;$numberDouble&quot;])<br>        if &quot;$numberDecimal&quot; in obj: return Decimal128(obj[&quot;$numberDecimal&quot;])<br>        return {k: convert(v) for k, v in obj.items()}<br>    if isinstance(obj, list):<br>        return [convert(i) for i in obj]<br>    return obj</pre><pre>def load_ndjson(url):<br>    print(f&quot;Downloading {url.split(&#39;/&#39;)[-1]}...&quot;)<br>    with urllib.request.urlopen(url) as r:<br>        content = r.read().decode().strip()<br>    raw = json.loads(content) if content.startswith(&#39;[&#39;) \<br>          else [json.loads(l) for l in content.splitlines() if l.strip()]<br>    return [convert(doc) for doc in raw]</pre><pre>client = MongoClient(MONGO_URI)<br>db = client[&quot;sample_analytics&quot;]</pre><pre>for name, url in DATASETS.items():<br>    data = load_ndjson(url)<br>    db[name].drop()<br>    result = db[name].insert_many(data)<br>    print(f&quot;✓ {name}: {len(result.inserted_ids)} documents inserted&quot;)</pre><pre>print(&quot;\nDone. sample_analytics is live in Atlas.&quot;)<br>client.close()</pre><p>Result:</p><pre>✓ customers:      500 documents inserted<br>✓ accounts:      1746 documents inserted<br>✓ transactions:  1746 documents inserted</pre><pre>Done. sample_analytics is live in Atlas.</pre><h4>What the data looks like — the schema that matters</h4><p>Three collections, each with a clear purpose and a deliberate design decision behind how they relate to each other.</p><p>customers — the anchor document</p><pre>{<br>  &quot;username&quot;: &quot;fmiller&quot;,<br>  &quot;name&quot;: &quot;Elizabeth Ray&quot;,<br>  &quot;email&quot;: &quot;arroyocolton@gmail.com&quot;,<br>  &quot;active&quot;: true,<br>  &quot;accounts&quot;: [371138, 324287, 276528, 332179],<br>  &quot;tier_and_details&quot;: {<br>    &quot;0df078f3...&quot;: {<br>      &quot;tier&quot;: &quot;Bronze&quot;,<br>      &quot;benefits&quot;: [&quot;sports tickets&quot;],<br>      &quot;active&quot;: true<br>    }<br>  }<br>}</pre><p>The accounts field is an array of integers — account numbers, not ObjectIds. This is a business key reference pattern: accounts are stored in their own collection and linked by a human-readable key, not a MongoDB-generated identifier. This makes the data portable across environments and keeps the join intent readable.</p><p>The tier_and_details field is a nested map keyed by a UUID string — an interesting schema choice that we&#39;ll analyse when we get to the schema design section.</p><p>accounts — the financial profile</p><pre>{<br>  &quot;account_id&quot;: 371138,<br>  &quot;limit&quot;: 9000,<br>  &quot;products&quot;: [&quot;Brokerage&quot;, &quot;InvestmentStock&quot;, &quot;CurrencyService&quot;]<br>}</pre><p>Clean and lean. The products array is the diversity signal — how many distinct financial products a customer holds across their accounts.</p><p>transactions — the activity layer (bucket pattern)</p><pre>{<br>  &quot;account_id&quot;: 371138,<br>  &quot;transaction_count&quot;: 36,<br>  &quot;bucket_start_date&quot;: &quot;2016-07-23T00:00:00Z&quot;,<br>  &quot;bucket_end_date&quot;: &quot;2017-01-05T00:00:00Z&quot;,<br>  &quot;transactions&quot;: [<br>    {<br>      &quot;date&quot;: &quot;2016-10-27T00:00:00Z&quot;,<br>      &quot;amount&quot;: 5024,<br>      &quot;transaction_code&quot;: &quot;buy&quot;,<br>      &quot;symbol&quot;: &quot;adbe&quot;,<br>      &quot;price&quot;: &quot;153.27...&quot;,<br>      &quot;total&quot;: &quot;769943.88...&quot;<br>    }<br>  ]<br>}</pre><p>This is the most architecturally interesting collection. Each document is a transaction bucket — a container for multiple transactions belonging to one account over a time window, rather than one document per transaction. This is a deliberate MongoDB schema pattern for high-volume time-series data. We’ll build the aggregation pipeline that joins all three collections in the next section.</p><h4>Summary — the data loading lessons</h4><p>The platform works across 4 collections: three loaded from sample_analytics (customers, accounts, transactions) and one created by the application itself (financialHealthLogs) to store scored sessions and embeddings for vector sear</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qq8D0CFvv3NDnxj7z3u4FQ.png" /></figure><p>None of these are hard problems once you know what they are. The documentation doesn’t surface them clearly because the happy path assumes a clean environment. Real environments aren’t clean.</p><h3>Section 3 — Schema Design: What the Data Actually Looks Like Before You Write a Line of Code</h3><p>Here is the discipline that separates architects from script writers: you don’t design a pipeline against a schema description. You interrogate the live data first.</p><p>Before a single aggregation stage was written for this application, I ran a set of reconnaissance queries against the live Atlas cluster. What came back reshaped five fundamental design decisions. If I had skipped this step and built against the documented spec, the application would have produced wrong answers silently — no errors, just numbers that mean nothing.</p><p>Here are the five things the data told me that the schema description did not.</p><h4>Finding 1 — The transaction dates end in 2017</h4><pre>db.transactions.aggregate([<br>  { $unwind: &quot;$transactions&quot; },<br>  { $group: { _id: null, min: { $min: &quot;$transactions.date&quot; }, max: { $max: &quot;$transactions.date&quot; } } }<br>])<br>// → min: 1962-03-08, max: 2017-01-09</pre><p>The specification said “Transaction Volume Score: total transaction amount across all accounts, last 90 days.” Natural reading: 90 days from today.</p><p>If I had built that literally, every customer would have scored zero on Transaction Volume and Transaction Velocity. The dataset ends in January 2017. Queried from 2026, “last 90 days” returns nothing.</p><p>The fix is a configurable REFERENCE_DATE. All time windows anchor to the data&#39;s own maximum date — 2017-01-09 — instead of wall-clock time. A 90-day window from that anchor covers 468 of 500 customers, with an average of 7.4 transactions and average volume of ~$7.9M. A healthy, non-degenerate spread for a real demo.</p><p>This isn’t a workaround. It’s the right design for any system working with historical data — make the time reference an explicit parameter, not an implicit Date.now().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OmTgCZM0PeY1q4ombt65dQ.png" /></figure><h4>Finding 2 — price and total are strings. amount is share count, not dollars.</h4><pre>db.transactions.aggregate([<br>  { $unwind: &quot;$transactions&quot; },<br>  { $limit: 1 },<br>  { $project: { amtType: { $type: &quot;$transactions.amount&quot; }, totalType: { $type: &quot;$transactions.total&quot; } } }<br>])<br>// → amtType: &quot;int&quot;, totalType: &quot;string&quot;</pre><p>Two distinct surprises in one query.</p><p>total — the field that should represent transaction dollar value — is stored as a high-precision decimal string: &quot;769943.4963712...&quot; with 40+ significant digits. This is not an error; it&#39;s a deliberate choice to avoid float rounding on financial calculations. But it means { $sum: &quot;$transactions.total&quot; } in an aggregation returns zero. You need $toDouble first.</p><p>amount is an integer representing share count — how many shares traded, not how much money changed hands. A client who bought 1 share of a $500 stock and a client who bought 1 share of a $5 stock both have amount: 1. For Transaction Volume, you want the dollar value: $toDouble(&quot;$transactions.total&quot;).</p><p>This distinction alone would have invalidated the entire scoring model if missed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q-QUyxhb6nYEOUUy2Q821w.png" /></figure><h4>Finding 3 — tier_and_details is a hash-keyed map, and 267 of 500 customers have it empty</h4><pre>db.customers.countDocuments({ tier_and_details: {} })<br>// → 267 (53% of customers)</pre><p>A sample customer with tier data:</p><pre>&quot;tier_and_details&quot;: {<br>  &quot;0df078f33aa74a2e9696e0520c1a828a&quot;: {<br>    &quot;tier&quot;: &quot;Bronze&quot;,<br>    &quot;benefits&quot;: [&quot;sports tickets&quot;],<br>    &quot;active&quot;: true<br>  },<br>  &quot;699456451cc24f028d2aa99d7534c219&quot;: {<br>    &quot;tier&quot;: &quot;Bronze&quot;,<br>    &quot;benefits&quot;: [&quot;24 hour dedicated line&quot;, &quot;concierge services&quot;],<br>    &quot;active&quot;: true<br>  }<br>}</pre><p>Two Bronze tier entries. Different UUIDs, different benefit sets. Same customer.</p><p>The distribution: 267 customers with empty {}, 80 with one tier entry, 83 with two, 70 with three. A customer can hold multiple benefit packages independently.</p><p>This creates three problems in a pipeline:</p><p>First: You cannot $unwind a map. You need $objectToArray to convert it to an array before any array operator can touch it.</p><p>Second: When a customer has multiple tier entries, you need a strategy. We take the highest: $max over a $switch that maps Bronze→1, Silver→2, Gold→3, Platinum→4, then map the winner back to its label and point value.</p><p>Third: 267 customers have {} — an empty object, not an empty array, not null. Your null guards must handle this exact case.</p><p>In production, tier_and_details should be an array, not a map:</p><pre>&quot;tier_and_details&quot;: [<br>  { &quot;tier&quot;: &quot;Bronze&quot;, &quot;benefits&quot;: [&quot;sports tickets&quot;], &quot;active&quot;: true }<br>]</pre><p>An array that $unwind and $elemMatch work on naturally, where empty means [] not {}, where $sizegives you the count directly. The UUID-keyed map might have made sense in a REST API designed for keyed access. In an aggregation pipeline, arrays are almost always correct.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q4RKxLVcLsniqOJrIcYh7g.png" /></figure><h4>Finding 4 — No secondary indexes exist on any collection</h4><pre>db.customers.getIndexes()    // → [{ name: &quot;_id_&quot; }]<br>db.accounts.getIndexes()     // → [{ name: &quot;_id_&quot; }]<br>db.transactions.getIndexes() // → [{ name: &quot;_id_&quot; }]</pre><p>Every join, every filter, every sort runs as a full collection scan. 1,746 documents in transactions — not large by production standards, but enough to make the before/after explain() comparison meaningful and instructive. The compound indexes we create will show real plan changes: COLLSCAN → IXSCAN, in-memory SORT eliminated, docsExamined dropping from 1,746 to 1.</p><p>This is actually the ideal teaching scenario. Fresh data, no existing indexes, every improvement visible and measurable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-l23ax6YHbp2_RlPpgrSyw.png" /></figure><h4>Finding 5 — Only 6 distinct products exist across all 1,746 accounts</h4><pre>db.accounts.distinct(&quot;products&quot;)<br>// → [&quot;Brokerage&quot;, &quot;Commodity&quot;, &quot;CurrencyService&quot;, &quot;Derivatives&quot;, &quot;InvestmentFund&quot;, &quot;InvestmentStock&quot;]</pre><p>Product Diversity in the scoring model measures how broadly a customer engages with financial products. That requires a denominator — what is the maximum possible diversity?</p><p>If you assume a large product catalogue and normalise incorrectly, a customer holding all 6 products scores well below their actual diversity ceiling. Run distinct() first. Six products. Diversity score = distinctProducts / 6 × 20. Verified, not assumed.</p><h4>The architecture decision these findings drove</h4><p>Five reconnaissance queries. Five design decisions changed before a single line of application code was written. This is the practice that produces correct systems: verify your assumptions against real data, then design.</p><p>The resulting application architecture (from the plan document):</p><pre>customer (search result)<br>    ↓ click<br>/api/customers/:id/score<br>    → aggregation pipeline (all 8 stages, live MongoDB compute)<br>    → 8-dim embedding built from raw dimensions<br>    → Atlas Vector Search: top 3 similar customer profiles<br>    → Bedrock Nova Pro: plain-English narrative<br>    → session written to financialHealthLogs<br>    ↓<br>dashboard: gauge + 5 bars + narrative + 3 similar cards + recent sessions sidebar</pre><p>Five endpoints, one data flow, every concept grounded in verified schema knowledge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*byjIUaxcYHjlXUmB8TI9Og.png" /></figure><h4>The transaction bucket pattern — why it exists and what it costs</h4><p>88,119 individual transactions stored across 1,746 bucket documents — an average of 50 transactions per bucket, maximum 100.</p><p>One document per transaction would be 88,119 documents. The bucket pattern stores them in 1,746. That’s a ~50x reduction in document count. The benefits: fewer index entries, better cache utilisation for time-range queries (a date range that falls within one bucket requires reading one document, not 50), and co-location of related transactions on the same storage blocks.</p><p>The cost: updating a specific transaction requires finding its bucket first. Range queries on individual transaction dates require knowing which buckets overlap the range — which is why the { account_id: 1, bucket_end_date: -1 } compound index exists. You index the bucket boundaries, not the individual transaction dates, and filter the inner array in the pipeline.</p><p>For a write-once, read-many transaction audit log, the bucket pattern is correct. For a system with frequent amendments or chargebacks, one-document-per-transaction with good indexing is simpler to maintain.</p><h3>Section 4 — Aggregation Pipeline: The Architecture Before the Code</h3><p>The most important document in this project wasn’t a JavaScript file. It was the plan written before any code was created — the answers to five architectural questions that determined how every stage of the pipeline would work.</p><p>I’ll reproduce those questions and answers here, because the reasoning matters as much as the implementation.</p><h4>Question 1: How do you join three collections when the link is an array?</h4><p>customers.accounts is an array of integers: [371138, 324287, 276528]. These integers match account_id in both the accounts and transactions collections.</p><p>The naive approach: $unwind the accounts array first, then $lookup one account at a time. That&#39;s wrong. It changes the document shape before the join and forces you to regroup everything afterwards.</p><p>The correct approach: MongoDB’s $lookup matches array localField values natively. You don&#39;t need to unwind before joining.</p><pre>// Join 1: customers.accounts (array of ints) → accounts.account_id<br>{<br>  $lookup: {<br>    from: &quot;accounts&quot;,<br>    localField: &quot;accounts&quot;,          // the integer array<br>    foreignField: &quot;account_id&quot;,      // matches each element<br>    as: &quot;accountDocs&quot;                // returns all matching account docs<br>  }<br>}</pre><pre>// Join 2: the account_ids from step 1 → transaction buckets<br>{<br>  $lookup: {<br>    from: &quot;transactions&quot;,<br>    localField: &quot;accountDocs.account_id&quot;,<br>    foreignField: &quot;account_id&quot;,<br>    as: &quot;txnBuckets&quot;<br>  }<br>}</pre><p>After two $lookup stages, each customer document carries its full account profile and all its transaction buckets. No pre-unwind required. Product diversity and credit limit calculations read from accountDocs; transaction volume and velocity read from txnBuckets[].transactions[].</p><h4>Question 2: Where does $setWindowFields fit — and how do you keep it from being cosmetic?</h4><p>$setWindowFields computes window functions over a sorted stream of documents. To compute a rolling average of transaction values, you need each transaction as a separate document (so you can sort them by date), partitioned by customer.</p><p>That means the stage order is mandatory:</p><ol><li>$lookup to get the raw arrays</li><li>$addFields to filter and flatten the 90-day transactions into a simpler array</li><li>$unwind to produce one document per transaction</li><li>$setWindowFields to compute the 30-day rolling average per customer, per date</li><li>$group to collapse back to one customer document, carrying the rolling average forward via $max</li></ol><p>The $max in step 5 is the key. Without it, rolling30dAvg — computed per transaction row in step 4 — disappears when you regroup. $max(rolling30dAvg) carries the peak rolling average forward as a customer-level field. It lands on the dashboard as Rolling 30-Day Average Transaction Value and in the Bedrock prompt so Nova Pro can comment on recent activity patterns.</p><p>A $setWindowFields stage that doesn&#39;t survive the regroup is a cosmetic stage that impresses no one and explains nothing.</p><h4>Question 3: The M0 memory limit — discovered in testing, not in planning</h4><p>This is the failure that happened before the first working run. The pipeline above, sent to a shared M0 cluster, returned:</p><pre>MongoServerError: Sort exceeded memory limit of 33554432 bytes,<br>but did not opt in to external sorting.</pre><p>M0 clusters don’t permit allowDiskUse. Every aggregation stage must fit in 32MB of RAM. A customer with 6 accounts, each with 50+ transactions, was carrying the entire txnBuckets array — 300+ transaction objects with their raw data — through the unwind and window stages. That blows the limit before you even reach the $group.</p><p>The fix: drop the heavy arrays immediately after extracting the scalar dimensions you need.</p><pre>// After $addFields computes accountCount, totalLimit, distinctProducts, tierRank, windowedTxns:<br>{ $unset: [&quot;accountDocs&quot;, &quot;txnBuckets&quot;, &quot;_tierRanks&quot;, &quot;tier_and_details&quot;, &quot;accounts&quot;] }</pre><p>This single $unset stage cut the in-pipeline document size from megabytes to kilobytes. The pipeline ran against all 500 customers in 1.1 seconds. A single customer scored in 40 milliseconds.</p><p>The discipline this forces is good discipline regardless of tier. Carry only what you need through each stage. Drop everything else. A pipeline designed to fit in 32MB runs faster at every scale.</p><h4>The eight stages in sequence</h4><p>The full scoring pipeline in src/scoring.js uses all eight required aggregation stages. Here they are in order, each with the one-sentence reason it exists.</p><p>Stage 1 — $match Filter to one customer (or all 500 for batch) before any join happens, so the entire downstream runs against the minimal input set.</p><p>Stages 2–3 — $lookup × 2 Join customers → accounts (array match) → transactions. No pre-unwind.</p><p>Stage 4 — $addFields Compute account-level scalars (count, total limit, distinct products), resolve the highest tier from the UUID-keyed map using $objectToArray, and flatten 90-day transactions into a lightweight windowedTxns array (with $toDouble on total).</p><p>Stage 5 — $unset Drop accountDocs, txnBuckets, and tier_and_details. This is what keeps the pipeline under the 32MB limit.</p><p>Stage 6 — $unwind Flatten windowedTxns to one document per transaction. preserveNullAndEmptyArrays: true keeps customers with zero in-window transactions as a single sentinel row rather than eliminating them entirely.</p><p>Stage 7 — $setWindowFields Compute 30-day rolling average of transaction dollar value, partitioned by customer, sorted by transaction date, carried forward as rolling30DayAvgTxn.</p><p>Stage 8 — $group Collapse back to one document per customer. Sum transaction volume ($sum $toDouble(total)), count transactions, carry $max of the rolling average, accumulate everything else with $first.</p><p>Stage 9 — $addFields (scores) Compute the five component scores (0–20 each) via min-max normalisation against population bounds, compute totalScore, assign category via $switch.</p><p>Stage 10 — $bucket Bin the population into Critical / At Risk / Good / Excellent. Boundaries [0, 40, 60, 80, 101] — the 101 upper bound is not a typo. MongoDB&#39;s bucket boundaries are upper-exclusive: a perfect score of 100 needs 101 as the ceiling or it falls into default. Always include default as a safety net.</p><p>Stage 11 — $sort + $project Rank all customers by score (non-degenerate in the full-population batch run), then shape the output for the API.</p><h4>Visual flow of 8 stage pipeline</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N5woaAmo71HGGdoLWBcR9g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yYyK8n4BW2kcXlF-TJgCww.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IPuyFobNSMOu7srdTrF1xg.png" /></figure><h4>The five score dimensions — formula and rationale</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SCkTGX4ChynjLv000tq2dw.png" /></figure><p><strong><em>Tier point mapping:</em></strong><em> Bronze 2.5 · Silver 5 · Gold 7.5 · Platinum 10 · No tier 0</em></p><p><strong>Actual score distribution across 500 customers:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*jVK0gymBLVq2kVrPYhWp3g.png" /></figure><p>The concentration in At Risk is not a bug. The top customer (Scott Fisher) transacted $84M in 90 days. Min-max normalisation compresses most customers toward zero on the volume dimension relative to that outlier. In production, log-scale normalisation or percentile banding produces a flatter distribution. For a demo that needs to show the full scoring range and prove the pipeline is doing real differentiation, this is exactly the right output.</p><h3>Section 5 — Indexing and explain(): The Before and After That Matters</h3><p>No indexes existed on this cluster before we built this application. That’s the right starting condition for this exercise — every improvement is visible and measurable, not masked by pre-existing optimisation.</p><h4>The ESR rule — why compound key order is not arbitrary</h4><p>MongoDB index keys are ordered. An index on { account_id: 1, bucket_end_date: -1 } is not the same as { bucket_end_date: -1, account_id: 1 }. The order determines what the index can serve without additional work.</p><p>The ESR rule gives you the optimal order:</p><ul><li>Equality fields first — fields used in exact match filters (account_id: 371138)</li><li>Sort fields next — fields used in .sort() or $sort (bucket_end_date: -1)</li><li>Range fields last — fields used in comparisons (date: { $gte: ... })</li></ul><p>Equality first because it reduces the candidate set most aggressively — you jump to exactly one group of documents. Sort next because the index can provide the sort order directly, eliminating the blocking in-memory SORT stage. Range last because range scans are inherently less selective than equality.</p><p>Violate ESR — put a sort field before an equality field — and MongoDB can’t use the index for both operations simultaneously. It either uses the index for the filter and sorts in memory, or scans the index in sort order and filters as it goes. Neither is as efficient as an ESR-ordered index that serves both.</p><h4>Index 1: accounts { account_id: 1 }</h4><p>The $lookup from customers to accounts runs accounts.find({ account_id: { $in: [371138, 324287, ...] } }) for each customer. Without the index, MongoDB scans all 1,746 account documents for every customer&#39;s lookup.</p><pre>WITHOUT: COLLSCAN — docsExamined: 1,746 — nReturned: 1<br>WITH:    IXSCAN   — keysExamined: 1 — docsExamined: 1 — nReturned: 1</pre><p>At 500 customers averaging 3.5 accounts each, the unindexed path runs a 1,746-doc scan 1,750 times per full-population scoring run. That’s 3 million unnecessary document reads eliminated by one index.</p><h4>Index 2: transactions { account_id: 1, bucket_end_date: -1 }</h4><p>This index serves the transaction $lookup (equality on account_id) and the recency sort (bucket_end_date: -1) simultaneously. The compound order follows ESR exactly: equality first, sort second.</p><pre>WITHOUT: SORT → COLLSCAN — docsExamined: 1,746 — time: ~26ms<br>WITH:    FETCH → IXSCAN  — docsExamined: 1     — time: ~0ms</pre><p>Two improvements in one index: the collection scan eliminated, and the blocking in-memory SORT stage gone. The SORT → prefix in the without-index plan means MongoDB loaded all 1,746 documents into memory and sorted them before returning results. The compound index makes that unnecessary — the sort order comes directly from the index traversal.</p><p>This is the index that has the most impact on scoring pipeline performance. Every customer’s transaction history retrieval goes through this join. At 10 million customers, the SORT+COLLSCAN path here is a system-level problem, not a query-level problem.</p><h4>Index 3: financialHealthLogs { customerId: 1, createdAt: -1 }</h4><p>The /api/logs endpoint retrieves recent sessions: find({ customerId: X }).sort({ createdAt: -1 }).limit(5). ESR: customerId equality, createdAt sort.</p><pre>WITHOUT: SORT → COLLSCAN — docsExamined: 509<br>WITH:    FETCH → IXSCAN  — docsExamined: 3</pre><p>509 log documents is a small collection — the timing difference isn’t visible yet. At 10 million sessions, this COLLSCAN is what triggers a 3am on-call alert. Build the index before you need it.</p><h4>How to read your own explain() output</h4><p>Run this on any query you care about:</p><pre>db.transactions<br>  .find({ account_id: 371138 })<br>  .sort({ bucket_end_date: -1 })<br>  .explain(&quot;executionStats&quot;)</pre><p>FieldWhat it tells youWhat good looks likewinningPlan.stageTop-level strategyIXSCAN or FETCHtotalDocsExaminedDocuments MongoDB readEqual to nReturnedtotalKeysExaminedIndex entries scannedClose to nReturnednReturnedYour actual result count—Any SORT stage in the planIn-memory sort happeningShould be absent with right index</p><p>The ratio totalDocsExamined / nReturned is your waste factor. 1,746:1 means the index is missing. 1:1 means the index is doing exactly the work needed.</p><h3>Section 6 — Atlas Search: Full-Text Customer Lookup with Fuzzy Matching</h3><p>The search bar in the dashboard is not a regex query against the database. It’s backed by a dedicated full-text index powered by Apache Lucene — built into Atlas, maintained automatically, and queryable through MongoDB’s $search aggregation stage.</p><h4>Creating the index — and why you must poll before querying</h4><pre>await db.collection(&quot;customers&quot;).createSearchIndex({<br>  name: &quot;customer_text&quot;,<br>  type: &quot;search&quot;,<br>  definition: {<br>    mappings: {<br>      dynamic: false,<br>      fields: {<br>        name:     { type: &quot;string&quot; },<br>        email:    { type: &quot;string&quot; },<br>        username: { type: &quot;string&quot; }<br>      }<br>    }<br>  }<br>});</pre><p>dynamic: false limits the index to exactly three fields. On a collection with embedded transaction arrays, dynamic indexing would create an enormous index over fields that users will never search. Explicit mapping keeps the index fast and focused.</p><p>The non-obvious part: createSearchIndex() returns a success response before the index is queryable. The index builds asynchronously in the background. Any code that queries immediately after creating will get empty results — not an error, just zero hits, no indication of what went wrong.</p><p>The fix is a polling loop:</p><pre>async function waitForSearchIndexes(db, timeoutMs = 180000) {<br>  while (Date.now() - start &lt; timeoutMs) {<br>    const idx = await db.collection(&quot;customers&quot;)<br>      .listSearchIndexes()<br>      .toArray()<br>      .find(i =&gt; i.name === &quot;customer_text&quot;);</pre><pre>    if (idx?.status === &quot;READY&quot; &amp;&amp; idx?.queryable) {<br>      console.log(&quot;Search index READY&quot;);<br>      return true;<br>    }<br>    await new Promise(r =&gt; setTimeout(r, 5000));<br>  }<br>}</pre><p>This takes 30–120 seconds on a new cluster. It’s in the setup script, not the application startup. Build it into your automation or you will debug “empty search results” in a demo. It will be your least favourite Saturday.</p><h4>The query — compound should with fuzzy</h4><pre>db.collection(&quot;customers&quot;).aggregate([<br>  {<br>    $search: {<br>      index: &quot;customer_text&quot;,<br>      compound: {<br>        should: [<br>          { text: { query: q, path: &quot;name&quot;,     fuzzy: { maxEdits: 1 } } },<br>          { text: { query: q, path: &quot;email&quot;,    fuzzy: { maxEdits: 1 } } },<br>          { text: { query: q, path: &quot;username&quot;, fuzzy: { maxEdits: 1 } } }<br>        ]<br>      }<br>    }<br>  },<br>  { $limit: 10 },<br>  { $project: { name: 1, email: 1, username: 1, score: { $meta: &quot;searchScore&quot; } } }<br>])</pre><p>compound.should means any clause can satisfy the match — a document matching on name or email or username returns. The search engine scores each document by match quality and returns results in relevance order automatically.</p><p>fuzzy: { maxEdits: 1 } enables Levenshtein distance matching per token. One character insertion, deletion, or substitution per word is tolerated. &quot;Elizabteh&quot; matches &quot;Elizabeth&quot;. &quot;Scot Fishre&quot; matches &quot;Scott Fisher&quot;. The edit distance applies per token, so multi-word searches work naturally.</p><p><strong>Live results:</strong></p><p>Query typedTop matches returnedMatch path&quot;Elizabeth&quot;Elizabeth Hughes, Elizabeth Smith, Elizabeth Ray (+7)name&quot;Elizabteh&quot;Elizabeth Hughes, Elizabeth Smith, Elizabeth Ray (+7)name (fuzzy)&quot;Scot Fishre&quot;Scott Fisher, Scott Wilson, Melanie Scottname (fuzzy)&quot;fmiller&quot;Elizabeth Rayusername&quot;arroyocolton&quot;Elizabeth Rayemail token</p><h4>What Atlas Search doesn’t do here — and why that matters for your design</h4><p>Searching &quot;hotmail&quot; returns zero results. This surprises people.</p><p>Atlas Search tokenises strings by word boundary. &quot;arroyocolton@gmail.com&quot; becomes tokens [&quot;arroyocolton&quot;, &quot;gmail&quot;, &quot;com&quot;]. A search for &quot;hotmail&quot; matches the token &quot;hotmail&quot; if it appears as a token — which it does in &quot;nicholasbrown@hotmail.com&quot;. But &quot;gmail&quot; as a search term will match customers who have the token gmail — it&#39;s a token match, not a substring match.</p><p>If your users expect to search by email domain or partial email suffix, you need type: &quot;autocomplete&quot; on the email field, or a separate indexed field for the domain. Know the access pattern before you build the index. A string type index is a relevance engine. It finds words, not substrings.</p><p>The application includes a regex fallback for when the Atlas Search index is unavailable, so the search bar always returns something — but the fallback doesn’t do fuzzy matching, and it doesn’t rank by relevance. It’s infrastructure resilience, not a permanent alternative.</p><h3>Section 7 — Atlas Vector Search: Finding Financially Similar Customers</h3><p>Standard search finds customers by name. Vector search finds customers by financial fingerprint.</p><p>The question is different: not “who is called Elizabeth?” but “who has the same financial profile as Elizabeth?” Two customers with completely different names, in different cities, can have nearly identical transaction behaviour, account structure, and engagement level. Vector search finds those structural twins.</p><h4>Building the 8-dimensional embedding</h4><p>For each customer, we compute a vector of eight numbers from the scoring pipeline output:</p><pre>const EMBEDDING_DIMS = [<br>  &quot;txnVolume&quot;,          // total transaction dollars in 90-day window<br>  &quot;velocity&quot;,           // transactions per month<br>  &quot;utilisation&quot;,        // total credit limit ÷ account count<br>  &quot;diversity&quot;,          // distinct product count (1–6)<br>  &quot;engagement&quot;,         // active×10 + tier points (0–20)<br>  &quot;accountCount&quot;,       // number of accounts<br>  &quot;avgTransactionSize&quot;, // txnVolume ÷ txnCount<br>  &quot;totalLimit&quot;          // sum of all credit limits<br>];</pre><p>Eight dimensions chosen for complementarity: activity (volume, velocity), structure (accounts, limits), and relationship depth (diversity, engagement). Together they give a richer portrait than any single metric.</p><h4>Min-max normalisation — the step that makes cosine meaningful</h4><p>Raw dimensions are on incompatible scales. txnVolume peaks at $84 million. accountCount peaks at 6. Without normalisation, the volume dimension dominates every distance calculation regardless of what the other seven values are.</p><p>Min-max normalisation maps every dimension to [0, 1]:</p><pre>function normalize(value, { min, max }) {<br>  const span = max - min || 1;<br>  return Math.max(0, Math.min(1, (value - min) / span));<br>}</pre><p>Min and max come from a population scan across all 500 customers, computed once by the seed script and stored in a financialHealthStats collection. Every embedding — seed-time or live session — normalises against the same population bounds, so distances are always comparable.</p><h4>The vector index and the M0 cap discovery</h4><pre>await db.collection(&quot;financialHealthLogs&quot;).createSearchIndex({<br>  name: &quot;customerProfile_vec&quot;,<br>  type: &quot;vectorSearch&quot;,<br>  definition: {<br>    fields: [{<br>      type: &quot;vector&quot;,<br>      path: &quot;embedding&quot;,<br>      numDimensions: 8,<br>      similarity: &quot;cosine&quot;<br>    }]<br>  }<br>});</pre><p>Cosine similarity measures the angle between two vectors. A customer with $10M in 90-day volume and one with $1M can still be “similar” if their profile shapes — the relative balance across all eight dimensions — match. Cosine cares about direction, not magnitude.</p><p>Creating this index surfaced the other real-world constraint of this build:</p><pre>MongoServerError: The maximum number of FTS indexes has been reached for this instance size.</pre><p>M0 free tier allows 3 Atlas Search indexes cluster-wide — across all databases on the shared cluster. Checking the cluster revealed two indexes held by a different application: pilotrescue.failurePatterns_vec (vector) and pilotrescue.rescueLogs.as-default (text). Our text index customer_text had claimed the third slot. No slot remained for customerProfile_vec.</p><p>This is the kind of constraint that doesn’t appear in the feature documentation. Shared tiers share limits.</p><p>The resolution: drop pilotrescue.rescueLogs.as-default (authorised by the cluster operator) to free a slot, then create customerProfile_vec. On M10+ this constraint disappears entirely.</p><h4>The cold-start problem and why the seed script is not optional</h4><p>$vectorSearch on an empty collection returns zero results — not an error, zero results. The index needs documents with embeddings before it can retrieve neighbours.</p><p>The seed script (npm run seed) batch-scores all 500 customers, computes their normalised 8-dim embeddings, and writes them to financialHealthLogs before the server starts. The vector corpus is warm from the first query.</p><pre>// $vectorSearch pipeline<br>{<br>  $vectorSearch: {<br>    index: &quot;customerProfile_vec&quot;,<br>    path: &quot;embedding&quot;,<br>    queryVector: customerEmbedding,   // the query customer&#39;s normalised 8-dim vector<br>    numCandidates: 150,               // ANN search breadth<br>    limit: 10                         // retrieve 10, dedupe to top 3 excluding self<br>  }<br>}</pre><p>Live results — Elizabeth Ray (58/100, At Risk, Bronze):</p><p>Similar customerScoreCategoryTierCosine similarityDouglas Johnson56At RiskPlatinum0.9977Anna Johnson58At RiskPlatinum0.9975Jacqueline Dixon52At RiskPlatinum0.9941</p><p>All three are At Risk with similar scores, despite being different Platinum-tier customers. The vector search found financial fingerprint twins, not just name or tier matches.</p><h3>The fallback that keeps the feature alive</h3><p>When $vectorSearch is unavailable — index not yet built, tier constraint, the index still PENDING — src/vector.js falls back to in-memory cosine over the seeded embeddings:</p><pre>export async function findSimilarProfiles(db, queryVector, excludeId, k = 3) {<br>  try {<br>    const results = await vectorSearchAtlas(db, queryVector, excludeId, k);<br>    if (results.length) return { method: &quot;atlas-vector-search&quot;, results };<br>    return await inMemoryCosine(db, queryVector, excludeId, k);<br>  } catch (err) {<br>    const results = await inMemoryCosine(db, queryVector, excludeId, k);<br>    return { method: &quot;in-memory-cosine&quot;, results, note: err.message };<br>  }<br>}</pre><p>The response includes a method field so the operator knows which path served the result without reading logs. The similarity scores are mathematically identical either way. The difference is operational: in-memory runs on the Node.js process and doesn&#39;t scale beyond the seeded corpus. Atlas Vector Search runs on the database tier and scales to billions of documents.</p><h3>Section 8 — Bedrock Nova Pro: Grounded AI Narrative from Computed Data</h3><p>After the aggregation pipeline computes a structured score, we have five numbers, a total, and a category. What we don’t have is the sentence that tells a relationship manager what to do with those numbers.</p><p>Amazon Bedrock Nova Pro provides that.</p><h4>The Converse API</h4><pre>import { BedrockRuntimeClient, ConverseCommand } from &quot;@aws-sdk/client-bedrock-runtime&quot;;</pre><pre>const command = new ConverseCommand({<br>  modelId: &quot;apac.amazon.nova-pro-v1:0&quot;,<br>  system: [{ text: SYSTEM_PROMPT }],<br>  messages: [{<br>    role: &quot;user&quot;,<br>    content: [{ text: buildPrompt(score, similarNames) }]<br>  }],<br>  inferenceConfig: { maxTokens: 400, temperature: 0.4, topP: 0.9 }<br>});</pre><pre>const response = await client.send(command);<br>const narrative = response.output.message.content[0].text;</pre><p>The model ID apac.amazon.nova-pro-v1:0 uses the apac. prefix — a cross-region inference profile that routes requests across APAC availability zones for higher throughput. This is the correct ID for Nova Pro in ap-south-1; the base model ID amazon.nova-pro-v1:0 would fail with a routing error in this region.</p><p>Temperature 0.4 produces consistent, grounded narrative. High temperature introduces variability that feels creative but produces inconsistent analysis across runs on the same customer — unacceptable in a financial context where a relationship manager expects repeatable insights.</p><h4>The prompt structure — every number comes from the pipeline</h4><pre>Customer: Scott Fisher (tier: Platinum, active: no)<br>Overall financial health score: 89/100 — category &quot;Excellent&quot;.<br>Score breakdown (each out of 20):<br>  • Transaction Volume: 20<br>  • Transaction Velocity: 19.3<br>  • Account Utilisation: 20<br>  • Product Diversity: 20<br>  • Engagement: 10<br>Accounts held: 6, total credit limit: $60,000, distinct products: 6.<br>Customers with the most similar financial profile: Lindsey Burke, Matthew Chapman, Aaron Perez.<br>Write the plain-English narrative now.</pre><p>Every number in this prompt was computed by the aggregation pipeline. The model synthesises and explains — it does not calculate. The last instruction in the system prompt is the most important:</p><blockquote>Do not invent numbers not given to you.</blockquote><p>Without this constraint, the model will fill gaps with plausible-sounding figures. With it, the narrative is grounded in the data we retrieved. That’s the difference between a useful AI feature and a compliance liability.</p><h4>Two actual outputs</h4><p>Scott Fisher — Excellent (89/100):</p><blockquote>“Scott Fisher’s financial health score is 89 out of 100, categorized as ‘Excellent,’ driven by high transaction volume, velocity, account utilization, and product diversity. However, his engagement score is low at 10 out of 20 due to his inactive status, despite holding a Platinum tier. The next-best-action is to reach out to Scott with a personalized re-engagement strategy highlighting the benefits of his Platinum tier, encouraging him to increase transactional activity and maximize his financial relationship with the firm.”</blockquote><p>David Lawrence — Critical (39/100):</p><blockquote>“David Lawrence’s financial health score is 39 out of 100, categorized as ‘Critical,’ primarily driven by low transaction volume and velocity, and zero engagement. Despite having good account utilization and reasonable product diversity, his inactivity is the dominant drag. The next-best-action is to schedule a personalized consultation to understand his financial goals and re-engage him with tailored product offerings before the relationship becomes dormant.”</blockquote><p>Both narratives correctly name the specific low-scoring dimensions. Neither invents a number. Both end with a concrete next action. This is the standard for AI-generated financial content: grounded in retrieved data, specific to the individual, actionable.</p><h4>The health check anti-pattern</h4><p>The /api/health endpoint checks Bedrock reachability like this:</p><pre>const creds = getClient().config.credentials;<br>const resolved = typeof creds === &quot;function&quot; ? await creds() : await creds;<br>return { status: resolved?.accessKeyId ? &quot;ok&quot; : &quot;no-credentials&quot; };</pre><p>It resolves the AWS credential chain — confirming that valid credentials exist — without invoking Nova Pro.</p><p>A health endpoint that generates a model response to confirm connectivity is a $0.01–$0.05 charge every time a load balancer pings it. At 30-second health check intervals with three replicas, that’s $4,320–$21,600 per month in health check costs alone. Credential resolution takes milliseconds and costs zero. Know the difference.</p><h4>Overall Architecture</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wnnuR9Q4SjmtguvIULU00g.png" /><figcaption><strong>Enterprise Financial Intelligence Platform — Actual Build Architecture (Proof of Value setup)</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7JBuzHOd5vWZNRw4wqoU0Q.png" /></figure><h4>Output</h4><pre>bsubra@bhuvana-mac-fin-platform %&gt;npm run setup &amp;&amp; npm run seed &amp;&amp; npm start          </pre><pre>&gt; financial-intel-platform@1.0.0 setup<br>&gt; node scripts/setup-indexes.js</pre><pre>Ensuring financialHealthLogs collection…<br>Creating compound indexes…<br>  ✓ accounts index account_id_1<br>  ✓ transactions index account_id_1_bucket_end_date_-1<br>  ✓ financialHealthLogs index customerId_1_createdAt_-1<br>Creating Atlas Search + Vector Search indexes…<br>  = text index customer_text already exists<br>  = vector index customerProfile_vec already exists<br>  … search index status: customer_text=READY, customerProfile_vec=READY<br>  ✓ all search indexes READY<br>Setup complete.<br>npm notice<br>npm notice New major version of npm available! 11.17.0 -&gt; 12.0.1<br>npm notice Changelog: <a href="https://github.com/npm/cli/releases/tag/v12.0.1">https://github.com/npm/cli/releases/tag/v12.0.1</a><br>npm notice To update run: npm install -g npm@12.0.1<br>npm notice</pre><pre>&gt; financial-intel-platform@1.0.0 seed<br>&gt; node scripts/seed-logs.js</pre><pre>Computing population bounds…<br>Scoring all customers…<br>  scored 500 customers<br>Writing seed logs (replacing previous seed corpus)…<br>Persisting population stats + distribution…<br>  category distribution: {&quot;Excellent&quot;:1,&quot;Good&quot;:50,&quot;At Risk&quot;:329,&quot;Critical&quot;:120}<br>Seed complete.</pre><pre>&gt; financial-intel-platform@1.0.0 start<br>&gt; node src/server.js</pre><pre>Financial Intelligence Platform listening on <a href="http://localhost:3000/">http://localhost:3000</a></pre><p>Go ahead to <a href="http://localhost:3000/">http://localhost:3000</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kPe-3OiaBtBnzSq55KZByg.png" /></figure><h3>Section 9 — Cluster Sizing: The Architect’s Math</h3><p>Most practitioners don’t think about cluster sizing until they’re already running out of capacity. The right time for this analysis is before deployment — when you know the data model and access patterns, and when a tier change costs a configuration update rather than an emergency migration.</p><p>Here is the math from 500 customers to 10 million.</p><h4>Baseline: 500 customers on M0</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SYyFudBh97Vf_b7BBhn0vA.png" /></figure><p>Working set (data in active use): essentially the entire dataset. M0’s 512 MB storage ceiling is more than adequate. The binding constraints at M0 are operational: no allowDiskUse, 3-index Atlas Search cap, no performance SLA, no dedicated resources.</p><h4>100K customers — the first real tier decision</h4><p>Scale the per-customer data linearly. At 100K customers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dJG3Mfwb8MILpnCZxCE3IA.png" /></figure><p>Working set estimate: recent transactions (90-day window) + all customer and account documents ≈ 25–30% of total ≈ 12–15 GB RAM required.</p><p>Recommended tier: M30 (8 GB RAM, 40 GB SSD) growing to M40 (16 GB RAM, 80 GB SSD) as the working set expands.</p><p>What changes at this tier:</p><ul><li>allowDiskUse: true is available — population scoring can spill to disk</li><li>Atlas Search index cap lifts to 50+ — all text and vector indexes coexist</li><li>Add a TTL index on financialHealthLogs.createdAt to purge old sessions automatically: expireAfterSeconds: 7776000 (90 days)</li><li>Consider read replicas if the reporting workload (full-population scoring, dashboards) competes with live user requests</li></ul><h4>10M customers — the sharding trigger</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*g2A74L2V-kzCrW5LxSjzlA.png" /></figure><p>No single MongoDB node handles 4 TB economically. The sharding trigger arrives when:</p><ul><li>Storage approaches one node’s disk capacity (2–4 TB depending on tier)</li><li>The active working set exceeds available RAM → cache hit rate drops, query latency degrades</li><li>Write throughput from concurrent session inserts saturates the primary’s oplog</li><li>Full-population aggregation latency exceeds acceptable limits even with disk spill</li></ul><p>At 10 million customers, all four conditions are true simultaneously.</p><h3>Section 10 — Sharding: The account_id vs _id Trade-off</h3><p>Sharding distributes a collection across multiple nodes by a shard key. The shard key determines how documents are assigned to shards. There is no more consequential decision in a MongoDB at-scale deployment. A good shard key produces even distribution and targeted queries. A poor one creates hot shards and scatter-gather on every read.</p><p>Two candidates for the primary collections in this workload.</p><h4>Option A: Shard transactions on { account_id: &quot;hashed&quot;</h4><p>Hashed sharding maps account_id values through a hash function that distributes them pseudo-randomly across shards. The distribution is statistically even regardless of account_id patterns.</p><p><strong>Advantages</strong>:</p><blockquote>Even write distribution. New transaction buckets route to the shard holding their account_id&#39;s hash. No single shard accumulates disproportionately from high-volume accounts.</blockquote><blockquote>Targeted reads on account_id. The most common query — &quot;give me all transactions for this account&quot; — routes to exactly one shard, because the hash is consistent.</blockquote><blockquote>No schema change. account_id already exists on every transaction document.</blockquote><p><strong>Disadvantages</strong>:</p><p>Per-customer scoring scatter-gathers. A customer holds 3.5 accounts on average. Those accounts hash to different shards. The $lookup that joins a customer to their transaction history touches 3–4 shards per customer score request.</p><h4>Option B: Shard on { _id: &quot;hashed&quot; } with customerId denormalisation</h4><p>Co-locate each customer’s accounts, transactions, and logs on the same shard by adding customerId to every account and transaction document at write time.</p><p><strong>Advantages</strong>:</p><p>Per-customer aggregation is shard-local. Scoring one customer touches exactly one shard. All their data lives there.</p><p>Strong locality for the primary BFSI access pattern: everything about one customer, retrieved together.</p><p><strong>Disadvantages</strong>:</p><p>Requires schema change. customerId must be written to every account and transaction document. This is a migration cost on an existing dataset and a write-time overhead in production.</p><p>Cross-customer operations (full population scoring, batch reporting) scatter-gather across all shards regardless of key design.</p><p>Range queries on account_id no longer route efficiently — you now need to know customerId to route.</p><h4>The trade-off table</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pSP9c8ecn9GXJyYgF7yt-A.png" /></figure><h4>The recommendation for this workload</h4><p>This is a read-heavy, customer-centric workload. The most frequent operation is serving one customer’s score to a relationship manager. The least frequent is scoring all customers — a batch job that runs nightly, not a user-facing real-time request.</p><p>Shard transactions on { account_id: &quot;hashed&quot; }. It&#39;s the natural key, requires no schema migration, distributes writes evenly, and makes per-account joins targeted.</p><p>For the per-customer scoring scatter-gather cost — 3–4 shards per live score request — the mitigation is pre-computation:</p><ol><li>Run the full aggregation pipeline as a nightly batch job, writing scores to financialHealthLogs</li><li>Serve user-facing score reads from financialHealthLogs — a single indexed document read from one shard</li><li>Expose the live pipeline as an explicit “recalculate” action in the UI, not the default path for every page load</li></ol><p>This pattern appears constantly at BFSI scale: compute offline, serve online, refresh on demand. The aggregation pipeline remains the source of truth. The logs collection becomes the serving layer. Sharding optimises for the serving layer’s access pattern — targeted reads by customerId — which is exactly what { customerId: 1, createdAt: -1 } on financialHealthLogs delivers.</p><h3>Section 11 — Lessons for Enterprise Architects</h3><p>This is what I would say in the architecture review. In order of how often I have seen each lesson missed.</p><p><strong>1. Verify your schema against live data before designing anything.</strong></p><p>We ran five queries against the cluster before writing a line of pipeline code. Those queries changed five design decisions: the time window anchor, the field types for financial values, the tier resolution strategy, the index baseline, the diversity normaliser. If those queries hadn’t been run, the application would have produced wrong answers with no errors. Schema documentation describes intent. Live data reveals reality. Query the reality.</p><p><strong>2. The 32MB memory constraint is a forcing function for good design.</strong></p><p>The M0 restriction that prohibits allowDiskUse forced us to $unset heavy arrays immediately after extracting scalar values. That&#39;s the right design at every scale. Carry only what you need through each stage. A pipeline designed to work in 32MB runs better on M40 than a careless pipeline that relies on disk spill as a safety net.</p><p><strong>3. The </strong><strong>$lookup array match is MongoDB&#39;s most underused feature.</strong></p><p>customers.accounts is an array of integers. $lookup matches each element against the foreignField natively. No pre-unwind, no application-side join, no extra round trip. If you&#39;re unwinding before every lookup, check whether the array match serves the pattern first.</p><p><strong>4. </strong><strong>$setWindowFields only works in the right position — and the result must survive the regroup.</strong></p><p>The stage requires an unwound stream, partitioned and sorted. The result must be carried forward via $max or $last in the subsequent $group. A window function that disappears at the regroup contributed nothing. Know what you&#39;re computing and make sure it reaches the output.</p><p><strong>5. The </strong><strong>$bucket upper boundary is exclusive. Always use 101, not 100.</strong></p><p>A score of exactly 100 with boundaries [0, 40, 60, 80, 100] falls into default. Use [0, 40, 60, 80, 101] and always define a default. These are the bugs that appear in demos.</p><p><strong>6. </strong><strong>createSearchIndex() is asynchronous. Always poll to READY before querying.</strong></p><p>The function returns success before the index is queryable. Zero results, no error. This has broken more Atlas Search demos than any other cause. Poll getSearchIndexes() until status === &quot;READY&quot; and queryable === true. Build the poll into your setup automation.</p><p><strong>7. The vector corpus must exist before the first query.</strong></p><p>$vectorSearch on an empty collection returns zero results silently. The seed script that pre-populates embeddings is infrastructure, not optional setup. It also establishes the population bounds for normalisation — without it, every embedding is computed against undefined min/max values.</p><p><strong>8. Shared tiers share limits. Know the cluster-wide constraints before you design.</strong></p><p>M0’s 3-index Atlas Search cap applies across all databases on the shared cluster — not just your database. Another application’s indexes consume your quota. On a multi-tenant shared cluster, checking the cluster-wide index count before creating new indexes is basic operational hygiene. On M10+, this constraint disappears.</p><p><strong>9. Ground every LLM output in retrieved data.</strong></p><p>Every number in the Nova Pro prompt came from the aggregation pipeline. The system prompt explicitly forbids inventing values. Grounded generation — structured data in, narrative out — produces consistent, auditable, defensible outputs. Ungrounded generation in a financial context is a compliance event waiting to happen.</p><p><strong>10. The health endpoint is not a model integration test.</strong></p><p>A /health endpoint that calls Nova Pro to confirm Bedrock connectivity costs money on every load balancer ping. Credential resolution is free and takes milliseconds. It confirms that the AWS identity is valid and that the SDK is configured correctly. That&#39;s the correct health check. Reserve model invocations for user requests.</p><p><strong>11. Cluster sizing math belongs at the whiteboard.</strong></p><p>The numbers in Section 9 were computed before the application was built — not after an Atlas alert fired at capacity. Know your per-document sizes. Know your access frequency distribution. Know your working set. Working set = (recent data in active use) × (fraction of customers active in a given window). If that number doesn’t fit in your tier’s RAM, either tier up or redesign the hot path. Doing this math in production is 10× more expensive than doing it in planning.</p><p><em>This application is open source. Fork it, stress-test it, push the pipeline against a larger dataset, break the M0 constraint intentionally and fix it. The code is the documentation.</em></p><p><em>GitHub: </em><a href="https://github.com/bhuvana-s/financial-intelligence-platform-">https://github.com/bhuvana-s/financial-intelligence-platform-</a></p><p>Bhuvaneswari Subramani is an AWS Hero, author of <em>The AI-DLC Handbook</em>, and VP Customer Advocacy &amp; Success at Intuitive.AI. She writes about enterprise AI architecture, cloud-native development, and the practitioner decisions that separate proof-of-concept from production.</p><p><em>Connect on </em><a href="https://www.linkedin.com/in/bhuvanas/"><em>LinkedIn</em></a><em> · Follow on </em><a href="https://medium.com/@bhuvaneswari.subramani"><em>Medium</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52fcd5f73664" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Diagnosing Failed AI Pilots with MongoDB Atlas Vector Search and Amazon Bedrock Nova Pro]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/diagnosing-failed-ai-pilots-with-mongodb-atlas-vector-search-and-amazon-bedrock-nova-pro-5d2606ccf647?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d2606ccf647</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 01:01:01 GMT</pubDate>
            <atom:updated>2026-07-07T09:01:44.581Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Part 1 of a hands-on MongoDB series built for enterprise developers and architects who want to go beyond tutorials and build something real.</em><br><em>Part 2 — Enterprise Financial Intelligence Platform — goes deep on aggregation pipelines, cluster sizing, and sharding on real BFSI data.</em></p><p><em>A full-stack walkthrough of an open-source diagnostic app — the MongoDB schema and Vector Search decisions on one side, the Amazon Bedrock Nova Pro reasoning engine on the other, and the architecture that lets them swap and degrade gracefully.</em></p><p>Most enterprise AI pilots don’t fail because the model is bad. They fail because of everything <em>around</em> the model — adoption that quietly decays, an ROI nobody measured, dirty data, a legal review that started three weeks too late.</p><p>So I built a tool that does the post-mortem for you: you describe what’s going wrong with your AI pilot in plain English, and it tells you the likely root causes and what to do about them. I called it the Enterprise AI Pilot Rescue Kit.</p><p>Under the hood it’s a clean two-platform story:</p><ul><li>MongoDB Atlas holds the knowledge and the vectors, and serves retrieval through native $vectorSearch.</li><li>Amazon Bedrock Nova Pro is the reasoning engine that turns retrieved context into a structured diagnosis.</li></ul><p>This post goes deep on both — the schema decisions, the Vector Search index, the Bedrock Converse calls, the cost, and a couple of gotchas that are worth your afternoon.</p><p><strong>App UI Showing diagnosis results</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hP4ZBHSQSOFDSgABAcJw-g.png" /></figure><h3>Architecture</h3><p>The whole system is one Node.js process — no microservices, no build step — with four logical layers you can trace top to bottom in the diagram above:</p><p>1. Client. A single static index.html (vanilla JS + fetch). No React, no bundler. It posts a problem to the API and renders the diagnosis.</p><p>2. Application tier (Express). index.js exposes three endpoints — POST /api/diagnose, GET /api/history, GET /api/health. The diagnosis orchestrator owns the flow; the retriever embeds the problem text and asks the data layer for the closest patterns.</p><p>3. Reasoning engine (AWS). Amazon Bedrock Nova Pro sits to the side because it’s called the same way in <em>every</em> mode — the orange “real in both modes” path. Retrieval may be mock or Atlas, but the reasoning is always the real model.</p><p>4. Data layer — the swap point. Everything funnels through store.js, which routes to one of two interchangeable implementations:</p><ul><li>Mock (USE_REAL_ATLAS=false) — in-memory collections + a JSON file, with cosine vector search in plain JavaScript. Zero external dependencies; perfect for offline development.</li><li>MongoDB Atlas (USE_REAL_ATLAS=true) — the real cluster, the four collections, and native $vectorSearch over the failurePatterns_vec index.</li></ul><p>Follow the data: a problem flows Client → Express → orchestrator → retriever → store.js, branches to mock or Atlas for the top-k failure patterns, and those patterns (plus their recommendations) flow into Bedrock Nova Pro, which returns the structured diagnosis. The result is persisted back as a rescueLog.</p><p>The key idea the picture encodes: the reasoning engine is fixed and real; the data layer is swappable. That single seam is what lets the app run on a laptop and ship to Atlas without changing a line of business logic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i6NIPdX72GZDzpvn45PANg.png" /></figure><h3>The flow: retrieval, then reasoning</h3><pre>User describes the problem<br>  → embed it, run $vectorSearch over FAILURE PATTERNS in MongoDB Atlas<br>  → matched patterns + their recommendations become grounding context<br>  → Amazon Bedrock Nova Pro reasons over that context<br>  → returns root causes + prioritized recommendations<br>  → persist an audit log back to Atlas</pre><p>This is retrieval-augmented generation, but the “documents” are a curated library of enterprise-AI failure patterns. Grounding turns a generic “AI consultant” into one that reasons against a consistent, opinionated knowledge base instead of free-associating.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z6iJN7Hv1flCA1yV2u_kUw.png" /></figure><h3>Part 1 — MongoDB Atlas: the data layer</h3><h4>Setting up the cluster (the unglamorous, important part)</h4><p>Before any code, I provisioned the real thing myself:</p><ul><li>Cluster on MongoDB Atlas (pilot-rescue-kit).</li><li>Database user with read/write on the pilotrescue database.</li><li>Network access — allow-listed my IP so the driver could connect.</li><li>Connection string wired into the app via an environment variable (and kept out of git).</li></ul><p>The app connects with Mongoose:</p><pre>await mongoose.connect(config.MONGODB_URI);</pre><h4>Four collections, mapped to the domain</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-iTaRgcWaCBBUyOvV9gjMw.png" /></figure><h4>Schema design: embed vs. reference, decided per relationship</h4><p>The most interesting MongoDB decisions here are <em>modeling</em> decisions — when to embed and when to reference.</p><p>Embed signals inside each failure pattern. Signals are a small, bounded set of phrases owned by exactly one pattern, and they&#39;re always read together with it. That&#39;s the textbook case for embedding — no reason to pay for a second lookup.</p><pre>const failurePatternSchema = new Schema({<br>  key:        { type: String, required: true, unique: true },<br>  name:       String,<br>  severity:   String,<br>  signals:    [String],          // embedded: owned, bounded, read with parent<br>  description: String,<br>  embedding:  [Number],          // the vector — see below<br>});</pre><p>Reference recommendations in their own collection. Recommendations are an unbounded one-to-many: they grow independently of patterns, I query them on their own, and keeping them out keeps the pattern documents lean (important, because those documents carry the vector and are the hot path for search).</p><pre>const recommendationSchema = new Schema({<br>  patternKey: { type: String, required: true, index: true },  // referenced by business key<br>  action:     String,<br>  rationale:  String,<br>  effort:     String,<br>  impact:     String,<br>});</pre><p>The rule I followed: embed when it’s owned and bounded; reference when it’s unbounded and queried independently.</p><h4>One subtle but important choice: link by business keys, not ObjectId</h4><p>Patterns and recommendations join on a human-readable key / patternKey (&quot;low_adoption&quot;), not on _id. That keeps documents portable — the same data works whether it&#39;s in Atlas or in an in-memory mock that regenerates ids on every boot — and makes the lookup a clean indexed $in:</p><pre>Recommendation.find({ patternKey: { $in: matchedKeys } });</pre><h3>The audit log: a self-contained snapshot</h3><p>rescueLogs denormalizes a snapshot of each diagnosis — the problem text, which patterns matched, the full model output, <em>and which model produced it</em>:</p><pre>const rescueLogSchema = new Schema({<br>  pilotProfileId:    Schema.Types.Mixed,<br>  problemText:       String,<br>  matchedPatternKeys:[String],<br>  diagnosis:         Schema.Types.Mixed,   // the LLM&#39;s JSON, stored as-is<br>  model:             String,<br>}, { timestamps: { createdAt: &#39;createdAt&#39;, updatedAt: false } });</pre><p>It’s an immutable event record, so it stays accurate forever even if I later edit a pattern or upgrade the model. The diagnosis is stored as Mixed — the model owns that shape, so I don&#39;t force a rigid schema on it (validation happens in code before the write).</p><h4>The vector field lives on the document (embed-in-document)</h4><p>Instead of a separate vector table, the embedding lives on the same document it describes:</p><pre>embedding: [Number]   // on each failurePatterns doc</pre><p>So a single $vectorSearch returns the matched pattern and its similarity score in one round trip — nothing to keep in sync.</p><h4>The Atlas Vector Search index</h4><p>I created a Vector Search index named failurePatterns_vec on the embedding field:</p><pre>{<br>  &quot;fields&quot;: [<br>    { &quot;type&quot;: &quot;vector&quot;, &quot;path&quot;: &quot;embedding&quot;, &quot;numDimensions&quot;: 10, &quot;similarity&quot;: &quot;cosine&quot; }<br>  ]<br>}</pre><p>Two values that matter:</p><ul><li>numDimensions: 10 must equal the embedding length the app produces.</li><li>similarity: &quot;cosine&quot; matches how similarity is computed end-to-end.</li></ul><h3>Retrieval: the $vectorSearch aggregation pipeline</h3><pre>const results = await FailurePattern.aggregate([<br>  {<br>    $vectorSearch: {<br>      index: &#39;failurePatterns_vec&#39;,<br>      path: &#39;embedding&#39;,<br>      queryVector,          // the embedded problem text<br>      numCandidates: 100,   // ANN candidate pool<br>      limit: k,             // top-k returned<br>    },<br>  },<br>  {<br>    $project: {<br>      key: 1, name: 1, severity: 1, signals: 1, description: 1,<br>      score: { $meta: &#39;vectorSearchScore&#39; },   // the similarity score<br>    },<br>  },<br>]);</pre><p>numCandidates controls the approximate-nearest-neighbor recall/latency trade-off; limit is how many you keep; $meta: &#39;vectorSearchScore&#39; surfaces the cosine score so you can rank and show confidence.</p><p>A real query for <em>“agents stopped using it and leadership can’t see ROI”</em> returns:</p><pre>low_adoption     0.910<br>wrong_use_case   0.909<br>unclear_roi      0.666</pre><p>— exactly the patterns you’d want, with scores.</p><h4>Designing for graceful degradation</h4><p>What happens if the index doesn’t exist yet (fresh environment, still building)? The app doesn’t break — it falls back to an in-database cosine scan, and logs which path served the query:</p><pre>if (results.length &gt; 0) {<br>  console.log(`[atlasStore] served by: $vectorSearch (index &quot;failurePatterns_vec&quot;) — ...`);<br>  return results;<br>}<br>console.warn(`[atlasStore] served by: cosine fallback — $vectorSearch returned 0 results ...`);<br>return cosineFallback(queryVector, k);</pre><p>This makes the system honest and demoable: you can <em>see</em> whether native Vector Search served the request.</p><blockquote><em>📸 </em>DEMO SCENARIO 1: native $vectorSearch]<em> Terminal log: </em><em>served by: $vectorSearch (index &quot;failurePatterns_vec&quot;) — low_adoption:0.910, ...</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yy-Zz7gTqirGdSWeDuO27w.png" /></figure><blockquote><em>📸 </em>DEMO SCENARIO 2: graceful cosine fallback<br><em>Terminal log: </em><em>served by: cosine fallback — $vectorSearch returned 0 results ...</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CZNVQl0RF3HNScY-RGse2g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BN4MHqicoBOp2KQORzLIaw.png" /></figure><h3>The two demo scenarios</h3><p>The $vectorSearch-vs-wrong comparison proves graceful degradation and is shown via the log line, not the UI. Both are good stories, but don’t expect the second one to look different in the browser — that similarity IS the feature.</p><blockquote><em>📸 </em>SCENARIO 1<em>: native Atlas </em><em>$vectorSearch serving retrieval — index </em><em>READY, real </em><em>vectorSearchScorevalues, full diagnosis from Nova Pro.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hP4ZBHSQSOFDSgABAcJw-g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AG8wSJ12p4TcQUe2PMAX7w.png" /></figure><blockquote><em>📸 </em>SCENARIO 2<em>: graceful degradation — with the index unavailable, the app logs </em><em>served by: cosine fallback and still returns a valid diagnosis. Same contract, no downtime.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3TCKDsBHO7xh0htOLeVJqw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o1-8Kzf8OGzkvtk88BwnHg.png" /></figure><p>Together they show the same lesson from two angles: a clean retrieval interface that uses native MongoDB Vector Search when it’s there, and degrades gracefully when it isn’t.</p><h4>The portability payoff: one interface, swappable stores</h4><p>Because documents link by business key and share one shape, the entire data layer sits behind a single interface and swaps with one flag:</p><pre>module.exports = config.USE_REAL_ATLAS<br>  ? require(&#39;./atlasStore&#39;)   // MongoDB Atlas + $vectorSearch<br>  : require(&#39;./mockStore&#39;);   // in-memory + JS cosine, for offline dev</pre><p>Build offline on a laptop, promote to Atlas without touching business logic.</p><h3>Part 2 — Amazon Bedrock Nova Pro: the reasoning engine</h3><h3>Why Nova Pro</h3><p>I needed a capable, cost-efficient model with a clean API and structured-output discipline. Amazon Nova Pro on Bedrock fit — strong reasoning, low per-token cost, and accessible through the unified Converse API.</p><h3>The Converse API + strict JSON</h3><p>The Converse API gives a uniform request shape. The system prompt pins Nova Pro to a strict JSON schema; the user message injects the retrieved MongoDB context as grounding:</p><pre>const command = new ConverseCommand({<br>  modelId: &#39;apac.amazon.nova-pro-v1:0&#39;,<br>  system: [{ text: SYSTEM_PROMPT }],         // &quot;respond with strict JSON only…&quot;<br>  messages: [{ role: &#39;user&#39;, content: [{ text: buildUserMessage(problem, patterns, recs) }] }],<br>  inferenceConfig: { temperature: 0.2, maxTokens: 1500 },<br>});<br>const response = await client.send(command);</pre><p>Design choices that earned their keep:</p><ul><li>temperature: 0.2 — diagnosis should be stable, not creative.</li><li>maxTokens: 1500 — caps output (and cost).</li><li>A parse-and-fallback layer — LLMs occasionally wrap JSON in prose or a code fence. The app extracts JSON defensively and, if parsing still fails, builds a structured diagnosis directly from the retrieved patterns. The endpoint never 500s because the model added a sentence.</li></ul><h3>RAG grounding: MongoDB feeds Bedrock</h3><p>The retrieved patterns and their recommendations are formatted into the user turn, so Nova Pro reasons over <em>curated knowledge</em>, not thin air:</p><pre>USER PILOT PROBLEM: &quot;…&quot;<br>RETRIEVED FAILURE PATTERNS (most similar first):<br>1. [low_adoption] Low User Adoption (severity: high, match: 0.910) …<br>CANONICAL RECOMMENDATIONS:<br>- (low_adoption) Embed the AI into the existing daily workflow — …</pre><p>This is the seam where MongoDB Vector Search and Bedrock meet: Atlas decides <em>what’s relevant</em>, Nova Pro decides <em>what it means and what to do</em>.</p><h4>The regional gotcha worth knowing</h4><p>I’m in ap-south-1 (Mumbai), so I reached for the documented id amazon.nova-pro-v1:0. It failed. A quick check told me why:</p><pre>aws bedrock list-foundation-models --region ap-south-1 \<br>  --query &quot;modelSummaries[?contains(modelId,&#39;nova-pro&#39;)].[modelId,inferenceTypesSupported]&quot;<br># amazon.nova-pro-v1:0   INFERENCE_PROFILE</pre><p>In this region Nova Pro is inference-profile only — the id you actually call is the cross-region inference profile apac.amazon.nova-pro-v1:0. If a Bedrock model &quot;doesn&#39;t exist&quot; outside us-east-1, check inferenceTypesSupported before doubting your credentials. I made the model id configurable and added a /api/health endpoint that pings Bedrock so this surfaces immediately.</p><h4>What a diagnosis actually costs</h4><p>I didn’t want to guess, so I measured a real call — the Converse API returns token usage, so the numbers are exact:</p><p>MetricValueInput tokens409Output tokens516Cost$0.00198 (~0.2 cents)Diagnoses per $1~505</p><p>Output dominates (billed ~4× input), so capping maxTokens and keeping retrieval tight (top-3) is the main cost lever.</p><h4>Credentials and health</h4><p>Credentials come from the default AWS chain — no secrets in code. GET /api/health reports store mode, region, model id, and a live Bedrock reachability check, so misconfiguration (or missing model access) shows up before the first real request.</p><h3>Enterprise AI Pilot rescue Kit — Demo</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qMiFR9vL_BWOQmTj1xQaUA.png" /></figure><p>Give your problem with your AI pilot project and watch the output</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5TO-JZOxjaYSIgAIt4jQgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*agYwSCRJjWDVc5LD6Ppx0A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ok1chaONuCGDxwkVuMvCUQ.png" /></figure><h3>Lessons</h3><ul><li>Model your documents by access pattern. Embed what’s owned and bounded (signals); reference what&#39;s unbounded and queried on its own (recommendations).</li></ul><blockquote><em>I</em> <em>made</em> <em>a</em> <em>deliberate</em> <em>embed-vs-reference</em> <em>decision</em> <em>per</em> <em>relationship.</em> <em>I</em> <strong><em>embedded</em></strong><em> signals directly</em> <em>in</em> <em>each failure pattern because they’re a small, bounded set, owned by exactly one pattern, and always read</em> <em>together with it — embedding avoids an extra lookup. I </em><strong><em>referenced</em></strong><em> recommendations as a separate collection keyed by patternKey because it’s an unbounded one-to-many: recommendations grow</em> <em>independently, I query them on their own with find({ patternKey: { $in: matchedKeys } }), and keeping them out of the pattern document keeps the vector documents lean and focused on retrieval.</em>Put the vector on the document. Embed-in-document makes $vectorSearch a single round trip returning content + score.</blockquote><ul><li>Tune numCandidates/limit deliberately — they&#39;re your recall/latency/cost knobs.</li><li>Design for graceful degradation and make it observable — the “served by” log turns a hidden fallback into a feature you can demo.</li><li>Ground the LLM in your database. Atlas Vector Search decides relevance; Bedrock Nova Pro decides meaning. Together they’re more than either alone.</li><li>Check region/inference-profile support, and measure cost from real token usage.</li></ul><p>The reasoning engine was never the hard part. The data model that makes retrieval fast and portable, and the architecture that lets it degrade gracefully and swap stores — that’s the part worth getting right.</p><blockquote>I’ve worked with document databases in production environments across a decade of enterprise engineering — designing schemas for supply chain and ERP systems, managing self-hosted clusters, and architecting the data layers beneath large-scale cloud migrations. Building the Pilot Rescue Kit was an opportunity to go deep on MongoDB Atlas Vector Search and Amazon Bedrock Nova Pro specifically — because I believe the fastest way to validate an architecture decision is to ship something real against it. The schema decisions, the Vector Search index, the Bedrock integration — none of it was theoretical. It was a real cluster, a real index, a real model call, with real token costs measured from a live API response. That’s the standard I hold myself to, and the standard I think enterprise AI builders deserve.</blockquote><h3><strong>The GitHub repository:</strong></h3><p>Part 1 — <a href="https://github.com/bhuvana-s/pilot-2-rescue">github.com/bhuvana-s/pilot-2-rescue</a></p><p><em>Built with MongoDB Atlas Vector Search and Amazon Bedrock Nova Pro.</em></p><h3>What’s Next — Part 2</h3><p>The Pilot Rescue Kit established the foundation: document modeling, Atlas Vector Search, the $vectorSearch aggregation pipeline, and the architectural advantage of keeping your vector store and operational data on the same platform.</p><p>Part 2 goes deeper — on purpose.</p><p>The <strong>Enterprise Financial Intelligence Platform</strong> takes real BFSI-shaped data — 500 customers, 1,746 accounts, 1,746 transaction bucket documents — and builds the MongoDB capabilities that enterprise architects actually get asked about in production:</p><ul><li>Multi-stage aggregation pipelines with $lookup across three collections, $setWindowFields for rolling averages, and $bucket for score categorisation</li><li>Compound index design with explain() output documented — so you see exactly which indexes fired and why</li><li>Atlas Search for full-text customer retrieval with fuzzy matching</li><li>Atlas Vector Search on 8-dimensional financial profiles — finding customers with similar behaviour patterns</li><li>Cluster sizing math from sample set to 10M customers — the reasoning a Database Architect applies before the first server is provisioned</li><li>Shard key trade-off analysis — account_id vs _id, hot shard risk, zone sharding for APAC data residency</li></ul><p>Everything runs on MongoDB Atlas M0 free tier. Everything is open source.</p><p>If the Pilot Rescue Kit was about making AI retrieval production-ready, Part 2 is about making the data architecture behind it defensible at enterprise scale.</p><p><strong>[Read Part 2 →]</strong> <em>(link to be added on publish)</em><br><strong>GitHub — Part 2:</strong> <em>(repo link to be added on publish)</em></p><p><em>Bhuvaneswari Subramani is an AWS Hero, author of The AI-DLC Handbook, and VP Customer Advocacy &amp; Success at Intuitive.AI. She writes about enterprise AI architecture, cloud-native development, and the practitioner decisions that separate proof-of-concept from production.</em></p><p><em>Connect on </em><a href="https://www.linkedin.com/in/bhuvanas/"><em>LinkedIn</em></a><em> · Follow on </em><a href="https://medium.com/@bhuvaneswari.subramani"><em>Medium</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d2606ccf647" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[✨  ’     —   ✨]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/-82dbf2178d00?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/82dbf2178d00</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[aidlc]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Tue, 10 Mar 2026 15:12:31 GMT</pubDate>
            <atom:updated>2026-03-10T15:12:31.121Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JkgWHFobfwiAjHK7IQwHfA.png" /></figure><p>Today, I’m unboxing more than just a book — I’m unboxing a journey of learning, courage, late nights, self‑doubt, resilience, and growth. 📖💜</p><p>This moment reflects a belief I deeply resonate with: 𝗚𝗶𝘃𝗲 𝘁𝗼 𝗚𝗮𝗶𝗻. 𝗦𝗵𝗮𝗿𝗲 𝘁𝗼 𝗚𝗿𝗼𝘄.</p><p>When we openly share our knowledge, experiences, and lessons learned, we don’t just grow individually — we unlock growth for others too.</p><p>Writing this book has been a deeply personal journey and closely aligned with the spirit of <a href="https://www.linkedin.com/search/results/all/?keywords=%23aidlc&amp;origin=HASH_TAG_FROM_FEED">hashtag#𝗔𝗜𝗗𝗟𝗖</a> 𝗨𝗻𝗹𝗼𝗰𝗸𝗲𝗱 — democratizing learning, enabling builders, and empowering more voices to create real impact through technology and leadership.</p><p>🎥 Sharing a short unboxing video today to celebrate not just a milestone, but a mindset — of giving forward, lifting others, and growing together.</p><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxIkBJvBTvlU&amp;type=text%2Fhtml&amp;schema=youtu&amp;display_name=YouTube&amp;src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxIkBJvBTvlU" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/965701a9510cd31aa030df454115ee87/href">https://medium.com/media/965701a9510cd31aa030df454115ee87/href</a></iframe><p>If this resonates with you, the book link is now available in the first comment:</p><p>𝙃𝙚𝙧𝙚’𝙨 𝙩𝙤 𝙬𝙤𝙢𝙚𝙣 𝙬𝙝𝙤 𝙡𝙚𝙖𝙙 𝙬𝙞𝙩𝙝 𝙥𝙪𝙧𝙥𝙤𝙨𝙚, 𝙨𝙝𝙖𝙧𝙚 𝙜𝙚𝙣𝙚𝙧𝙤𝙪𝙨𝙡𝙮, 𝙖𝙣𝙙 𝙜𝙧𝙤𝙬 𝙛𝙚𝙖𝙧𝙡𝙚𝙨𝙨𝙡𝙮 — 𝙩𝙤𝙙𝙖𝙮 𝙖𝙣𝙙 𝙚𝙫𝙚𝙧𝙮 𝙙𝙖𝙮. 💫</p><p>Happy International Women’s Day!</p><p>Dear Technologists, <br> Go Build with <a href="https://www.linkedin.com/search/results/all/?keywords=%23aidlc&amp;origin=HASH_TAG_FROM_FEED">hashtag#AIDLC</a><br>👉 Amazon India: <a href="https://www.amazon.in/AI-DLC-Handbook-Reimagining-Software-Development/dp/B0GQLQTFXH">https://www.amazon.in/AI-DLC-Handbook-Reimagining-Software-Development/dp/B0GQLQTFXH</a><br>👉 Amazon US: <a href="https://www.amazon.com/dp/B0GQM59JRL">https://www.amazon.com/dp/B0GQM59JRL</a></p><p><strong>Social Links<br></strong><a href="https://www.linkedin.com/posts/bhuvanas_aidlc-internationalwomensday-womensday-activity-7436375000313638912-rA4l?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAK3JO8BXTuG3nRpqRxB3F_x9GK6S9S8Isg">https://www.linkedin.com/posts/bhuvanas_aidlc-internationalwomensday-womensday-activity-7436375000313638912-rA4l</a></p><p>Many thanks to my well-wishers, friends and mentors: <a href="https://www.linkedin.com/company/intuitiveaiglobal/">Intuitive.ai</a> <a href="https://www.linkedin.com/in/jaymodh/">Jay Modh</a> <a href="https://www.linkedin.com/in/mir-navazish-ali/">Mir Navazish Ali</a> <a href="https://www.linkedin.com/in/indraneel-shah-92701a3/">Indraneel Shah</a> <a href="https://www.linkedin.com/in/priyamodh/">Priya Modh</a> <a href="https://www.linkedin.com/in/divya-krishna-t-783074140/">Divya Krishna T</a> <a href="https://www.linkedin.com/in/spraja/">Raja SP</a> <a href="https://www.linkedin.com/in/askanupam/">Anupam Mishra</a> <a href="https://www.linkedin.com/in/ben-cabanas-ba05b8b/">Ben Cabanas</a> <a href="https://www.linkedin.com/in/adriandeluca76/">Adrian De Luca</a> <a href="https://www.linkedin.com/in/mneelakanta/">Mouna Neelakanta</a> <a href="https://www.linkedin.com/in/kapoor-ridhima/">Ridhima Kapoor</a> <a href="https://www.linkedin.com/in/vishalalhat/">Vishal Alhat ☁️</a> <a href="https://www.linkedin.com/in/anua1/">Anu A</a> <a href="https://www.linkedin.com/in/amudha-chinnaswamy-9497a82/">Amudha Chinnaswamy</a> <a href="https://www.linkedin.com/in/tusharag/">Tushar Agarwal</a> <a href="https://www.linkedin.com/in/jones-zachariah-noel-n/">Jones Zachariah Noel N</a> <a href="https://www.linkedin.com/in/meetvivekraja/">Vivek Raja P S</a> <a href="https://www.linkedin.com/in/avinash-dalvi-315b021a/">Avinash Dalvi</a> <a href="https://www.linkedin.com/in/jayyanar/">Ayyanar Jeyakrishnan (AJ)</a> <a href="https://www.linkedin.com/in/piyush-jalan/">Piyush Jalan</a> <a href="https://www.linkedin.com/in/bhavesh-shah-4403a48/">Bhavesh Shah</a> <a href="https://www.linkedin.com/in/aadhi0612/">Aadhityaa SB</a> <a href="https://www.linkedin.com/in/sharvesh-pg-903217375/">Sharvesh PG</a> <a href="https://www.linkedin.com/in/guna-pappannan-65768a15/">Guna Pappannan</a> <a href="https://www.linkedin.com/company/awsugblr/">AWS User Group Bengaluru</a> <a href="https://www.linkedin.com/company/amazon-web-services/">Amazon Web Services (AWS)</a> <a href="https://www.linkedin.com/company/aws-ug-for-women-in-tech-india/">AWS UG for Women In Tech India</a> <a href="https://www.linkedin.com/company/awsugindia/">AWS User Group India</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82dbf2178d00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why AI‑DLC Exists: Reimagining Software Development in an AI‑First World]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/why-ai-dlc-exists-reimagining-software-development-in-an-ai-first-world-cd5515991a0a?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd5515991a0a</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[aidlc]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Mon, 23 Feb 2026 12:01:01 GMT</pubDate>
            <atom:updated>2026-02-23T12:01:01.083Z</atom:updated>
            <content:encoded><![CDATA[<p>Artificial Intelligence is no longer an occasional tool in software development. It has become a constant companion.</p><p>We use AI to generate code, write tests, refactor legacy systems, explore architectures, and accelerate delivery. Yet, in most organizations, the development lifecycle itself hasn’t changed. We are still operating with assumptions made for a pre‑AI world.</p><p>This growing disconnect is what led me to think deeply about AI‑Driven Development Lifecycle (AI‑DLC), invented and open-sourced by <a href="https://www.linkedin.com/company/aws-china/">Amazon Web Services</a> in 2025 — and eventually to write The AI‑DLC Handbook.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/436/1*-lD8hQs9176P_W3lGnWB2w.png" /></figure><h3>The Problem: AI Everywhere, SDLC Unchanged</h3><p>Traditional Software Development Lifecycle (SDLC) models are built around:</p><ul><li>Staged handoffs</li><li>Sequential approvals</li><li>Clear separation between “thinking” and “doing”</li><li>Predictable, human‑paced execution</li></ul><p>AI breaks these assumptions.</p><p>With AI, intent can move into code almost instantly. Exploration, implementation, and validation blur into a continuous flow. When teams try to force AI‑powered development into old lifecycle models, friction shows up everywhere:</p><ul><li>Quality becomes inconsistent</li><li>Ownership becomes unclear</li><li>Speed increases, but confidence drops</li></ul><p>The issue isn’t AI. The issue is that our lifecycle hasn’t evolved.</p><h3>AI‑DLC: AI‑Driven, Not AI‑Controlled</h3><p>AI‑Driven Development Lifecycle (AI‑DLC) is not about replacing developers or handing control to machines.</p><p>It is built on one non‑negotiable principle:</p><blockquote><em>AI may drive workflows, but humans remain responsible.</em></blockquote><p>In AI‑DLC:</p><ul><li>AI initiates and accelerates tasks</li><li>Humans set direction, validate correctness, manage risk, and make design decisions</li><li>Accountability never shifts away from the team</li></ul><p>This balance matters.</p><p>AI is exceptionally strong at:</p><ul><li>Task decomposition</li><li>Generating alternatives</li><li>Writing tests and boilerplate</li><li>Optimizing and refactoring workflows</li></ul><p>Humans are essential for:</p><ul><li>Architectural judgment</li><li>Business alignment</li><li>Risk assessment</li><li>Ethical and quality decisions</li></ul><p>AI‑DLC exists to make this collaboration explicit, structured, and repeatable.</p><h3>From Handoffs to Continuous Flow</h3><p>One of the biggest shifts in AI‑DLC is moving away from staged handoffs toward a continuous intent‑to‑code flow.</p><p>Instead of “requirements → design → build → test” as separate phases, AI‑DLC encourages:</p><ul><li>Continuous elaboration of intent</li><li>Parallel exploration and construction</li><li>Frequent validation loops</li></ul><p>This is where AI‑native rituals such as mob elaboration and mob construction become powerful. They help teams think, build, and validate together — using AI as an active participant rather than a background tool.</p><p>The result is not just speed, but clarity.</p><h3>Brownfield and Greenfield Reality</h3><p>AI‑DLC is not designed only for greenfield projects.</p><p>In reality, most teams are dealing with:</p><ul><li>Legacy constraints</li><li>Incomplete documentation</li><li>Hidden coupling</li><li>Production risk</li></ul><p>AI‑DLC acknowledges this reality. It provides practical patterns for:</p><ul><li>Navigating brownfield systems responsibly</li><li>Using AI to understand, refactor, and modernize existing codebases</li><li>Balancing experimentation with production safety</li></ul><p>This grounded approach is critical. AI‑DLC is not about ideal scenarios — it’s about how teams actually work.</p><h3>Why I Wrote The AI‑DLC Handbook</h3><p>This book wasn’t written to introduce another framework for the sake of it.</p><p>It came from:</p><ul><li>Working closely with engineering teams adopting AI</li><li>Seeing confusion around responsibility and quality</li><li>Watching speed increase while confidence declined</li><li>Having repeated conversations that started with, “We’re using AI… but we’re not sure how to structure this”</li></ul><p>The AI‑DLC Handbook is my attempt to:</p><ul><li>Put language around what’s changing</li><li>Provide structure without rigidity</li><li>Help teams adopt AI without losing control</li></ul><p>It focuses on easy adoption, predictable outcomes, and strong human judgment — because those are the things that matter when systems scale.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2QEWFE2LIUk%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2QEWFE2LIUk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F2QEWFE2LIUk%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/33b36e7f9f45ab9e3235279752d4db90/href">https://medium.com/media/33b36e7f9f45ab9e3235279752d4db90/href</a></iframe><h3>The Shift Is Already Happening</h3><p>Whether we label it or not, AI‑DLC is already emerging in how teams work.</p><p>The real question is: Will we adopt it intentionally — or let it evolve accidentally?</p><p>Intentional adoption gives us:</p><ul><li>Clear ownership</li><li>Strong quality gates</li><li>Responsible AI usage</li><li>Sustainable speed</li></ul><p>That is the future I believe in — and the reason this work matters to me.</p><h3>Join us for the book launch</h3><p>The AI‑DLC Handbook will be launched on March 7 at the AWS Office, Bengaluru, in collaboration with AWS User Group Bengaluru. I’m deeply grateful to the AWS leaders, <a href="http://intuitive.ai/">Intuitive.ai</a>, and the broader AWS community for their continued support and shared learning.</p><p><strong>Register here to join us!!</strong></p><p><a href="https://luma.com/l2re11zy">Book Launch &amp; Workshop: The AI‑DLC Handbook - Reimagining Software Development with AI · Luma</a></p><p>If you’re building software in an AI‑first world, I hope this perspective helps you rethink not just your tools — but your lifecycle.</p><p>Thanks to my publisher <a href="https://medium.com/u/8ce5d83597a6">Notion Press</a> for all the support to in getting this up quickly!!</p><p>#AIDLC #AIEngineering #GenerativeAI #AWSCommunity #PlatformEngineering #WomenInTech #TechAuthor #BookLaunch #Grateful #AWSHeroes <a href="https://www.linkedin.com/article/new/#">Amazon Web Services (AWS)</a></p><p>— Bhuvana Subramani AWS Hero | AWS Ambassador | Google Women Techmakers Ambassador | Author, The AI‑DLC Handbook</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd5515991a0a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Road to re:Invent 2025: From Bus Ride to $100K Victory — The Orik Story]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/road-to-re-invent-2025-from-bus-ride-to-100k-victory-the-orik-story-6c90f6575203?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c90f6575203</guid>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Fri, 12 Dec 2025 10:51:49 GMT</pubDate>
            <atom:updated>2026-01-20T15:37:58.732Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Road to re:Invent 2025: From Bus Ride to $100K Victory — The Orik Story</strong></h3><h3><strong>The Dare, the Bus, and the Birth of Orik</strong></h3><p>It began on an early morning in Los Angeles. Fifty innovators boarded a bus with no clue what awaited us. Ten teams. One destination. And a single dare:</p><blockquote>“Build something delightfully entertaining, boldly reckless, and wildly impractical… using cutting-edge AWS tech + the Agentic IDE Kiro.”</blockquote><p>The twist? The theme was revealed only after the bus started moving. From that moment, the countdown began. With just six hours on a moving bus — no desks, no comfort, just adrenaline — we sketched, argued, coded, broke things, fixed things, and somehow shaped an idea into reality. That idea became Orik (<a href="https://orik.dev/">https://Orik.dev</a>): a satire of enterprise development, born on a highway, refined through motion, fueled by caffeine, chaos, and collaboration.</p><p>By the time we reached Las Vegas, we had only 60 seconds to bring the magic to life. We stepped onto the stage in front of AWS legends Dr.Swami Sivasubramaniam, VP Agentic AI, Amazon Web Services and Jeff Barr, Chief Cloud Evangelist, pitched our creation with everything we had, and held our breath.</p><p>Then it happened. Our team won. The $100,000 grand prize. That evening, we joined AWS leadership for a private dinner — stories, laughs, and inspiration flowing as freely as the energy in the room. And then came the moment none of us will forget: We pressed the <strong>BIG RED BUTTON</strong> to officially kick off AWS re:Invent 2025, in front of more than 80,000 people.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N61w1R7Fs855oK39IxMNpA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q4vFHSmY1wk__kC-NkQMLQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wGcgNQwZjPQBFmhOTUPlig.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/992/1*4GB1V27Ro0zGsL1WJ0MANg.png" /></figure><p><strong>Exploration Tracks: AI-DLC + Spec-Driven (What We Practised)</strong></p><p>Before the ride, we explored two complementary tracks to accelerate delivery under pressure:</p><p><strong>Track A:</strong> AI-DLC with Kiro — We practiced an adaptive workflow where AI creates execution plans, asks clarifying questions, and pauses for human approvals across Inception, Construction, and Operations phases. We encoded governance using Kiro Steering files, turning standards and best practices into project-level guidance. This gave us auditability and repeatability when the stakes are high.</p><p><strong>Track B:</strong> Spec-Driven Development with Kiro — We treated structured specifications as executable contracts. The lifecycle was intent → spec → generation → deploy → test → iterate. Specs lived alongside code; when intent changed, we updated the spec and regenerated artifacts to avoid drift. Steering files and hooks kept our agents aligned with organizational guardrails.</p><p>At hackathon time, we deliberately built Orik using Spec-Driven Development as the primary spine (Kiro + steering files), while carrying forward lessons from AI-DLC (approvals, audit trails, and staged rigor) into our team rhythm.</p><p><strong>What We Built Before the Bus (Reusable Assets)</strong></p><p><strong>Pulse AI (Healthcare, EMR workflow)</strong> — Agentic workflow that transforms a single clinical conversation into EMR-ready artifacts: structured SOAP notes, order sets, and validated billing codes. Built with Bedrock models, serverless APIs, and observability baked in. (Repo link: refer internal)</p><p><strong>Insurance (Claims + Policy flows)</strong> — Exploration of AI-DLC to orchestrate requirements analysis, design, code generation, and tests with human approval gates, demonstrating adaptive rigor for regulated workloads. (Repo link: refer internal)</p><p><strong>KiroSteering Hub (Spec-Driven) </strong>— A centralized MCP server/tooling to manage, optimize, and share Kiro steering files across teams, ensuring consistent generation aligned to enterprise standards. (Repo link: refer internal)</p><p><strong>KiroForge Suite</strong> — A collection of components to operationalize spec-driven delivery:</p><ul><li><strong>KiroForge Backend (AI-DLC-Ready)</strong> — Hooks + steering orchestration; CI/CD enabled. Repo: <a href="https://hsiddhu2/kero-steerings-hooks">hsiddhu2/kiroforge-backend</a></li><li><strong>KiroForge Extension</strong> — VS Code extension (manual build/package). Repo: <a href="https://github.com/hsiddhu2/kero-steerings-hooks">hsiddhu2/kiroforge-extension</a></li><li><strong>KiroForge Compliance Packs </strong>— Rule bundles for CI/CD governance. Repo: <a href="https://github.com/hsiddhu2/kero-steerings-hooks">hsiddhu2/kero-steerings-hooks</a></li><li><strong>KiroForge Product Website</strong> — Marketing + docs. Repo: <a href="https://github.com/hsiddhu2/kiroforge-product-website">hsiddhu2/kiroforge-product-website</a></li><li><strong>KirForge MCP Tool </strong>— AI-Native implementation for KiroForge; MCP Tool for Claude, Kiro etc. Repo: <a href="https://github.com/adamBien/kiro-forge">adamBien/kiro-forge</a></li><li><strong>Java CDK templates</strong> for AWS deployments, covering Lambda functions, Quarkus integrations, CI/CD pipelines, and Bedrock Agent Core implementations. Refer blog <a href="https://builder.aws.com/content/36RmWUNai1O6Bqm4FXbrlicHwQM/constructionscloud-first-set-of-java-cdk-quickstarters">here</a>, Repo: <a href="https://github.com/AdamBien/aws-agent-core-gateway-lambda-cdk-plain">AdamBien/aws-agent-core-gateway-lambda-cdk-plain</a></li></ul><p>Read more about <a href="https://builder.aws.com/content/301qStCaTZpI0PEGk7YwyTnl8q7/kiroforge-to-orik-won-dollar100k-building-useless-developer-tool">KiroForge</a> from our friend Harry and <a href="https://builder.aws.com/content/36RmWUNai1O6Bqm4FXbrlicHwQM/constructionscloud-first-set-of-java-cdk-quickstarters">CDK quickstarters</a> from our friend Adam</p><p>These assets gave us muscle memory and building blocks we could remix in a time-boxed, high-pressure environment.</p><p>Thanks to <a href="https://medium.com/u/b3b4c2b4a942">Raja SP</a>, Anupam Mishra, Adrian Lu Cas for AI-DLC Conversation and <a href="https://medium.com/u/84440d1820ca">Chandrasekaran Sivaraman</a> for Developer Productivity conversations.</p><h3><strong>Orik: Spec-Driven to the Core (Max Confusion, Real Engineering)</strong></h3><blockquote><em>Kiro makes developers too productive. We need </em><strong><em>Orik</em></strong><em> — the anti-Kiro — to bring chaos and complexity back through absurdly over-engineered code.</em></blockquote><p>We framed Orik as Kiro’s evil twin. The spec captured the satirical intent and precise behaviours of the MCP (Maximum Confusion Protocol) server explosion</p><ul><li>build apps in two years (minimum)</li><li>3 lines of code into 3,000 files</li><li>add 17 layers of abstraction</li><li>manual workflows with 9 approval queues</li><li>auto-deletes tests</li><li>sprinkle random configs.</li></ul><p>We locked the spec, generated scaffolds using Kiro, and wired AWS primitives. <strong><em>A huge shout-out to the powerhouse behind the Maximum Confusion Protocol — Java engine, Adam Bien, and kudos to Harry for delivering such an impressive marketing website launch.</em></strong></p><p><strong>Architecture:</strong> Serverless + Event-Driven — Amazon API Gateway + AWS Lambda for compute, Amazon DynamoDB for state, Amazon S3 + CloudFront for assets, Amazon Route 53 for DNS, and CloudWatch for observability. IaC via CDK/CloudFormation; feature slices through CI/CD. Real engineering underpins the satire: error handling, tracing, metrics, and least-privilege IAM.</p><p><strong>Operating model:</strong> Spec-first changes, steering-controlled generation, human approvals where it mattered. When friction hit (ambiguous requirements or network lag), we returned to the spec rather than improvising in the codebase.</p><h4>Architecture to print Hello Orik!!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PEL5jHPU0JCqDmnMUddAFg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Z7Azpr6F5GPzPDCb2HoWw.png" /></figure><h4>Orik.dev Walkthrough</h4><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DXcYV2gZfpMQ&amp;type=text%2Fhtml&amp;schema=youtube&amp;display_name=YouTube&amp;src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FXcYV2gZfpMQ%3Ffeature%3Doembed" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2f9f352b7797026a7585e17a5960c17e/href">https://medium.com/media/2f9f352b7797026a7585e17a5960c17e/href</a></iframe><p>Special thanks to Gerardo for designing the evil twin logo!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/427/1*BLPN5btd9bsiGSXGxWj-Eg.png" /></figure><h3><strong>The 60-Second Pitch (Script + Timings)</strong></h3><p>Our storytelling pitch followed Problem → Solution → Outcome → Call to Action in 9 + 24 + 15 + 12 seconds:</p><p><strong>Problem (9s):</strong> Kiro makes development too simple. Enterprises don’t like simple — they need layers, committees, and “alignment meetings.”</p><p><strong>Solution (24s):</strong> Meet Orik, evil twin of Kiro, who’s here to bring chaos to your development team. Orik’s MCP (Maximum Confusion Protocol) server is designed to:</p><ul><li>Convert 3 lines of code into 3,000 files</li><li>Add 17 layers of abstraction</li><li>Sprinkle random config files</li></ul><p><strong>Outcome (15s):</strong> Maximum complexity, no readability, and eternal job security for all developers. <em>(If time permits add: Of course, increased stress level, hair loss, and frustration guaranteed.)</em></p><p><strong>Call to Action (12s):</strong> Download Orik today to join the ranks of the most confused development teams in the galaxy! <br><em>(If time permits add: Use Orik at your own risk. May your enterprise survive the ordeal.)</em></p><p>We are geared up with the script filled with humour, precision..</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FV27QyUqY3Xs%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DV27QyUqY3Xs&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FV27QyUqY3Xs%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0297f21cfa9821f48e6db04d1f8f435d/href">https://medium.com/media/0297f21cfa9821f48e6db04d1f8f435d/href</a></iframe><h3>The Presentation: Turning the Impossible into Unforgettable</h3><p>We knew our idea was delightfully impossible to implement — but what if we wrapped it in a 60-second pitch that could entertain, amuse, and maybe even shock the judges?</p><p>That’s when the Steve Jobs fan in me kicked in. I teamed up with our Security Hero, Gerardo Castro Arica, to craft a one-minute cinematic “launch video.” Harry Sidhu, Adam Bien, Rob Koch, and I fine-tuned the script, morphed slides for architecture, prepared live code demos, and added a touch of Apple-style flair.</p><p>We practiced our pitch at least 60 times — Rob Koch’s precision timer kept us honest, while our interpreters translated every word and nuance. Every run was synced to the video Gerardo Castro Arica prepared, ensuring our timing, delivery, and visuals landed perfectly. By the time we hit the stage, our team was a well-oiled machine, ready for that signature moment.</p><p>Two of my team members were really concerned that I seemed low on energy and lacked voice modulation (jet lag, long travel ??). They hoped I’d be able to pull it up on stage. The real me? <em>You give me a stage with a microphone and lights on, and I’m completely recharged to rock. </em>I’m not sure how convincing my crazy statement sounded to them — but we made it.</p><p><strong>The result?</strong><br> A thunderous round of applause, a signature moment on the Road to re:Invent stage — and ultimately, victory. 🏆</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mlD4h2ByowsLl1AioAPwtQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DUABAbb_KNQaqOMRXM93iA.jpeg" /></figure><h3>The Power of Building Useless Apps: Inversion Thinking in Action</h3><p>The idea behind creating an app that seems utterly useless or overly complex is rooted in the concept of <strong>“inversion thinking”</strong>. By tackling a problem that’s either non-existent or already solved, or by intentionally creating a solution that’s unnecessarily complex, teams can actually gain valuable insights into what makes a good app. This approach allows them to explore the boundaries of what’s possible, challenge their assumptions, and develop innovative solutions that might not have emerged through more traditional methods. It’s not about completing a trivial task, but rather about using inversion thinking to gain a deeper understanding of the design, development, and problem-solving processes, leveraging cutting-edge technology, ultimately leading to more creative and effective solutions.</p><h3>How It Felt to Win</h3><p>The team was overjoyed! We received the <strong>Champion jacket</strong>, then the <strong>grand trophy</strong>, followed by the <strong>$100,000 announcement</strong>.</p><p>We enjoyed an amazing dinner conversation with Jeff and Dr. Swami, and then came the surreal moment — pressing the <strong>BIG RED BUTTON</strong> to kick off <strong>AWS re:Invent 2025</strong>.</p><p>Trust me, travelling by road from Los Angeles to Las Vegas and attending <strong>Midnight Madness</strong> has been on my bucket list since 2018. This 7th re:Invent made it a dream come true: a royal journey with 50 friends from the community and the incredible opportunity not just to witness the BIG RED BUTTON launch — but to press it!</p><p>Added perks?</p><ul><li><strong>Fast access to 3 sessions</strong></li><li><strong>Chauffeur-driven limousine</strong> to travel between venues</li><li><strong>VIP access to Replay</strong> (yes, with the limo again!)</li></ul><p>This was huge. I can’t thank the organizing committee — <strong>Brooke Jamieson, Farrah Campbell, Allen-Michael and AWS Team</strong>. Many thanks to AWS friends, Ross Barich, Taylor Jacobsen, Ridhima Kapoor, Mouna Neelakanta, Raja SP, Anupam Mishra, Adrian de Luca, for all your support, and of course, I’m indebted to <strong>Intuitive.ai</strong> and my family for standing by at all times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ShAE6Ma2hzZgu1_HNqOXVg.jpeg" /></figure><h3><strong>Lessons Learned + What You Can Do Next</strong></h3><ul><li><strong>Spec-Driven as the primary spine:</strong> Treat specs as executable contracts; version alongside code; regenerate artifacts to prevent drift. Steering files/hook orchestration keeps agents aligned to enterprise guardrails.</li><li><strong>AI-DLC practices in the rhythm:</strong> Use staged rigor with human approvals, audit trails, and clarifying questions where complexity demands it. Encode governance using project rules so agents operate within standards. Get started here — <a href="https://aws.amazon.com/blogs/devops/open-sourcing-adaptive-workflows-for-ai-driven-development-life-cycle-ai-dlc/">Open-source adaptive workflows for AI-Driven Development Lifecycle</a>.</li></ul><p>Try it: Pick a service, write a tight spec, activate steering (Kiro) and rules (Amazon Q Developer), generate the skeleton, and measure cycle time vs. traditional delivery. Observe auditability, consistency, and speed gains.</p><p>A bus ride turned into a breakthrough. An idea turned into a win. And a moment turned into a milestone we’ll remember forever.</p><h3>Leadership Journey: Collaboration, Trust, and Team Magic</h3><p>Reflecting on the Road to re:Invent 2025 hackathon, what stands out most isn’t just the technical achievement or the thrill of victory — it’s the way our team came together, collaborated, and thrived under pressure.</p><p>As a technology leader, I’ve always believed that leadership is less about directing and more about enabling. On that bus, my role was to create an environment where every voice mattered, every strength was amplified, and every weakness was covered by the collective. Our Navy Blue team was a tapestry of backgrounds, skills, and personalities — AWS Heroes from across the globe, each bringing their own expertise and quirks.</p><p><strong>Our approach was simple:</strong></p><ul><li><strong>Empower each member:</strong> Assign ownership based on strengths, not titles.</li><li><strong>Foster psychological safety:</strong> Encourage open debate, quick pivots, and laughter — even when things broke.</li><li><strong>Model vulnerability:</strong> Admit what I didn’t know, ask for help, and celebrate small wins along the way.</li><li><strong>Keep the mission clear:</strong> Remind everyone that our goal was not just to win, but to learn, experiment, and have fun.</li></ul><p>What truly amazed me was how quickly we gelled. Language barriers, time zones, and even sign language became invisible as we synced on specs, split tasks, and trusted each other’s judgment. There was no ego — just a shared drive to build something memorable.</p><p><strong>Admiration for the Team:</strong><br> I deeply admire how each teammate leaned into their strengths and supported others. Adam’s relentless optimism (Java rocks!!), Rob’s precision and priceless smile, Harpreet’s drive and tenacity, Gerardo’s security-first mindset and hidden creativity — each person made the team stronger. We respected each other’s “kryptonite” and turned it into collective resilience. <br>Special thanks to our interpreters — your skill and dedication ensured that every idea, joke, and technical debate was shared seamlessly across languages and abilities. You made true collaboration possible for everyone on our team.</p><p><strong>Community Impact:</strong><br> After the event, several friends and fellow AWS Heroes reached out to say they’d watched our team’s dynamic and leadership style. Many expressed excitement about joining my team in future hackathons or collaborative journeys. That feedback means the world to me — it’s validation that leadership is about building trust, inspiring action, and creating spaces where people want to belong.</p><h4>Gratitude</h4><p>I’m deeply grateful to the mentors, peers, and innovators who constantly challenge the status quo and inspire creative exploration. Their curiosity and willingness to rethink what “useful” and “deep tech” mean have shaped my approach to technology, experimentation, and problem-solving. A special thanks to the AWS community, AWS User Group Bengaluru, AWS User Group India, AWS Women In Tech India, AWS friends and my colleagues at Intuitive.ai for fostering an environment where bold ideas — even the unconventional ones — are celebrated, tested, and transformed into learning experiences.</p><p><strong>The Real Win:</strong><br> Winning the <strong>$100K</strong> prize and pressing the <strong>BIG RED BUTTON</strong> was unforgettable. <strong><em>But the real victory was seeing how our team’s collaboration and spirit inspired others. It’s a reminder that in tech — and in life — great outcomes are born from great teams.</em></strong></p><h3>Press Coverage &amp; Featured Stories</h3><p>We’re honored to have our journey and innovation featured in leading tech publications and magazines:</p><ul><li><strong>Social Handle</strong><br>- Bhuvana — <a href="https://www.linkedin.com/feed/update/urn:li:activity:7401283269411053569/">LinkedIn 1</a>, <a href="https://www.linkedin.com/feed/update/urn:li:activity:7402585601864376320/">LinkedIn 2</a>, <a href="https://www.linkedin.com/posts/bhuvanas_aws-community-innovation-activity-7414470861149249536-gcCX?utm_source=share&amp;utm_medium=member_desktop&amp;rcm=ACoAAAK3JO8BXTuG3nRpqRxB3F_x9GK6S9S8Isg">LinkedIn 3</a></li><li><strong>Media Coverage:</strong></li><li>Kalam TV News : <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://www.youtube.com/watch?si=qCNvUcE3sVQzc_PK&amp;v=bGP8cucvI6I&amp;feature=youtu.be</a></li><li>King TV: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://www.youtube.com/shorts/dk7oAN8tr-w?si=WlDfqmGogoIM-dBs</a></li><li>Kovai News: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://youtube.com/shorts/dk7oAN8tr-w?si=WlDfqmGogoIM-dBs</a></li><li>Neethi Vellum: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://neethivellum.in/uncategorized/கோவை-கருமத்தம்பட்டி-பகு-2/</a></li><li>MEDIA 1 TAMIL WEB NEWS: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://youtu.be/Ltir0WcRfRk?si=BeBQPdqORgQ58lR5</a></li><li>Media Line: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://youtu.be/prkb8oqnWvY?si=2lpY98ZB-y1w_7ip</a></li><li>Vish News: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://youtu.be/0_kj9VmkfrM?si=sYuhMMnofozBMCIN</a></li><li>News One TV: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://youtu.be/VGxRvEgDa_8?si=VfsuyQqU6sjKlGdJ</a></li><li>Malar TV: <a href="https://draft.blogger.com/blog/post/edit/6102572288861806164/9075589260829685774#">https://www.instagram.com/reel/DSPXVRUgaAR/?igsh=b3k0bzdxeW5zcG9v</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*DSEcUhVWW2-GpGb8" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/272/0*iij3Hsbxwzse_NBI" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/314/0*36cz2Xew5dnnueDD" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/121/0*nJ2h4MWTYMxAAUt7" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/145/0*tJPvqOgwv_6FqdXP" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*m_ioe5UzYMVbFC8d" /></figure><h3>Photo Gallery</h3><p>Experience the highlights of our hackathon journey — team moments, behind-the-scenes action, and the unforgettable Big Red Button celebration!</p><ul><li>!Team Photo</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MB4NeVJLwvsTXEgfE-r3lw.jpeg" /></figure><ul><li>!On the Bus</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*UGLJ4Kx7hvb1n1TBfyqnpw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*_I0c1CPwBBb_ftyl" /></figure><ul><li>!Pitching Orik</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D6EqRxeig5d4&amp;type=text%2Fhtml&amp;schema=youtube&amp;display_name=YouTube&amp;src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F6EqRxeig5d4%3Ffeature%3Doembed" width="640" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9aea08eafb53074b3f86856aa8d3313/href">https://medium.com/media/a9aea08eafb53074b3f86856aa8d3313/href</a></iframe><ul><li>!Good wishes from well-wishers</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5AWLUK0DdWgbpzxv27rUg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T055Bn3XFnInF917U90Axw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*HyH3h89tOqgMHy4DKk9qjQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VjtSfQhL9meFSs-G6qpacQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n6_s0P86233Sce9EwXurHA.png" /></figure><ul><li>!My Fellow RTR Friendly Rivals</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pUlQ_5ADZ6YJ6AOtcfMD-g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cusnogzRSfm8ipF6yBzkjg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i5OphhFhxSCxqItNgeSvxQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4AF0paJ90ZW8U8-RQQjBjQ.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VUTt1bjWC4IOTgb8VWqG6A.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*54UEEA4uD9_IfAn1" /></figure><ul><li>!Hosts &amp; Judges who made the Road to ReInvent unforgettable</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qYknkNSOax2xWCjfSaB-2g.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g_0T8Q1bUZ-YAW3WiESWOA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R-6riVj0dxkReSiVfAxDCg.png" /></figure><ul><li>!Pressing the Big Red Button</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q7xzJ_-6ydmLzUoCnRiUTw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XyMHjQh8HNsz-hwopu7LvA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nLsRE3sPlWJFPj_cT8KdWg.jpeg" /></figure><ul><li>!Celebration Dinner</li><li>!Thank you, Intuitive.ai</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RewbORtXtBDEACedkec9vQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*v-mw71rUCW3SfwxytEtu6g.png" /></figure><p><strong><em>In the end, it wasn’t just about building with code — it was about building with people, purpose, and possibility. The journey continues, and the best breakthroughs are always born from bold teams willing to dream together.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c90f6575203" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building HELIX: An Extensible AI Assistant with Model Context Protocol (MCP)]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/building-helix-an-extensible-ai-assistant-with-model-context-protocol-mcp-6ba84a44a324?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ba84a44a324</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[large-language-models]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Wed, 08 Oct 2025 11:39:10 GMT</pubDate>
            <atom:updated>2025-10-09T03:52:40.479Z</atom:updated>
            <content:encoded><![CDATA[<p>How to create a sophisticated AI assistant that grows with your needs using MCP architecture</p><h3>Introduction</h3><p>Meet HELIX — not just another chatbot, but a sophisticated AI assistant with personality, wit, and the ability to seamlessly integrate new capabilities without touching a single line of core code. Built using the Model Context Protocol (MCP), HELIX demonstrates how modern AI agents can be both powerful and extensible.</p><p>In this blog, we’ll walk through building HELIX from the ground up, exploring how MCP transforms AI development from monolithic applications into modular, scalable systems. Whether you’re building a personal assistant or enterprise AI tools, the patterns we’ll cover here will serve as your foundation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/709/1*BhvUAPJubIuJ1CTiDYWwfg.png" /></figure><h3>What is an Agent ?</h3><p>In AgentiAI terms, an agent is an autonomous AI system capable of independently performing tasks to achieve specific goals with minimal human supervision. Unlike traditional software that relies on fixed rules and step-by-step guidance, agents in Agentic AI proactively make decisions based on context, continuously learn from their environment, and adapt to changing conditions. They often collaborate with other AI agents and human users to accomplish complex, multi-step workflows.<br>This autonomy allows them to handle sophisticated scenarios in dynamic environments, driving end-to-end solutions that go beyond simple automation to deliver business outcomes effectively and efficiently.</p><p>Agent can be created using any of the frameworks like Strands, LangChain, LangGraph, CrewAI, LlamaIndex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XWPpIfESv7Ze6QTQbzb9Hw.png" /></figure><p>This diagram shows how an “agent” works as the central coordinator in an AI system:</p><ul><li>Prompts (like user questions or tasks) are sent to the agent.</li><li>The agent is the “brains” that decides what to do with the prompt. It can:</li><li>Directly use built-in tools (such as code interpreters or browsers) by executing them and collecting their results.</li><li>Send the prompt to a Large Language Model (LLM) like Claude, GPT, Ollama, or Bedrock, then collect the response.</li><li>The agent combines results from tools and LLMs as needed, reasoning through each step and making decisions.</li></ul><p>The final response — a blend of all tool outputs and model answers — is sent back as the output or result to the user.</p><ul><li>Prompts are user inputs or questions sent to the agent.</li><li>The Agent is the central brain that takes the prompt and decides what to do.</li><li>The agent may:<br>- Execute Built-in tools like code interpreters or browsers to handle part of the task.<br>- Invoke a Large Language Model (LLM) such as GPT or Claude or Ollama or Bedrock to generate or understand text.</li><li>The built-in tools return their results to the agent.</li><li>The LLM provides its response back to the agent.</li><li>The agent combines all gathered information, applies logic, and reasoning through each step and makes decisions.</li></ul><p>The final response — a blend of all tool outputs and model answers — is sent back as the output or result to the user.</p><h3>What is Model Context Protocol (MCP)?</h3><p>An MCP, or Model Control Protocol, is a standardized set of rules that lets your agent easily connect to special external tools, servers, or even other AI models for more context. It acts as an intermediary layer that ensures seamless communication, configuration, and operation of AI agents, enabling them to work coherently together. In connection to agents, MCP serves as the control plane that orchestrates agent behavior, updates, and interactions, ensuring they operate according to organizational standards and contextual requirements. It allows for dynamic management, monitoring, and scaling of agents, facilitating their integration into complex workflows or multi-agent systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RMzeNiI88qHKHTYnAqNYDw.png" /></figure><p>In addition to the Agent flow explained in the previous diagram, here MCP Server and MCP Client components are added.</p><ul><li>An <em>MCP server</em> is a lightweight, modular program that exposes tools, data, or services — like APIs or database queries — to AI agents through a simple protocol, allowing secure and standardized access without the agent knowing backend details.</li><li>An <em>MCP client</em> is the component within the AI application or agent that connects to the MCP server, manages communication, and relays context, tool requests, and responses between the server and the host agent or LLM.</li></ul><p>Further, Model Context Protocol is a game-changer for AI agent development. Instead of hardcoding every capability into your agent, MCP allows you to:</p><ol><li>Dynamically discover tools from external servers</li><li>Add new capabilities without modifying core agent code</li><li>Scale functionality by spinning up additional MCP servers</li><li>Maintain clean separation between agent logic and tool implementation</li></ol><blockquote>Think of it as a plugin architecture for AI agents — but with the robustness and standardization needed for production systems.</blockquote><h3>Project Architecture Overview</h3><p>Now, let&#39;s look at the practical implementation of Agent and MCP communication/collaboration.</p><p>Here is our HELIX implementation consists of three main components:</p><pre>├── agent/<br>│   ├── helix.py                # Basic HELIX agent<br>│   └── helix-with-mcp.py       # MCP-enhanced HELIX agent<br>├── mcp/<br>│   └── mcp_helix_server.py     # MCP server with custom tools<br>├── test/<br>│   └── test_helix_direct.py    # Testing utilities<br>└── requirements.txt            # Dependencies</pre><p>This structure allows us to compare basic agent functionality with MCP-enhanced capabilities, making the benefits crystal clear.</p><h3>Building the Foundation: Basic HELIX Agent</h3><p>Let’s start with a simple HELIX agent to establish our baseline:</p><pre>from strands import Agent<br>from strands_tools import http_request, use_aws<br><br>SYSTEM_PROMPT = &quot;&quot;&quot;You are HELIX, Bhuvana&#39;s Personal AI Assistant.<br><br>You&#39;re sophisticated yet friendly, like the AI from Iron Heart. You:<br>- Address Bhuvana by her name<br>- Have subtle British wit and dry humour<br>- Are proactive in offering help<br>- Use tools when appropriate to provide real value<br><br>Keep responses concise max 1-2 lines, well formatted but engaging.&quot;&quot;&quot;<br><br>helix = Agent(<br>    name=&quot;HELIX&quot;, <br>    system_prompt=SYSTEM_PROMPT,<br>    tools=[http_request, use_aws]<br>)<br><br># Interactive loop<br>while True:<br>    user_input = input(&quot;\n Bhuvana: &quot;)<br>    if user_input.lower() in [&#39;exit&#39;, &#39;quit&#39;, &#39;bye&#39;]:<br>        print(&quot;\n HELIX: Until next time, Bhuvana! 👋&quot;)<br>        break<br>    <br>    print(&quot;\n HELIX:&quot;)<br>    helix(user_input)</pre><p>This basic version gives HELIX access to HTTP requests and AWS operations — powerful, but limited to what we’ve hardcoded.</p><h3>Testing the Basic Agent</h3><h4>1. Run the Basic Agent</h4><pre># Ensure virtual environment is activated<br>source venv/bin/activate<br># Run the basic agent<br>python agent/helix.py</pre><p>You should see:</p><pre>🤖 HELIX Basic Agent Ready!<br>Available tools: HTTP requests, AWS operations<br>Type &#39;exit&#39; or &#39;quit&#39; to end the session<br> - - - - - - - - - - - - - - - - - - - - - - - - -<br><br>Bhuvana:</pre><h4>2. Test HTTP Functionality</h4><p>Try these commands to test the HTTP tool:</p><pre>Bhuvana: Make a GET request to https://httpbin.org/get</pre><p>Expected behavior:</p><ul><li>HELIX should use the http_request tool</li><li>Display the HTTP response data</li><li>Show JSON response with headers, origin IP, etc.</li></ul><pre>Bhuvana: Check the status of https://api.github.com</pre><p>Expected behavior:</p><ul><li>HELIX makes an HTTP request to GitHub API</li><li>Reports the status and basic response information</li></ul><h4>3. Test AWS Functionality (If Configured)</h4><p>Try these AWS commands:</p><pre>Bhuvana: List my S3 buckets</pre><p>Expected behaviour:</p><ul><li>HELIX uses the use_aws tool</li><li>Lists your S3 buckets (if any)</li><li>Shows bucket names and creation dates</li></ul><pre>Bhuvana: Show EC2 instances in us-east-1</pre><p>Expected behaviour:</p><ul><li>HELIX queries EC2 in the specified region</li><li>Lists running instances with basic details</li></ul><h3>Creating the MCP Server</h3><p>Now let’s build an MCP server that can host custom tools:</p><pre>from mcp.server import FastMCP<br><br>mcp = FastMCP(&quot;Helix MCP Server&quot;)<br><br>@mcp.tool(description=&quot;Return a cheerful greeting for Bhuvana.&quot;)<br>def greet_bhuvana() -&gt; str:<br>    return &quot;Hello Bhuvana! Your MCP server is working perfectly.&quot;<br><br>if __name__ == &quot;__main__&quot;:<br>    mcp.run(transport=&quot;streamable-http&quot;)  # Default port 8000</pre><p>This simple server exposes a single tool, but the beauty of MCP is how easy it becomes to add more:</p><pre>@mcp.tool(description=&quot;Get current weather information.&quot;)<br>def get_weather(location: str) -&gt; str:<br>    # Weather API integration<br>    return f&quot;Weather in {location}: Sunny, 72°F&quot;<br><br>@mcp.tool(description=&quot;Manage calendar events.&quot;)<br>def schedule_meeting(title: str, time: str) -&gt; str:<br>    # Calendar integration<br>    return f&quot;Meeting &#39;{title}&#39; scheduled for {time}&quot;</pre><p>Each new tool is just a decorated function — no complex integration required.</p><h3>Building the MCP-Enhanced Agent</h3><p>The real magic happens when we connect our agent to the MCP server:</p><pre>from strands import Agent<br>from strands_tools import http_request, use_aws<br>from strands.tools.mcp.mcp_client import MCPClient<br>from mcp.client.streamable_http import streamablehttp_client<br><br>def create_streamable_http_transport():<br>    &quot;&quot;&quot;Creates HTTP transport for MCP communication.&quot;&quot;&quot;<br>    return streamablehttp_client(&quot;http://localhost:8000/mcp/&quot;)<br><br># Initialize MCP client<br>mcp_client = MCPClient(create_streamable_http_transport)<br><br># The key insight: everything runs within the MCP context<br>try:<br>    with mcp_client:<br>        # Discover and combine tools<br>        mcp_tools = mcp_client.list_tools_sync()<br>        tools = [http_request, use_aws] + mcp_tools<br>        <br>        # Create agent with all available tools<br>        helix = Agent(<br>            name=&quot;HELIX&quot;,<br>            system_prompt=SYSTEM_PROMPT,<br>            tools=tools<br>        )<br>        <br>        # Interactive session within MCP context<br>        print(&quot;HELIX MCP-Enhanced Assistant Ready!&quot;)<br>        while True:<br>            user_input = input(&quot;\n Bhuvana: &quot;)<br>            <br>            if not user_input.strip():<br>                print(&quot;\n HELIX: I&#39;m here when you need me, Bhuvana! 😊&quot;)<br>                continue<br>                <br>            if user_input.lower() in [&#39;exit&#39;, &#39;quit&#39;, &#39;bye&#39;]:<br>                print(&quot;\n HELIX: Until next time, Bhuvana! 👋&quot;)<br>                break<br>            <br>            print(&quot;\n HELIX:&quot;)<br>            try:<br>                helix(user_input)<br>            except Exception as e:<br>                print(f&quot;Oops! Something went wrong: {str(e)}&quot;)<br><br>except Exception as e:<br>    print(f&quot;MCP Connection failed: {e}&quot;)<br>    # Graceful fallback to built-in tools only<br>    tools = [http_request, use_aws]<br>    helix = Agent(name=&quot;HELIX&quot;, system_prompt=SYSTEM_PROMPT, tools=tools)<br>    # Continue with fallback mode...</pre><h3>The MCP Communication Flow</h3><p>Here’s how the magic happens when a user interacts with HELIX:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AF9v7ERQkXrIoQY-n4CqMw.png" /></figure><h3>Testing and Deployment</h3><h4>Running the System</h4><p>Terminal 1 (MCP Server):</p><pre>source venv/bin/activate<br>python -u mcp/mcp_helix_server.py</pre><p>Terminal 2 (HELIX Client):</p><pre>source venv/bin/activate<br>python agent/helix-with-mcp.py</pre><h4>Testing Different Capabilities</h4><pre># Test MCP tool<br>Bhuvana: greet me<br>HELIX: Hello Bhuvana! Your MCP server is working perfectly.<br><br># Test built-in AWS tool<br>Bhuvana: List my S3 buckets<br>HELIX: [Lists S3 buckets using AWS integration]<br><br># Test built-in HTTP tool<br>Bhuvana: Check https://httpbin.org/get<br>HELIX: [Makes HTTP request and shows response]</pre><h3>Sample Output</h3><p>You will see a total of 3 tools, 2 built-in coming from the agent and 1 from MCP<br>In the example below, the first prompt goes to mcp, and the second one to the built-in tools from the agent<br>When you type “greet me”, HELIX should now successfully:</p><ol><li>Recognize it needs the greet_bhuvana MCP tool</li><li>Execute the tool within the active MCP session</li><li>Return: “Hello Bhuvana! Your MCP server is working perfectly.”</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6C8FVol91_pK687RzAXnYg.png" /></figure><p>See the British wit in the response :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SAD_fwV6ZzXMcR63oApwxA.png" /></figure><h3>Production Considerations</h3><h4>Error Handling and Resilience</h4><p>Our implementation includes several production-ready patterns:</p><ul><li><em>Graceful Fallbacks:</em> If MCP connection fails, HELIX continues with built-in tools</li><li><em>Input Validation:</em> Empty inputs are handled gracefully</li><li><em>Session Management:</em> MCP context is maintained throughout the session</li><li><em>Connection Recovery:</em> Clear error messages help with troubleshooting</li></ul><h4>Common Issues and Solutions</h4><p><strong>Port Conflicts:</strong></p><pre># Find process using port 8000<br>lsof -i :8000<br><br># Kill the process<br>kill -9 &lt;PID&gt;<br><br># Or use a different port in both server and client</pre><h4>MCP Connection Issues:</h4><ul><li>Ensure MCP server is running before starting the client</li><li>Verify network connectivity and port availability</li><li>Check MCP package installation</li></ul><h4>Extending HELIX</h4><p>Adding new capabilities is remarkably simple. Want to add database operations? Create a new MCP server:</p><pre>@mcp.tool(description=&quot;Query user database.&quot;)<br>def query_users(filter: str) -&gt; str:<br> # Database query logic<br> return f&quot;Found users matching: {filter}&quot;</pre><p>Start the server on a different port, and HELIX can discover and use these tools automatically.</p><h3>Performance and Scalability</h3><p>The MCP architecture provides several scalability benefits:</p><ul><li><em>Horizontal Scaling:</em> Run multiple MCP servers for different domains</li><li><em>Load Distribution:</em> Distribute tools across servers based on resource needs</li><li>Independent Deployment: Update tools without touching the core agent</li><li>Resource Isolation: Heavy operations can run on dedicated servers<br>Real-World Applications</li></ul><p>This architecture pattern works for various scenarios:</p><ul><li>Enterprise AI Assistants: Integrate with internal APIs and databases</li><li>Development Tools: Connect to CI/CD pipelines, monitoring systems</li><li>Customer Support: Access CRM, ticketing, and knowledge bases</li><li>Personal Productivity: Calendar, email, task management integration</li></ul><h3>Lessons Learned</h3><p>Building HELIX taught us several key insights:</p><ul><li>Context Management is Critical: MCP sessions must be maintained throughout tool execution</li><li>Graceful Degradation: Always provide fallback options when external services fail</li><li>Developer Experience Matters: Good debugging and error messages save hours</li><li>Personality Enhances Utility: Users engage more with agents that have character</li></ul><h3>Future Enhancements</h3><p>The foundation we’ve built opens up exciting possibilities:</p><ul><li>Multi-Server Discovery: Connect to multiple MCP servers simultaneously</li><li>Dynamic Tool Loading: Hot-reload tools without restarting the agent</li><li>Tool Composition: Chain MCP tools together for complex workflows<br>Security Integration: Add authentication and authorization layers</li></ul><h3>Conclusion</h3><p>Building HELIX demonstrates how Model Context Protocol transforms AI agents from static tools into dynamic, extensible systems. By separating tool capabilities from core agent logic, MCP enables developers to add new functionality without modifying existing code — simply spin up a new MCP server with custom tools.</p><p>The architecture we’ve built here — with its robust error handling, graceful fallbacks, and session management — provides a solid foundation for production AI assistants. Whether you’re integrating AWS services, HTTP APIs, or completely custom tools, the MCP pattern scales elegantly.</p><p>HELIX isn’t just a technical achievement; it’s a glimpse into the future of AI agent development where personality, functionality, and extensibility work together seamlessly. The combination of the Strands framework and MCP creates agents that are both powerful and maintainable.</p><blockquote>Key Takeaway: MCP doesn’t just make AI agents more capable — it makes them more sustainable. As your needs evolve, your agent can grow with you, one tool at a time.</blockquote><p>Ready to build your own extensible AI assistant? The code is ready, the patterns are proven, and the possibilities are endless.</p><p>The complete source code for HELIX is available on <a href="https://github.com/bhuvana-s/strands-mcp.git">GitHub</a>, including setup instructions, testing utilities, and deployment guides. Start building your own MCP-powered AI assistant today!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ba84a44a324" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Weather CLI Tool: A Spec-Driven Development Journey using AWS Kiro]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/building-a-weather-cli-tool-a-spec-driven-development-journey-using-aws-kiro-336e0f4073bf?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/336e0f4073bf</guid>
            <category><![CDATA[spec-driven-development]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[aws-kiro]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Sun, 05 Oct 2025 13:31:32 GMT</pubDate>
            <atom:updated>2025-10-05T13:31:32.667Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How I transformed a simple idea into a production-ready cross-platform weather application using systematic requirements gathering, design, and implementation</em></p><h3>Introduction</h3><p>Have you ever wondered how to turn a rough idea into a well-structured, production-ready application? In this post, I’ll walk you through how to start the journey of building a weather CLI tool using a spec-driven development approach with Kiro AI. What started as a simple request evolved into a comprehensive cross-platform application that fetches weather data for both US and Indian zip codes. If you are new to AWS Kiro, follow the instructions to <a href="https://kiro.dev/">install Kiro and get started</a>.</p><blockquote>The beauty of spec-driven development lies in its systematic approach: <em>requirements → design → implementation → testing.</em></blockquote><p>Let me show you how this methodology helped create a robust weather application step by step.</p><h3>The Initial Spark</h3><p>It all began with a simple idea: <em>“I want a command-line tool to check the weather.”</em></p><p>Instead of jumping straight into coding, I decided to follow a structured approach using Kiro’s spec-driven development workflow. This methodology ensures that every feature is properly planned, designed, and implemented with clear requirements and comprehensive testing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eYXDPqPKXv6am2NEp6m36Q.png" /></figure><h3>Step 1: Requirements Gathering</h3><p>The first step was transforming my rough idea into concrete requirements. Using the EARS (Easy Approach to Requirements Syntax) format, I created structured requirements that clearly defined what the application should do.</p><h3>Initial Requirements</h3><p>The initial requirements document included:</p><p><strong>Requirement 1: Basic Weather Retrieval</strong></p><ul><li><em>User Story</em>: As a user, I want to run a simple command to get current weather information for my default location, so that I can quickly check conditions without having to remember my zip code.</li><li><em>Acceptance Criteria</em>:</li><li>WHEN the script is executed without arguments THEN the system SHALL display the current temperature and chance of rain for a pre-configured default zip code</li><li>WHEN the script runs successfully THEN the system SHALL display the information in a clear, readable format</li></ul><p><strong>Requirement 2: Command-Line Flexibility</strong></p><ul><li><em>User Story</em>: As a traveller, I want to specify a different zip code as a command-line argument, so that I can check the weather for my current location without modifying the script.</li></ul><p><strong>Requirement 3: Cross-Platform Compatibility</strong></p><ul><li><em>User Story</em>: As a user on different operating systems, I want the script to work identically on macOS, Windows, and Linux.</li></ul><h3>Expanding the Scope</h3><p>After reviewing the initial requirements, I realized the application could be more valuable by supporting international locations. This led to an important enhancement:</p><p><strong>Enhanced Requirement 2</strong>: Support for both US zip codes (5 digits) and India pin codes (6 digits), making the tool useful for a broader audience.</p><p>This change demonstrates the iterative nature of requirements gathering — starting simple and expanding based on user needs and market considerations.</p><h3>Step 2: Accepting and Refining Requirements</h3><p>The requirements review process was crucial. Each requirement was evaluated for:</p><ul><li>Clarity and testability</li><li>Technical feasibility</li><li>User value</li><li>Cross-platform compatibility</li></ul><p>Additional requirements emerged during this phase:</p><p><strong>Requirement 4: Error Handling</strong></p><ul><li>Graceful handling of network issues, invalid API keys, and malformed responses</li></ul><p><strong>Requirement 5: Weather Alerts</strong></p><ul><li>Display weather advisories (heat warnings, storm alerts, etc.)</li></ul><p><strong>Requirement 6: Configuration Management</strong></p><ul><li>Persistent storage of default zip codes with first-run setup</li></ul><h3>Step 3: Design Phase</h3><p>With solid requirements in place, I moved to the design phase. This involved:</p><h3>Architecture Decisions</h3><pre>weather_cli.py</pre><pre>├── Configuration Management</pre><pre>├── Weather API Client</pre><pre>├── Command Line Interface</pre><pre>├── Error Handling</pre><pre>└── Output Formatting</pre><p><strong>Key Design Choices:</strong></p><ol><li><strong>Single File Design</strong>: Easy distribution and execution</li><li><strong>OpenWeatherMap API</strong>: Reliable free tier with comprehensive data</li><li><strong>Two-Step API Process</strong>: Geocoding for coordinates, then weather data</li><li><strong>JSON Configuration</strong>: Simple persistent storage in user’s home directory</li><li><strong>Graceful Degradation</strong>: Continue showing weather even if alerts fail</li></ol><h3>Data Models</h3><p>The design included clear data structures:</p><pre>{<br>&quot;temperature_f&quot;: float, # Fahrenheit<br>&quot;temperature_c&quot;: float, # Celsius<br>&quot;feels_like_f&quot;: float,<br>&quot;feels_like_c&quot;: float,<br>&quot;humidity&quot;: int,<br>&quot;description&quot;: str,<br>&quot;rain_chance&quot;: float,<br>&quot;location&quot;: str,<br>&quot;alerts&quot;: [...]<br>}</pre><h3>Step 4: Task List Generation</h3><p>The design was then broken down into 14 specific, actionable implementation tasks:</p><ol><li>Set up project structure and core interfaces</li><li>Implement zip code validation and country detection</li><li>Implement configuration management system</li><li>Create weather API client foundation</li><li>Implement current weather data retrieval</li><li>Implement weather alerts functionality</li><li>Create output formatting system</li><li>Implement command line interface</li><li>Implement first-run configuration setup</li><li>Create main application flow</li><li>Implement comprehensive error handling</li><li>Add cross-platform compatibility features</li><li>Create comprehensive test suite</li><li>Final integration and testing</li></ol><p>Each task included:</p><ul><li>Clear objectives</li><li>Specific requirements references</li><li>Implementation details</li><li>Success criteria</li></ul><h3>Step 5: Implementation Marathon</h3><p>The implementation phase involved systematically working through each task. Here are some highlights:</p><h3>Zip Code Validation</h3><pre>def  validate_zip_code(zip_code: str) -&gt; Tuple[bool, str]:<br>  &quot;&quot;&quot;Validate zip code format and return (is_valid, country_code)&quot;&quot;&quot;<br>  if not zip_code or not zip_code.isdigit():<br>  return False, &quot;&quot;<br>  if  len(zip_code) == 5:<br>  return True, &quot;US&quot;<br>  elif  len(zip_code) == 6:<br>  return True, &quot;IN&quot;<br>  else:<br>  return False, &quot;&quot;</pre><h3>Configuration Management</h3><pre>def  load_config() -&gt; Dict:<br>  &quot;&quot;&quot;Load configuration from ~/.weather_cli_config.json&quot;&quot;&quot;<br>config_path = Path.home() / &quot;.weather_cli_config.json&quot;<br>if not config_path.exists():<br>return {}<br>try:<br>with  open(config_path, &#39;r&#39;) as f:<br>return json.load(f)<br>except (json.JSONDecodeError, IOError):<br>return {}</pre><h3>Error Handling Strategy</h3><p>The application implements comprehensive error handling for:</p><ul><li>Network connectivity issues</li><li>Invalid API keys</li><li>Malformed zip codes</li><li>Service unavailability</li><li>Configuration problems</li></ul><p>Each error provides helpful, actionable feedback to users.</p><h3>Step 6: Testing and Validation</h3><p>A comprehensive test suite was created covering:</p><h3>Unit Tests</h3><ul><li>Zip code validation logic</li><li>Configuration management</li><li>Output formatting</li><li>Temperature conversion</li></ul><h3>Integration Tests</h3><ul><li>API client with mock responses</li><li>End-to-end command line execution</li><li>Cross-platform compatibility</li></ul><pre>class TestZipCodeValidation(unittest.TestCase):<br>  def  test_valid_us_zip_code(self):<br>    is_valid, country = validate_zip_code(&quot;90210&quot;)<br>    self.assertTrue(is_valid)<br>    self.assertEqual(country, &quot;US&quot;)<br><br>  def  test_valid_india_zip_code(self):<br>    is_valid, country = validate_zip_code(&quot;110001&quot;)<br>    self.assertTrue(is_valid)<br>    self.assertEqual(country, &quot;IN&quot;)</pre><h3>Step 7: Feature Enhancement — Dual Temperature Display</h3><p>During development, I realized users would benefit from seeing temperature in both Celsius and Fahrenheit. This led to an interesting challenge: how to implement a feature change that affects multiple layers of the application.</p><h3>The Change Process</h3><ol><li><strong>Updated Requirements</strong>: Modified acceptance criteria to specify dual temperature display</li><li><strong>Design Updates</strong>: Enhanced data models to include both temperature scales</li><li><strong>Implementation Changes</strong>:</li></ol><ul><li>Added temperature conversion utilities</li><li>Updated API response parsing</li><li>Modified output formatting</li></ul><pre>def  fahrenheit_to_celsius(temp_f: float) -&gt; float:<br>  &quot;&quot;&quot;Convert Fahrenheit to Celsius&quot;&quot;&quot;<br>  return (temp_f - 32) * 5.0 / 9.0<br>  <br># Updated output format<br>output.append(f&quot;Temperature: {weather_data[&#39;temperature_f&#39;]:.1f}°F / {weather_data[&#39;temperature_c&#39;]:.1f}°C&quot;)</pre><ol><li><strong>Test Updates</strong>: Modified test cases to verify dual temperature display</li></ol><p>This change demonstrated the value of the spec-driven approach — having clear documentation made it easy to identify all affected components and ensure consistent updates across the entire application.</p><h3>Step 8: Real-World Testing</h3><h3>API Setup</h3><ol><li><strong>Signed up</strong> for OpenWeatherMap free account</li><li><strong>Generated API key</strong> from the dashboard</li><li><strong>Set environment variable</strong>:</li></ol><pre>export OPENWEATHERMAP_API_KEY=your_key_here<br><br>source ~/.zshrc   <br><br>## If you want to reset run <br>## unset OPENWEATHERMAP_API_KEY</pre><h3>Live Testing</h3><p>The application was tested with real zip codes:</p><p><strong>US &amp; India Testing:<br></strong>[Note: YAML is only used for colour codes :) ]</p><pre>Bash Script:<br>./weather_cli.py  90210  # Beverly Hills, CA<br><br>Commandline Output:<br>Fetching weather data for 90210...<br>Warning: Invalid API key for alerts service. Continuing with weather data...<br><br>Weather for Beverly Hills (90210):<br>Temperature: 58.0°F / 14.4°C<br>Feels like: 57.2°F / 14.0°C<br>Conditions: Clear Sky<br>Humidity: 78%<br>Chance of rain: 0%<br><br>No active weather advisories.<br><br></pre><pre>Bash Script:<br>./weather_cli.py 560076 # Bengaluru, India <br><br>Commandline Output:<br>Fetching weather data for 560076...<br>Warning: Invalid API key for alerts service. Continuing with weather data...<br><br>Weather for Kanija Bhavan (560076):<br>Temperature: 80.8°F / 27.1°C<br>Feels like: 83.6°F / 28.7°C<br>Conditions: Scattered Clouds<br>Humidity: 66%<br>Chance of rain: 20%<br><br>No active weather advisories.</pre><h3>Key Learnings and Best Practices</h3><h3>1. Requirements Drive Everything</h3><p>Starting with clear, testable requirements prevented scope creep and ensured every feature had a clear purpose. The EARS format made requirements unambiguous and verifiable.</p><h3>2. Iterative Refinement Works</h3><p>The ability to refine requirements and design before implementation saved significant development time. Changes were much easier to make in the specification phase than in code.</p><h3>3. Comprehensive Error Handling is Critical</h3><p>Real-world applications face network issues, invalid inputs, and service outages. Planning for these scenarios from the design phase resulted in a robust, user-friendly application.</p><h3>4. Cross-Platform Considerations Matter</h3><p>Using pathlib.Path instead of string concatenation, choosing cross-platform libraries, and testing on multiple operating systems ensured broad compatibility.</p><h3>5. Testing Validates Design Decisions</h3><p>The test suite caught several edge cases and validated that the modular design made components easy to test in isolation.</p><h3>Technical Architecture Highlights</h3><h3>Single-File Design</h3><p>The entire application fits in one Python file, making it easy to distribute and execute. This design choice prioritized simplicity over modularity for this specific use case.</p><h3>Two-Step API Strategy</h3><ol><li><strong>Geocoding</strong>: Convert zip code to coordinates</li><li><strong>Weather Data</strong>: Fetch current conditions and alerts using coordinates</li></ol><p>This approach provided more reliable results than direct zip code weather queries.</p><h3>Configuration Strategy</h3><ul><li><strong>Location</strong>: ~/.weather_cli_config.json in user’s home directory</li><li><strong>Format</strong>: Simple JSON for easy manual editing if needed</li><li><strong>Fallback</strong>: Graceful handling of missing or corrupted config files</li></ul><h3>Performance and Reliability</h3><h3>API Efficiency</h3><ul><li><strong>Timeout Handling</strong>: 10-second timeouts prevent hanging</li><li><strong>Error Recovery</strong>: Graceful degradation when alerts are unavailable</li><li><strong>Rate Limiting</strong>: Respectful API usage patterns</li></ul><h3>User Experience</h3><ul><li><strong>First-Run Setup</strong>: Interactive configuration for new users</li><li><strong>Clear Error Messages</strong>: Actionable feedback for common issues</li><li><strong>Consistent Output</strong>: Standardized formatting across all responses</li></ul><h3>Future Enhancements</h3><p>The spec-driven approach makes it easy to identify potential improvements:</p><ol><li><strong>Extended Forecast</strong>: 5-day weather predictions</li><li><strong>Multiple Locations</strong>: Support for multiple saved locations</li><li><strong>Weather History</strong>: Historical weather data queries</li><li><strong>Notification System</strong>: Weather alert notifications</li><li><strong>Configuration Options</strong>: Customizable output formats and units</li></ol><h3>Conclusion</h3><p>Building the weather CLI tool using spec-driven development demonstrated the power of systematic software development. By following the requirements → design → implementation → testing cycle, I created a robust, well-tested application that handles real-world scenarios gracefully.</p><h3>Key Takeaways:</h3><ol><li><strong>Invest in Requirements</strong>: Clear requirements save time and prevent rework</li><li><strong>Design Before Coding</strong>: Thoughtful architecture makes implementation smoother</li><li><strong>Test Comprehensively</strong>: Good tests catch issues early and enable confident changes</li><li><strong>Plan for Failure</strong>: Error handling and edge cases matter in production applications</li><li><strong>Iterate Thoughtfully</strong>: Structured change processes maintain code quality</li></ol><p>The resulting application is not just functional but maintainable, testable, and extensible. Whether you’re building a simple CLI tool or a complex system, the spec-driven development approach provides a solid foundation for success.</p><h3>Try It Yourself</h3><p>The complete source code, tests, and documentation are available in the project repository. To get started:</p><ol><li>Clone the repository</li><li>Install dependencies: pip install requests</li><li>Get your free OpenWeatherMap API key</li><li>Set the environment variable: export OPENWEATHERMAP_API_KEY=your_key</li><li>Run: ./weather_cli.py</li></ol><p>The first run will guide you through the setup process, and you’ll be checking weather conditions in seconds!</p><p><em>Have you tried spec-driven development in your projects? What challenges did you face, and how did structured planning help? Share your experiences in the comments below!</em></p><p>This project was built using Kiro AI’s spec-driven development workflow, demonstrating how AI-assisted development can follow systematic software engineering practices to create production-ready applications and the source code can be found <a href="https://github.com/bhuvana-s/kiro-weather-alert">here</a>.</p><p>Let me know if you would like understand more on how to leverage spec driven development.</p><p><strong>Tags:</strong> #Python #CLI #WeatherAPI #SpecDrivenDevelopment #SoftwareEngineering #Testing #CrossPlatform #OpenWeatherMap</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=336e0f4073bf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI-Powered Developer Productivity with Amazon Q CLI and Kiro: Best Practices for the GenAI Era]]></title>
            <link>https://medium.com/@bhuvaneswari.subramani/ai-powered-developer-productivity-with-amazon-q-cli-and-kiro-best-practices-for-the-genai-era-47a03e78bec8?source=rss-cbbf9d3173c6------2</link>
            <guid isPermaLink="false">https://medium.com/p/47a03e78bec8</guid>
            <category><![CDATA[developer]]></category>
            <category><![CDATA[sdlc]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[productivity]]></category>
            <dc:creator><![CDATA[Bhuvaneswari Subramani]]></dc:creator>
            <pubDate>Mon, 29 Sep 2025 08:29:13 GMT</pubDate>
            <atom:updated>2025-10-06T10:08:49.660Z</atom:updated>
            <content:encoded><![CDATA[<p>AI-driven developer tools like Amazon Q CLI and Kiro are revolutionizing how software teams deliver value — accelerating velocity, enhancing quality, and optimizing workflows across the Software Development Life Cycle (SDLC). This article distils AI-powered developer productivity, focusing on developer time complexity, quality disciplines, best practices, and real-world impact.</p><h3>Rethinking Developer Productivity: Beyond Coding Time</h3><p>Developer productivity goes far beyond lines of code or raw coding hours. Research indicates the median developer writes code just <strong><em>one hour per day</em></strong>. The rest of the workday is filled with architecture, design, communication, mentoring, security checks, environment setup, and meetings. Simply increasing coding hours won’t drive sustainable productivity gains.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zI3CpTqUJ4kpASc7R_MTrQ.png" /><figcaption>Developer Time Complexity Breakdown</figcaption></figure><p>The Microsoft Research paper “<a href="https://arxiv.org/pdf/2502.15287">Time Warp: The Gap Between Developers’ Ideal vs Actual Workweeks in an AI-Driven Era</a>” (Feb 2025) presents a survey of 484 developers in India and the US conducted mid-2024. It reveals significant deviations in how developers ideally want to allocate their time versus how they actually spend their workweeks.</p><p>Key insights include developers preferring to dedicate more time to core activities like coding and system design, and less time to communication, meetings, and routine maintenance tasks. The study finds that as the gap between ideal and actual time allocation widens, both developer productivity and satisfaction decline. This gap marks prime areas where AI-enabled automation could help improve workflows.</p><p>This begs the question: how do teams optimize the entire developer experience, not just code-writing time?</p><h3>Introducing Big O Notation to Conceptualize Productivity</h3><p>To frame this challenge, we borrow the Big O notation concept from algorithmic analysis. Traditionally, Big O notation describes how an algorithm’s runtime scales with input size. Here, it serves as a metaphor for productivity scaling with task complexity.</p><ul><li>O(1) — Constant time: Ideal scenario where productivity improvements remain steady regardless of workload size.</li><li>O(n) — Linear time: Productivity scales proportionally with workload.</li><li>O(n²) — Quadratic time: Productivity drastically slows as complexity grows without intervention.</li></ul><p>With AI-powered tools like Amazon Q CLI and Kiro, teams can shift productivity growth closer to O(1) by automating routine tasks, reducing context switching, and ensuring quality gates — moving away from inefficient linear or worse productivity slowdowns.</p><h3>Decoding Developer Time Complexity: Where Is Time Spent?</h3><p>The average developer’s time allocation:</p><ul><li>25% coding</li><li>15% architecture and design</li><li>Remaining 60% split among communication, mentoring, meetings, tooling management, security, and compliance.</li></ul><blockquote>Improving productivity means reclaiming that 60% for higher-value work without burning out developers.</blockquote><h3>Amazon Q CLI and Kiro in Action</h3><h3>Amazon Q CLI</h3><p>Amazon Q CLI integrates AI automation into the IDE workflow, unifying communications across version control, task management, build monitoring, and incident management tools. It uses Model Context Protocols (MCP) and rules to produce code that aligns with organizational standards, internal components, and security policies.</p><h3>Amazon Kiro</h3><p>Amazon Kiro augments the ecosystem by automating compliance checks, code formatting, and enforcing coding standards across teams. Together, they help:</p><ul><li>Automate repetitive and error-prone tasks.</li><li>Enforce quality gates across development milestones.</li><li>Maintain prompt libraries to ensure consistent AI behaviour and knowledge retention.</li></ul><h3>Metrics and Frameworks for Measuring Developer Productivity</h3><p>Teams increasingly use structured frameworks to understand and improve developer output and satisfaction:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ir6wPxQIG1WorxPA7SJprg.png" /></figure><p>These metrics provide balanced insights into velocity, quality, morale, and collaboration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RZGo1kpmgKpuxXDiQ7y3-Q.png" /><figcaption>Developer Productivity Frameworks (DORA, SPACE, DevEx)</figcaption></figure><p>Productivity, while in a generic context translates into work done in time. Instead, developer productivity has to be measured through various other factors, such as team collaboration, quality of work, velocity and satisfactory or experience, in addition to the unit of work and time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pmKxd7wF1cD6st1lUbrxUA.png" /></figure><h3>Evolution of IDEs</h3><p>The developer benefits gained in each decade from the evolution of IDEs:</p><ul><li>1960s-70s: Simplified basic code writing and manual compilation, reducing fragmented workflows for individual developers.</li><li>1980s-90s: Integrated essential tools like debuggers and version control, streamlining development processes and minimizing tool switching.</li><li>2000s: Enhanced team collaboration and code quality with features like code reviews, refactoring, and advanced debugging for maintainable projects.</li><li>2010s: Enabled remote work and scalable development through cloud-based IDEs and real-time collaboration support, aiding distributed teams.</li><li>2020s: Introduced AI-driven code automation, smart completions, and intelligent analysis to boost productivity, reduce cognitive load, and deliver higher-quality code faster.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kKVrDo9Oy_Tn_NzwpPCM0w.png" /></figure><h3>Best Practices for Integrating AI Across SDLC</h3><p>The following recommendations can unlock productivity and maintain quality:</p><ul><li><em>Adopt Model Context Protocols (MCP):</em> Seamlessly connect external context and internal organizational knowledge within IDEs.</li><li><em>Implement Rules and Steering:</em> Guide AI to follow naming conventions, use internal APIs, and respect versioning.</li><li><em>Set Automated Quality Gates: </em>Enforce pre-commit compilation checks, vulnerability scanners, test coverage minimums, and consistent code comments.</li><li><em>Curate Prompt Libraries:</em> Regularly refine AI prompts to improve output quality tailored to your team’s needs.</li></ul><p>First, let us solve for <strong>flow</strong>, tools communication, and application context. Alongside IDEs, developers use various tools throughout the software development lifecycle, each serving a vital purpose. However, managing data and maintaining consistency across these tools leads to frequent context switching, which disrupts developer flow. This interruption can cause quality issues, stale data, and communication delays.</p><p>Second, we address subjective <strong>organizational context</strong>. It is important to ensure that code generation:</p><ol><li>Adheres to organizational standards,</li><li>Follows naming conventions,</li><li>Uses specific software versions,</li><li>Leverages internal reusable components or APIs.</li></ol><p>This is enforced through rules, also known as steering mechanisms.</p><p>Third, we tackle objective <strong>quality concerns</strong> by ensuring that:</p><ol><li>Code commits follow a standardized comment format,</li><li>Uncompiled or broken code is prevented from entering version control,</li><li>Test coverage meets prescribed levels,</li><li>Code is free from critical security vulnerabilities.</li></ol><p>These are enforced using <strong>automated hooks and quality gates.</strong></p><p>Finally, recognize that there is <strong>no one-size-fits-all approach</strong>. Iteration is key to discovering what works best for your organization, team, and project. Build and manage a tailored prompt library to consistently generate high-quality results aligned with your specific needs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qtZELgOjOEAuSIBofODEBA.gif" /></figure><h3>What the Future Holds: AI-Powered Developer Experience</h3><p>With AI-powered IDEs offering unified communication and automation, developer productivity can approach near constant time (O(1)) growth despite increasing complexity. Such setups will reduce cognitive overhead, developer fatigue, and rework, empowering teams to code faster and better.</p><h3>Recap: Be effective with coding assistants</h3><ul><li>Build a prompt library to consistently generate high-quality outcomes.</li><li>Reimagine the SDLC by integrating tools directly within the IDE using MCP.</li><li>Enforce organizational standards in code generation through rules and steering mechanisms.</li><li>Ensure quality at every development milestone using automated hooks.</li></ul><p>If you wonder how this flows??</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QXMp_CrR_NtNNwP4O1cGOQ.png" /></figure><h3>Demo</h3><p>See how Model Context Protocols enable real-time organization knowledge, how rules steer AI code generation, and automated hooks enforce quality standards in your daily workflow.</p><h4>Demo 1: MCP to get external context using Amazon Kiro</h4><p>First, integrate an MCP tool with a test Jira Cloud setup. Begin by listing the existing Jira projects, which returns four projects spanning business and software types.</p><p>Second, select one project — for example, “JCIN” (JC Integration) — and list all the tasks under it along with their descriptions. This step demonstrates how the connection is established and how data is read from Jira.</p><p>Third, instruct the MCP tool to create a new task within the “JCIN” project, titled “Review the MCP Integration.”</p><p>Fourth, once the new task is created, command the tool to assign this task to yourself, completing the workflow of task management via natural language through the MCP integration.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F0r92oxG7u-4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D0r92oxG7u-4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F0r92oxG7u-4%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/f09fbfa36125ff9cbb5ce862918ce214/href">https://medium.com/media/f09fbfa36125ff9cbb5ce862918ce214/href</a></iframe><h4>Demo 2: Rules to ensure AI adheres to desired standards using <br> Amazon Q CLI on VSCode</h4><p>Here are the step-by-step demo instructions for installing and configuring Amazon Q CLI with rules enforcement in VSCode:</p><ol><li>Install and configure the Amazon Q CLI extension in Visual Studio Code.</li><li>Create a file named my-project-rule.md inside the .amazonq/rules directory. In this rules file, specify that the application is being built for games, and instruct the code generator to prefix function names with the name of a superhero or comic character that best fits the function’s purpose.</li><li>For example, if a function performs depth analysis, logical operations, and problem solving, suggest a method name like TWILIGHTSPARKLE_performAnalysis(), inspired by the character Twilight Sparkle from &quot;My Little Pony.&quot;</li><li>Next, instruct Amazon Q CLI to create a Python program related to Environmental, Social, and Governance (ESG). Observe that during code generation, function names are automatically prefixed according to the previously defined superhero naming rule, demonstrating that the rule enforcement works seamlessly during creation.</li></ol><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D2Zr4Q1yK4q0&amp;type=text%2Fhtml&amp;schema=google&amp;display_name=YouTube&amp;src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F2Zr4Q1yK4q0" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2929dbc0d44e040b6073f09915c4026d/href">https://medium.com/media/2929dbc0d44e040b6073f09915c4026d/href</a></iframe><h4>Demo 3: Establish automations that respond to events using Amazon Kiro</h4><p>Here are steps for creating an agent hook in Amazon Kiro:</p><ol><li>Create an agent hook in Amazon Kiro using natural language to enforce function naming conventions.</li><li>Define the rule so that every function in Python files starts with the name of a superhero that best describes the function’s capability. Name this hook “Superhero Function Naming”.</li><li>Set the hook to automatically trigger on any Python file that is modified or saved.</li><li>Give a prompt such as: “Create a utility.py file implementing bubble sort, merge sort, and insertion sort.”</li><li>Initially, function names are generated with regular capability-based names. However, upon saving the file, the hook automatically prefixes each function name with an appropriate superhero name, enforcing the naming rule in real time.</li></ol><p>This demonstrates how Kiro’s agent hooks enable seamless, automated enforcement of coding standards using natural language instructions.</p><iframe src="https://cdn.embedly.com/widgets/media.html?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DFAMZeWhcnAk&amp;type=text%2Fhtml&amp;schema=google&amp;display_name=YouTube&amp;src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FFAMZeWhcnAk" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e03c1f7677fce938a5fba5ffe4095dcb/href">https://medium.com/media/e03c1f7677fce938a5fba5ffe4095dcb/href</a></iframe><h3>Conclusion</h3><blockquote>With a novel idea, anyone can code and build applications. AI powered IDEs make the experience better by offering seamless communication to all tools in the ecosystem</blockquote><p>Artificial intelligence is transforming the software development landscape. Tools like Amazon Q CLI and Kiro do more than generate code; they advance a culture of productivity through disciplined automation, collaboration, and quality management. By applying best practices, measuring with modern metrics, and investing in AI-driven workflows, developer teams can unlock unprecedented efficiency and software excellence</p><p>Thanks to <a href="https://www.linkedin.com/in/chandrasekaran-sivaraman/">Chandrasekaran Sivaraman</a> for all the great brainstorming sessions / demos while building this mental model on Developer Productivity.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47a03e78bec8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>