<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pinaki Brahma on Medium]]></title>
        <description><![CDATA[Stories by Pinaki Brahma on Medium]]></description>
        <link>https://medium.com/@pinaki.brahma?source=rss-a9b1b8d48e87------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Vy8ymp7FvAgTISo6.jpg</url>
            <title>Stories by Pinaki Brahma on Medium</title>
            <link>https://medium.com/@pinaki.brahma?source=rss-a9b1b8d48e87------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 21:11:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pinaki.brahma/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Survive a Corporate Re-Org Without Losing Your Mind (or Team)]]></title>
            <link>https://medium.com/@pinaki.brahma/how-to-survive-a-corporate-re-org-without-losing-your-mind-or-team-9a2c7449681a?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a2c7449681a</guid>
            <category><![CDATA[leadership]]></category>
            <category><![CDATA[change-management]]></category>
            <category><![CDATA[corporate-culture]]></category>
            <category><![CDATA[strategy]]></category>
            <category><![CDATA[team]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Tue, 04 Aug 2026 03:01:02 GMT</pubDate>
            <atom:updated>2026-08-04T03:01:02.681Z</atom:updated>
            <content:encoded><![CDATA[<h4>A practical survival guide for managers and leads navigating sudden team shifts, chaotic handoffs, and workplace uncertainty.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4kpJK6fvj2bT0BSilP_TQQ.png" /><figcaption>Image: Humorous take on “re-org” in corporate. Created using Gemini based on author’s prompt.</figcaption></figure><p>In a re-org, a team swap is not just a staffing move. It is a test of whether the company can preserve context, ownership, and execution speed while people are moving across reporting lines.</p><blockquote>Corporate re-orgs are rarely about org charts alone. They are about strategy, accountability, and who gets to make decisions when the structure changes. And the most disruptive moment is often not the announcement. It is the team swap.</blockquote><p>A team swap sounds simple: one group moves under a new leader, or people are redistributed across functions. In practice, it creates friction in three places at once: people lose familiarity, processes lose clarity, and delivery loses momentum.</p><p>The mistake many leaders make is treating this as a people problem. It is not. It is an operating model problem.</p><h3>Why team swaps create hidden risk</h3><p>When a team moves, the obvious change is reporting structure. The deeper change is context. People carry informal knowledge that does not live in documents: customer issues, delivery history, decision rationale, and working relationships. That context is often the real asset of a team.</p><p>If that context is not protected, the re-org creates three predictable failures:</p><ul><li>decision-making becomes slower because ownership is unclear</li><li>delivery quality drops because critical knowledge is fragmented</li><li>morale falls because people feel that their work is being rearranged without a clear reason</li></ul><p>In other words, the company does not just lose structure. It loses momentum.</p><h3>What leaders should learn</h3><h4>1. Do not confuse re-orgs with role changes</h4><p>A re-org is not only about who reports to whom. It is also about what stays stable while the map changes. The best leaders define what must remain intact: customer commitments, product priorities, escalation paths, and core operating rituals.</p><h4>2. Make ownership explicit before the move</h4><p>This is where most re-orgs fail. If a team is moved, the new structure must answer three questions immediately:</p><ul><li>Who owns the outcome?</li><li>Who makes the call when priorities conflict?</li><li>Who is accountable for cross-team dependencies?</li></ul><p>If those answers are fuzzy, the move creates confusion instead of alignment.</p><h4>3. Protect knowledge, not just headcount</h4><p>A team swap should include a deliberate knowledge transfer plan. That means more than handing over a wiki. It means preserving the practical intelligence that keeps work moving.</p><p>Useful practices include:</p><ul><li>a short transition document with key dependencies</li><li>named owners for critical processes and decisions</li><li>a clear list of open priorities and risks</li><li>a handoff window where the old and new teams overlap</li></ul><h4>4. Treat transition speed as a metric</h4><p>Many organizations announce a re-org and assume people will adapt naturally. That is rarely true. Leaders should monitor the transition like they would monitor any other business change: cycle time, incident rates, decision latency, and employee confidence.</p><p>A good re-org does not just move people. It preserves execution quality during the move.</p><h3>A practical mindset for leaders</h3><p>The best leaders approach re-orgs with a simple rule: keep the business running while the structure changes. That means communicating clearly, reducing ambiguity, and protecting continuity.</p><p>Here is a useful way to think about it:</p><pre>| What changes | What should stay stable |<br>|---|---|<br>| Reporting lines | Customer commitments |<br>| Team boundaries | Execution priorities |<br>| Leadership ownership | Escalation paths |<br>| Working relationships | Operating cadence |</pre><p>What changesWhat should stay stableReporting linesCustomer commitmentsTeam boundariesExecution prioritiesLeadership ownershipEscalation pathsWorking relationshipsOperating cadence</p><p>If the company can preserve the stable parts while absorbing the change, the re-org becomes a strategic reset. If it cannot, the re-org becomes a costly distraction.</p><p>In the end, team swaps in re-orgs are less about hierarchy and more about trust. When leaders protect context, define ownership, and manage the transition with discipline, the business does not just survive the change. It becomes more resilient because of it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a2c7449681a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TurboQuant: How Google Quietly Changed the Economics of Running LLMs]]></title>
            <link>https://medium.com/@pinaki.brahma/turboquant-how-google-quietly-changed-the-economics-of-running-llms-1dabedce08d5?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/1dabedce08d5</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[kv-cache]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[quantization]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Sat, 11 Apr 2026 12:21:40 GMT</pubDate>
            <atom:updated>2026-04-11T12:21:40.103Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hLUVMxKHjoFmWqXcasJdcQ.png" /><figcaption>Image: Turbo Quant as introduced by Google. Rendition using Gemini based on author’s prompt.</figcaption></figure><h3>Introduction</h3><p>If you have ever tried running a large language model in production, you know the drill. The model weights are just the beginning. The moment users start having real conversations — long ones, the kind that span hundreds of turns, entire documents, code repositories — you run headlong into a wall called the <strong>KV Cache</strong>.</p><p>On March 25, 2026, Google Research published a blog post and two peer-reviewed papers that landed quietly but hit hard. No press conference. No product launch event. Just a research update. And within hours, memory chip stocks across Asia and the US were in freefall.</p><p>The research was called <strong>TurboQuant</strong>, and it claims to do something that engineers have been told for years is a zero-sum game: compress AI working memory by 6x, speed up inference by 8x on NVIDIA H100 GPUs, and suffer absolutely zero accuracy degradation — all without touching the model weights.</p><p>For those of us who spend our days trying to keep inference costs in check, this deserves a very close look.</p><h3>The Elephant in the GPU Room: What is the KV Cache?</h3><p>Before we understand the solution, we need to feel the pain.</p><p>When an LLM generates text, it does not simply “think” in isolation. Every word it generates needs to be informed by every word that came before it — in the prompt and in the conversation history. The model uses a mechanism called <strong>attention</strong> to do this: for each new token being generated, it looks back at all previous tokens and decides how much to “attend” to each one.</p><p>The intermediate computations from those lookups — specifically, the <strong>Key (K)</strong> and <strong>Value (V)</strong> matrices computed for every previous token — are cached in memory so they don’t have to be recomputed from scratch at every step. This cache is the <strong>KV Cache</strong>.</p><p>Think of it like a detective’s case board. Every clue gets pinned up there, and the detective consults the whole board every time before drawing the next conclusion. The longer the case, the bigger the board.</p><p>Now imagine running 500 detectives simultaneously on a single machine.</p><p>Here’s the brutal math: for a 100,000-token conversation with a large model, the KV Cache alone can demand <strong>~100 GB of GPU memory</strong>. That is multiple top-of-the-line A100s for a single chat session. When you are serving thousands of users concurrently, this becomes the single biggest cost driver in your infrastructure.</p><h3>Why Old Quantization Wasn’t Enough</h3><p>The obvious fix has always been <strong>quantization</strong> — representing each cached value with fewer bits. Instead of 32-bit or 16-bit floating point numbers, why not use 8-bit integers? Or 4-bit?</p><p>Several techniques have tried exactly this. The most notable are:</p><ul><li><strong>Standard INT8 Quantization</strong> — compresses 16-bit values to 8-bit, giving roughly 2x savings. Works reasonably well but requires storing <em>quantization constants</em> (scale factors) alongside every compressed block. This overhead quietly eats back a significant chunk of your savings.</li><li><strong>KIVI (INT4 KV Compression)</strong> — pushes further to 4-bit, getting to roughly 4x compression. Still needs zero-point constants per channel. Also, fine-tuning variants require you to modify model weights, meaning every base model update forces you to redo the compression work.</li><li><strong>KVSharer / Pruning approaches</strong> — achieve 2–3x savings through structural pruning but require fine-tuning and come with moderate accuracy degradation.</li></ul><p>The pattern is consistent: every prior technique forces a tradeoff. Compress harder → store more overhead metadata → lose accuracy → require retraining. You end up picking two of the three: speed, quality, cost.</p><p>The overhead problem is not a minor footnote. When you store scale factors alongside every compressed block, the net compression ratio drops dramatically. You might aim for 80% savings and end up with 30% after the metadata tax. The cure has a co-pay.</p><blockquote>If you like this article so far, do not forget to <strong>hit 10 claps</strong>. This will help me get the feedback on what my readers like and what they don’t. And in turn, this will help me focus on writing articles that matter to my readers.</blockquote><h3>Enter TurboQuant: The Two-Stage Pipeline</h3><p>TurboQuant is a <strong>training-free, data-oblivious KV cache compression algorithm</strong> that compresses each value from 16–32 bits down to just <strong>3 bits</strong>, with no scale factors to store, no retraining needed, and no measurable accuracy loss.</p><p>It is the capstone of a three-paper research arc:</p><ol><li><strong>QJL</strong> — presented at AAAI 2025, introduced the idea of using 1-bit random projections to correct inner-product bias after quantization.</li><li><strong>PolarQuant</strong> — accepted at AISTATS 2026, introduced coordinate-geometry-based compression that eliminates the need to store quantization constants entirely.</li><li><strong>TurboQuant (PolarQuant + QJL)</strong> — accepted at ICLR 2026, combines both into a single pipeline that achieves 6x compression with 8x inference speedup.</li></ol><p>The key insight is deceptively elegant: <strong>if you change your coordinate system before compressing, the overhead disappears on its own.</strong></p><h3>Stage 1 — PolarQuant: Geometry to the Rescue</h3><p>Every KV cache value is a high-dimensional vector — a list of floating-point numbers. In the standard Cartesian representation (x, y, z, …), to compress these numbers, you need to store a scale factor per block, so the decompressor knows how to “undo” the compression. That is the overhead everyone has been struggling with.</p><p>PolarQuant makes a different choice. It takes pairs of coordinates and converts them from Cartesian to <strong>polar coordinates</strong> — a radius and an angle. Here is the key mathematical insight: when the input data has been normalized (as it typically is in attention layers), <strong>the angles in polar space are uniformly distributed</strong>. A uniform distribution is the most information-theoretically efficient distribution to quantize — because every bucket is equally likely, and you need no extra constants to describe the distribution.</p><p>In other words, the uniform angle distribution means the quantization grid perfectly matches the data distribution by construction. No scale constants needed. The math <em>is</em> the compression format.</p><p>The radius component is handled separately — it gets carried forward to the next stage in a compact form.</p><h3>Stage 2 — QJL: Johnson-Lindenstrauss Meets 1-Bit Quantization</h3><p>After PolarQuant, the remaining information is passed through <strong>QJL (Quantized Johnson-Lindenstrauss)</strong>. This uses the classical Johnson-Lindenstrauss Transform — a theorem from the 1980s originally designed for dimensionality reduction in pure mathematics — and applies it with a twist: it projects the data into a lower-dimensional space and then keeps only the <strong>sign</strong> of each projected dimension: +1 or -1. Just 1 bit per projected dimension.</p><p>The mathematical guarantee of the JL transform is that even after this aggressive projection, <strong>the relative distances (dot products) between vectors are preserved within provable error bounds</strong>. Since the attention computation depends entirely on these inner products, preserving them means preserving the model’s reasoning.</p><p>Combined, PolarQuant eliminates overhead constants and QJL performs deep, theoretically grounded compression. The pipeline takes raw 32-bit KV cache values in, passes them through these two stages, and spits out 3 bits. No lookup tables. No stored metadata. No decompressor constants.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/1*9mrducVKpGB-AHOFeFbErg.png" /><figcaption>Table: How the comparison stands</figcaption></figure><p>On NVIDIA H100 GPUs, TurboQuant delivers up to <strong>8x speedup in attention computation</strong>. The compounding effects for enterprise workloads are significant:</p><ul><li>Serve <strong>6x more concurrent users</strong> on the same hardware</li><li>Run <strong>100K+ token contexts</strong> on GPUs that previously couldn’t support them</li><li>Achieve <strong>50%+ cost reduction</strong> in inference spend</li><li>Zero changes to model weights or training pipelines</li></ul><blockquote>If you like this article so far, do not forget to <strong>hit 10 claps</strong>. This will help me get the feedback on what my readers like and what they don’t. And in turn, this will help me focus on writing articles that matter to my readers.</blockquote><h3>Let’s Get Our Hands Dirty: Simulating TurboQuant in Python</h3><p>Let me show you a working simulation of the two-stage pipeline. This is not the full production implementation (that requires CUDA kernels), but it will give you an intuitive grasp of what each stage does to a vector. To get your hands on the code, visit <a href="https://github.com/datasciencewithpinaki/paper2code/blob/main/turboQuant.py">this repo</a>.</p><h3>What This Means in Practice</h3><p><strong>For teams running inference in production</strong>, TurboQuant is a drop-in optimization that requires no model retraining, no dataset-specific tuning, and no changes to your serving infrastructure beyond swapping the KV cache backend. The 50% cost reduction claim is not hypothetical — it follows directly from the 6x memory reduction and 8x attention speedup on the same hardware you are already running.</p><p><strong>For teams using cloud LLM APIs</strong>, this will gradually translate into lower token costs and better throughput for long-context workloads, as providers adopt the technique on the backend.</p><p><strong>For teams trying to self-host large models on constrained hardware</strong>, TurboQuant potentially unlocks 100K+ token context windows on GPUs that were previously too small for the task.</p><p><strong>For teams in cost-sensitive verticals</strong> — insurance, retail, airlines, healthcare — where you are running thousands of concurrent sessions with domain-specific long documents, the economics here compound fast. A 50% cut to your GPU bill is not a rounding error; it is a strategic shift in what is feasible to build.</p><h3>Conclusion</h3><p>TurboQuant is a rare thing in applied AI research: a result that is simultaneously theoretically grounded, practically deployable today, and economically significant at scale. The key intellectual contribution — converting to polar coordinates before quantizing, so the data distribution becomes uniform and overhead constants become unnecessary — is the kind of insight that, in hindsight, makes you wonder why it took this long.</p><p>The combination of PolarQuant and QJL is elegant. PolarQuant solves the overhead problem using geometry. QJL solves the depth problem using a 40-year-old theorem from mathematics. Together, they push a 32-bit KV value all the way down to 3 bits without asking the model to unlearn anything it already knows.</p><p>Whether you are an ML engineer optimising inference infrastructure, a data scientist exploring LLMs for enterprise use cases, or a researcher thinking about the next wave of efficiency gains, TurboQuant is worth understanding deeply. The fact that it is available as open-source and royalty-free makes the barrier to adoption essentially zero.</p><blockquote>If you like this article so far, do not forget to <strong>hit 10 claps</strong>. This will help me get the feedback on what my readers like and what they don’t. And in turn, this will help me focus on writing articles that matter to my readers.</blockquote><h3>Bonus Section :: Action Items for the Reader</h3><ol><li><strong>Read the original papers.</strong> The TurboQuant paper (ICLR 2026), PolarQuant (AISTATS 2026), and QJL (AAAI 2025) are all publicly available. The math is approachable if you have a linear algebra background — and the proofs are genuinely satisfying.</li><li><strong>Run the Python simulation above.</strong> Plug in your actual model’s vector dimension and your typical context length to see the concrete memory savings for your workload.</li><li><strong>Benchmark your current inference stack.</strong> Establish a baseline for your KV cache memory consumption, attention throughput, and cost per 1M tokens today. That baseline will help you quantify the impact if and when you integrate TurboQuant.</li><li><strong>Think beyond LLMs.</strong> The paper notes that TurboQuant also applies to vector search (RAG pipelines, embedding stores). If you are running a retrieval-augmented system, there may be a second win hiding here beyond just inference memory.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1dabedce08d5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Was the Quiet MVP of the 2025 US Holiday Shopping Season]]></title>
            <link>https://medium.com/@pinaki.brahma/ai-was-the-quiet-mvp-of-the-2025-us-holiday-shopping-season-dc89ab50b10b?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc89ab50b10b</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[retail-technology]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Sun, 04 Jan 2026 07:14:09 GMT</pubDate>
            <atom:updated>2026-01-04T07:14:09.502Z</atom:updated>
            <content:encoded><![CDATA[<p><em>And most people still think it’s just a chatbot.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1i7Kj0gkiHxURWMv.jpg" /><figcaption>Image: Retail outlet in the US (Source: abc.com)</figcaption></figure><p>I have been reading reports on how <strong>AI actually showed up in US retail this holiday season</strong>, especially in electronics and accessories. Not the hype decks. Not the “AI will change everything” think pieces. But <em>real numbers</em> tied to customer behavior and dollars.</p><p>What stood out was not that AI was present — it was <strong>how invisibly powerful it had become</strong>.</p><p>Here are a few data points that stopped me mid-scroll:</p><ul><li>Customers who used <strong>AI-powered search</strong> saw <strong>conversion lifts as high as 38%</strong> versus those who didn’t</li><li><strong>Average basket size increased materially</strong> when AI assisted the journey</li><li><strong>$3B in sales</strong> were directly driven by AI agents on <strong>Black Friday alone</strong></li><li>Retailers with <strong>chatbots saw a 9% lift in conversion</strong></li><li><strong>42% of shoppers used AI for gift buying</strong></li><li><strong>70% of GenAI-assisted shoppers reported a better overall experience</strong></li></ul><p>These aren’t experiments anymore. This is production-grade impact.</p><h3>The “Shopping Mall GPS” Moment</h3><p>The simplest mental model I’ve landed on is this:</p><blockquote><strong><em>Generative AI became a GPS for shopping.</em></strong></blockquote><p>Traditional retail search is like walking into a massive mall with a vague store directory:</p><ul><li>Too many aisles</li><li>Too many filters</li><li>Too many links</li><li>Too much cognitive load</li></ul><p>In 2025, shoppers stopped <em>wandering</em>.</p><p>They simply said:</p><blockquote><em>“I need a noise-cancelling headset, under $300, for travel, with great battery life.”</em></blockquote><p>And AI took them <strong>straight there</strong>.</p><ul><li>No endless scrolling.</li><li>No 15 tabs open.</li><li>No comparison fatigue.</li></ul><p>Just intent → outcome.</p><p>This shift from <em>navigation</em> to <em>destination </em>is the real disruption.</p><h3>Why This Matters (Beyond the Hype)</h3><p>What’s happening is deeper than chatbots or recommendation widgets:</p><ul><li><strong>Search is becoming conversational</strong></li><li><strong>Discovery is becoming intent-led</strong></li><li><strong>Decision-making is being compressed</strong></li><li><strong>Experience is finally measurable, not anecdotal</strong></li></ul><p>And most importantly: AI is no longer optimizing <em>clicks</em>. It’s optimizing <strong>confidence</strong>. Confident shoppers convert faster, buy more, and come back.</p><h3>A Thought for Leaders Still on the Fence</h3><p>If you’re still debating whether AI is “another bubble,” here’s the uncomfortable truth:</p><blockquote><em>By the time something shows up in holiday sales at this scale, the early-mover advantage is already gone.</em></blockquote><p>What’s left is execution:</p><ul><li>Where do you inject AI in the journey?</li><li>Which moments remove friction?</li><li>Which ones amplify trust?</li><li>And which metrics actually matter beyond CTR?</li></ul><p>For those already building — exciting times ahead.<br>For those still debating — contemplate quickly, lest you miss the bus.</p><p>I’m curious:<br><strong>Where have you seen AI genuinely improve customer experience — not demos, but real impact?</strong></p><p>If you’re thinking deeply about AI in retail, search, discovery, or decision science, let’s connect.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc89ab50b10b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Search Bar: A Deep Dive into the E-commerce Search Engine]]></title>
            <link>https://medium.com/@pinaki.brahma/beyond-the-search-bar-a-deep-dive-into-the-e-commerce-search-engine-cf8457602634?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/cf8457602634</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[ecommerce]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Tue, 23 Dec 2025 14:15:21 GMT</pubDate>
            <atom:updated>2025-12-23T14:15:21.926Z</atom:updated>
            <content:encoded><![CDATA[<p>As a technical writer who has spent years at the intersection of business analysis and data science, I’ve come to hold one truth as self-evident: you cannot solve a business problem you don’t understand. The most sophisticated algorithm is useless if it’s optimizing for the wrong outcome. For any aspiring data professional, my advice is always the same: before you import a single library, first import the domain knowledge.</p><p>Today, we’re applying this principle to one of the most critical, data-intensive, and often underestimated components of online retail: the <strong>e-commerce search engine</strong>. It’s not just a text box. It’s the digital store’s concierge, its most knowledgeable salesperson, and its primary engine for discovery and revenue. Consider this: industry studies consistently show that visitors who use the on-site search bar convert at a rate <strong>2–4 times higher</strong> than those who don’t. This isn’t a marginal feature; it’s a mission-critical system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PjNMOdcpBwwbYGJLoTNhfg.png" /><figcaption>Image: Search Box in eCommerce ; Image generated by Gemini based on writer’s prompt.</figcaption></figure><h3>A High-Level Overview of the E-commerce Business</h3><p>At its core, the e-commerce business model seems simple. Let’s trace the journey with a fictional online store, “Aura Gadgets”:</p><ol><li><strong>Sourcing &amp; Inventory:</strong> Aura Gadgets partners with manufacturers to acquire a <em>catalog</em> of the latest headphones, smartwatches, and chargers.</li><li><strong>Digital Storefront:</strong> They build a website displaying these products with high-quality images, detailed specifications, and prices.</li><li><strong>Customer Acquisition:</strong> Through targeted ads and tech blog reviews, Aura Gadgets attracts visitors to their site. A potential customer, looking for new earbuds, lands on their homepage.</li><li><strong>Conversion:</strong> This is the moment of truth. The visitor types “noise-canceling earbuds for running” into the search bar. The site must guide them from this initial intent to a completed purchase in the <em>checkout funnel</em>.</li><li><strong>Fulfillment:</strong> After the purchase, Aura Gadgets ships the product to the customer and manages the logistics of delivery and potential returns.</li></ol><p>While every step is vital, the bridge between <strong>Acquisition</strong> and <strong>Conversion</strong> is where the magic — or the misery — happens. A customer has arrived. They have an <em>intent</em>. The search engine is the primary tool for translating that intent into a purchase.</p><h3>The Role of Search: From Utility to a Strategic Powerhouse</h3><p>In the physical world, a customer might ask a store associate, “Where can I find waterproof running shoes for men, size 10, maybe in blue?” The associate uses their knowledge of the store’s inventory, current promotions, and popular brands to guide the customer.</p><p>The e-commerce search engine is that digital associate, but supercharged with data. Its job is to understand a user’s query — no matter how vague or specific — and return a relevant, compelling, and intelligently ranked list of products from a potentially massive catalog.</p><p>The business impact is staggering. According to Forrester Research, up to <strong>30% of visitors</strong> will use the on-site search bar, and these users can account for as much as <strong>45% of an e-commerce site’s revenue</strong>. A great search experience drives <strong>conversion</strong>, increases <strong>Average Order Value (AOV)</strong> by suggesting relevant upsells, and builds <strong>customer loyalty</strong>. A poor one, filled with irrelevant results or the dreaded “no results found” page, leads directly to bounces, frustration, and lost revenue.</p><h3>The Anatomy of a Modern Search Engine</h3><p>Let’s break down what happens in the milliseconds between a user hitting “Enter” and seeing a page of results. This isn’t a simple database SELECT query; it&#39;s a sophisticated, multi-stage pipeline.</p><ol><li><strong>Query Understanding (The Brain):</strong> This is where we decipher the user’s <em>intent</em>.</li></ol><ul><li><strong>Terminologies:<br>&gt;&gt; Query Parsing &amp; Tokenization:</strong> Breaking down “blue running shoes” into tokens: [blue, running, shoes].<br>&gt;&gt; <strong>Stemming &amp; Lemmatization:</strong> Understanding that “running,” “ran,” and “runs” all relate to the root word “run.”<br>&gt;&gt; <strong>Synonym Expansion:</strong> Knowing that “sneakers” and “trainers” are synonyms for “shoes.” This is crucial for avoiding a dreaded <strong>zero-results page</strong>.<br>&gt;&gt; <strong>Named Entity Recognition (NER):</strong> Identifying [color: blue], [category: shoes], and potentially brands like [brand: Nike] even if they are typed in different orders.</li><li><strong>Tech Flavor:</strong> This is a classic Natural Language Processing (NLP) task. While older systems relied on manually curated synonym lists, modern engines use word embedding models (like Word2Vec or FastText) or even powerful transformer-based models (like BERT) to understand semantic relationships. The Hugging Face Transformers library in Python has become the go-to for implementing these state-of-the-art techniques.</li></ul><p><strong>2. Information Retrieval (The Librarian):</strong> This is the initial “fetch” stage.</p><ul><li><strong>Terminologies:<br>&gt;&gt; Indexing:</strong> Before any search can happen, the entire product catalog is pre-processed and stored in a specialized database called an <em>inverted index</em>. This index maps words (tokens) to the products they appear in, allowing for lightning-fast lookups.<br>&gt;&gt; <strong>Candidate Generation:</strong> The system uses the parsed query to retrieve an initial, broad set of potentially relevant products from the index. Classic algorithms like <strong>TF-IDF</strong> (Term Frequency-Inverse Document Frequency) and <strong>BM25</strong> (Best Match 25) are workhorses here. They score documents based on term frequency and rarity.</li><li><strong>Tech Flavor:</strong> The undisputed kings of open-source search infrastructure are <strong>Apache Solr</strong> and <strong>Elasticsearch</strong>. They provide the core indexing and retrieval capabilities that everything else is built upon.</li></ul><p><strong>3. Ranking &amp; Re-ranking (The Secret Sauce):</strong> This is where the real business logic and machine learning come alive. Getting 10,000 potential results is easy; showing the <em>best</em> 10 is hard.</p><ul><li><strong>Terminologies:<br>&gt;&gt; Relevance Ranking:</strong> Ordering the candidate set. Is a product titled “Running Shoes in Blue” more relevant than a product titled “Comfortable Shoes” with “blue” in the description?<br>&gt;&gt; <strong>Business Ranking:</strong> We’re not an academic library; we’re a business. We must also rank based on business signals. This includes factors like product popularity, inventory levels, profit margin, and whether an item is on sale.<br>&gt;&gt; <strong>Personalization:</strong> Tailoring the ranking for the <em>specific user</em>. Has this user previously bought from a certain brand? Have they shown an affinity for higher-priced items?</li><li><strong>Tech Flavor:</strong> This is the domain of <strong>Learning to Rank (LTR)</strong>. Instead of using a static formula, LTR models are trained on historical user data (e.g., queries, clicked products, purchased products) to learn a function that optimally orders products. Popular LTR models are often gradient-boosted decision trees, and libraries like XGBoost and LightGBM are standard tools. Increasingly, deep learning models are being used to capture more complex user-item interactions.</li></ul><h3>Measuring Success: The Metrics That Matter</h3><p>How do we know if our search engine is doing a good job? We measure everything. A key challenge is balancing <strong>relevance metrics</strong> (which tell us if we’re good librarians) with <strong>business metrics</strong> (which tell us if we’re good retailers). A perfectly relevant result that is out of stock doesn’t help the business. A high-margin item that is irrelevant to the query will just frustrate users. The best teams find the sweet spot between the two.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RWqqIb2XhzkM1IEmxNlaxQ.png" /><figcaption>Chart: Search metrics &amp; what they mean</figcaption></figure><p>There are other, more evolved, metrics like NDCG, that are getting prominence in this space.</p><h3>The Evolution and Future of E-commerce Search</h3><p>The journey of search has been remarkable, moving from a simple tool to an intelligent system.</p><ul><li><strong>Past (Keyword-based):</strong> The engine was a glorified text matcher. If you searched for “cellphone,” you wouldn’t find products listed as “mobile phone.” It was rigid and unforgiving.</li><li><strong>Present (Semantic &amp; Personalized):</strong> We are in the era of semantic understanding. The engine knows “cellphone” and “mobile phone” are the same concept. It uses ML to rank results based on learned patterns of user behavior and personalizes the order for each individual user. The rise of <strong>vector search</strong> (using libraries like Faiss or ScaNN) allows us to find &quot;semantically similar&quot; items, not just textually similar ones, enabling image-based search and better recommendations.</li></ul><h4>The Horizon: What’s Next for Search?</h4><ul><li><strong>Next 1 Year:</strong> Expect deeper integration of <strong>Large Language Models (LLMs)</strong> for conversational search. Users won’t just type keywords; they’ll ask questions like, “Show me durable hiking boots under $150 that are good for wide feet.” The search engine will need to parse this complex intent and apply filters automatically.</li><li><strong>Next 5 Years:</strong> <strong>Multimodal search</strong> will become standard. Users will snap a picture of a chair they like, and the search engine will find visually similar items in the catalog. Voice search will be seamless and context-aware. Hyper-personalization will move beyond past behavior to predict future needs based on browsing context, seasonality, and even external events.</li><li><strong>Next 10 Years:</strong> The concept of an explicit “search” may begin to fade. We’ll see a shift towards <strong>proactive, ambient commerce</strong>. The storefront will reconfigure itself in real-time based on who you are, what you’re doing, and what it predicts you’ll need next. The “search” will be a continuous, background process of anticipating user intent.</li></ul><h3>Conclusion</h3><p>The humble search bar is a microcosm of the entire data science discipline. It requires a deep understanding of the business, a mastery of data processing, a sophisticated application of machine learning, and a relentless focus on measurable outcomes. It has evolved from a simple string-matching utility into the intelligent, beating heart of the e-commerce experience. For any data professional looking to make a tangible impact on a business’s bottom line, there are few more challenging or rewarding domains to master. Remember, it all starts with understanding what happens when a user has a need and turns to that little box for an answer.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf8457602634" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking Language Models]]></title>
            <link>https://medium.com/@pinaki.brahma/unlocking-language-models-c059e9d99b48?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/c059e9d99b48</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[byte-pair-encoding]]></category>
            <category><![CDATA[tokenization]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[nlp]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Sun, 24 Aug 2025 18:15:03 GMT</pubDate>
            <atom:updated>2025-08-24T18:15:03.360Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>The Power of Byte Pair Encoding (BPE) in Modern Tokenization</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPDi-w9ADHZE9qZ0oLbl9Q.png" /><figcaption>Image: Byte Pair Encoding | Created using Copilot as prompted by Author</figcaption></figure><h3>Why do we need to know this</h3><p>Hey there, data science enthusiasts! Ever wondered how those incredible Large Language Models (LLMs) like GPT and LLaMA actually “read” and process text? It all starts with a fundamental, yet often overlooked, step: <strong>tokenization</strong>. Today, we’re diving deep into one of the most influential tokenization techniques: <strong>Byte Pair Encoding (BPE)</strong>.</p><h3>What does this solve?</h3><p>Imagine trying to teach a machine to understand human language. The very first hurdle is how to break down raw text into units that a computer can process. This is where tokenization comes in.</p><p>Traditional approaches faced significant challenges:</p><ul><li><strong>Word-Level Tokenization:</strong> Splitting text by spaces seems intuitive, but it quickly hits limitations.</li><li><strong>Out-of-Vocabulary (OOV) Words:</strong> It struggles with words not seen during training, like “football” if only “foot” and “ball” were known, leading to dreaded &lt;UNK&gt; (unknown) tokens.</li><li><strong>Vocabulary Explosion:</strong> Handling every word variant (e.g., “test,” “test-ify,” “test-ing”) and punctuation attached to words (Transformers?, do.) would create an enormous vocabulary. This results in huge embedding matrices, drastically increasing memory and computational costs for models.</li><li><strong>Language-Specific Issues:</strong> Many languages, like Chinese, don’t use spaces to separate words, rendering simple word-splitting ineffective.</li><li><strong>Character-Level Tokenization:</strong> As an alternative, treating each character as a token solves OOV issues and is robust to misspellings.</li><li><strong>Bloated Sequences:</strong> However, it generates extremely long input sequences. A five-word sentence might become 30 or more character tokens, consuming vast computational resources and quickly filling an LLM’s “context window” (the amount of text it can consider at once).</li><li><strong>Loss of Semantics:</strong> Learning meaningful representations from individual characters like “t” is much harder for a model than from a word like “today,” potentially leading to incorrect semantics.</li></ul><p><strong>Byte Pair Encoding (BPE)</strong> emerged as a brilliant <strong>middle ground</strong>. It aims to strike a balance, allowing models to process manageable sequence lengths, handle rare words, and maintain a reasonable vocabulary size without sacrificing too much semantic coherence.</p><h3>Background: History &amp; Evolution</h3><p>BPE’s journey began not in natural language processing, but in <strong>data compression</strong>.</p><ul><li><strong>The Original Algorithm:</strong> First described in 1994 by Philip Gage, BPE was designed to encode strings of text into smaller strings. The original algorithm worked by iteratively replacing the most frequently occurring <strong>pair of bytes</strong> in a dataset with a new, unused byte, effectively compressing the text. A lookup table was needed to reverse this process during decompression.</li><li><strong>Adaptation for NLP:</strong> In 2015, Sennrich et al. adapted this compression technique for use in Neural Machine Translation. The modified version doesn’t strictly aim for maximal compression but rather to efficiently encode plaintext into “tokens” (natural numbers) for language models. This adaptation proved to be a game-changer.</li><li><strong>Modern Prominence:</strong> BPE gained significant popularity with the rise of deep learning architectures like RNN-based and Transformer-based models. Today, various forms of BPE are fundamental to many widely used Transformer models, including <strong>GPT, GPT-2, RoBERTa, BART, DeBERTa, BLOOM, and LLaMA</strong>.</li></ul><h3>How does this work?</h3><h4>1. Training Phase</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*RiVH4yiO0iOf6fUhGtDtnw.png" /></figure><h4>2. Tokenization Phase (New Input)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/410/1*IDy0AByRMWmz3lkvOUxLOA.png" /></figure><h4>Example Walkthrough (Training)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*nFyOxRJaeccRQcSf1jZtpg.png" /></figure><h3><strong>Why BPE is preferred in LLMs</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fGY4apy5iRD-A27JBwDY9w.png" /><figcaption>Image: BPE vs. Word-level vs. Character-level Tokenization</figcaption></figure><ul><li><strong>Context Window Efficiency:</strong> The primary reason BPE is favored is its ability to reduce sequence length compared to character-level tokenization. Longer token sequences quickly exhaust an LLM’s limited context window, making the model less effective. BPE provides a good balance, allowing more meaningful text to fit within the context.</li><li><strong>Computational Feasibility:</strong> The computational cost for LLMs often scales quadratically with sequence length due to attention mechanisms. Character-level tokenization would make training prohibitively expensive, even for large companies. BPE’s shorter sequences make training feasible.</li><li><strong>Semantic Coherence:</strong> BPE tokens often capture linguistically meaningful units (like “ing” or “pre-”), which helps the model learn more effective representations than raw characters.</li></ul><h3>Key Takeaways</h3><p>1. BPE offers a crucial <strong>balance</strong> between the high OOV rates of word-level tokenization and the computational inefficiency of character-level tokenization.</p><p>2. This significantly <strong>reduces the number of tokens</strong> needed to represent text compared to character-level methods, making better use of LLM context windows and lowering computational costs during training and inference.</p><p>3. BPE effectively eliminates the problem of unknown tokens and thus is highly robust across diverse scripts and languages.</p><p>4. BPE tokens often capture <strong>meaningful linguistic patterns</strong> like common prefixes, suffixes, and word stems, which aids models in understanding and generating coherent text.</p><p>5. The algorithm is <strong>data-driven</strong>, learning its merge rules from the training corpus by identifying and combining the most frequent character/subword pairs.</p><p>Understanding BPE is key to appreciating the engineering behind modern LLMs. It’s a testament to how seemingly simple algorithms, when cleverly adapted, can enable groundbreaking advancements in AI!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c059e9d99b48" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A New Perch for Smarter Search]]></title>
            <link>https://medium.com/@pinaki.brahma/a-new-perch-for-smarter-search-acf324ed9d02?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/acf324ed9d02</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[byte-pair-encoding]]></category>
            <category><![CDATA[named-entity-recognition]]></category>
            <category><![CDATA[search-engines]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Mon, 18 Aug 2025 18:48:23 GMT</pubDate>
            <atom:updated>2025-08-18T18:48:23.283Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Blending language, tech, and taste to make search human again.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nIWFB-J0BtYUL3lsUx63uA.png" /><figcaption>Image: Author’s depiction of ‘humanized search’ through understanding language. Rendered using MS Copilot.</figcaption></figure><p>This article is inspired by <a href="https://www.youtube.com/watch?v=JL9x9N6YSUc">this video</a> by Arpit Bhaiyani. Many thanks to him for his explanation of how modern search engines work, especially in tandem with NER and NLP. He has used <strong><em>Zomato’s search engine</em></strong> to bring out a point. Treat this as my notes on the video, plus a few other sources.</p><h3>Introduction to Traditional Search</h3><p>For years, search systems worked on a <strong>keyword-matching principle</strong>. If you typed “pizza,” the system would hunt for documents, menus, or restaurant names containing that exact word. While effective in controlled settings, it often fell short when queries were more conversational or ambiguous. A customer asking “best place for chai near me” could easily end up with incomplete or irrelevant results, because traditional search engines lacked context and understanding.</p><h3>Evolution of Search w.r.t NER and NLP</h3><p>The shift came with the rise of <strong>Natural Language Processing (NLP)</strong> and <strong>Named Entity Recognition (NER)</strong>. Instead of seeing queries as a string of characters, modern systems parse them as <strong>expressions of human intent</strong>.</p><ul><li><strong>NLP</strong> allows the system to process queries in natural, everyday language.</li><li><strong>NER</strong> helps extract specific entities like dish names, restaurant names, or locations.</li></ul><p>Together, they transform “chai and samosa” from just words into <strong>two distinct entities linked by a craving</strong>.</p><h3>Intent &amp; Entity Detection</h3><p>Modern search goes beyond “what is written” to decode <strong>“what is meant.”</strong></p><ul><li><strong>Intent</strong> is about the user’s goal — do they want to eat, discover, or compare?</li><li><strong>Entities</strong> are the building blocks — specific dishes, cuisines, restaurants, or locations.</li></ul><p>For instance, “Domino’s pizza near Indiranagar” contains:</p><ul><li><strong>Intent</strong>: Find a restaurant</li><li><strong>Entities</strong>: “Domino’s” (restaurant), “pizza” (dish), “Indiranagar” (location)</li></ul><p>This dual understanding makes results contextual, fast, and relevant.</p><h3>Query Categorization</h3><p>User queries typically fall into a few recognizable categories:</p><ol><li><strong>Dish + Dish</strong> → <em>“Chai and samosa”</em></li><li><strong>Restaurant + Dish</strong> → <em>“Domino’s pizza”</em></li><li><strong>Dish/Restaurant + Modifier</strong> → <em>“Pizza outlets near me”</em></li></ol><p>By pre-defining these query classes, systems can streamline processing pipelines and deliver precise outputs.</p><h3>Challenges &amp; Complexities</h3><p>Building such a system is no small feat. Teams face hurdles like:</p><ul><li><strong>Data Scarcity</strong>: Lack of high-quality, labeled datasets for training.</li><li><strong>Multilingual Queries</strong>: Customers often mix English, Hindi, and regional languages.</li><li><strong>Spelling Variations</strong>: “Biryani,” “Biriyani,” “Biriani” — all must converge to the same intent.</li><li><strong>Local Lingo</strong>: Regional dish names that vary from state to state.</li></ul><p>Overcoming these requires <strong>custom embeddings, domain-specific training</strong>, and a careful balance between <strong>accuracy and performance</strong>.</p><h3>High Level Design</h3><p>The architecture typically includes:</p><ul><li><strong>Domain-trained Word Embeddings (Word2Vec)</strong> to capture semantics.</li><li><strong>Byte Pair Encoding (BPE)</strong> to handle unseen or misspelled words.</li><li><strong>BiLSTM + CRF model</strong> to perform entity detection with high precision.</li><li><strong>A scalable pipeline</strong> optimized for speed, since every millisecond matters in a consumer app.</li></ul><p>This design ensures that the system is both <strong>robust</strong> (handling messy input) and <strong>efficient</strong> (serving millions of users daily).</p><h3>Impact</h3><p>The results speak for themselves:</p><ul><li><strong>Higher CTR (Click Through Rates)</strong> from more relevant search results.</li><li><strong>Improved customer satisfaction</strong>, as users feel understood even when they type conversationally.</li><li><strong>Broader inclusivity</strong>, as multilingual, misspelled, and casual queries are handled seamlessly.</li><li>A <strong>foundation for the future</strong>, where transformer-based models promise even richer understanding.</li></ul><p>In essence, the evolution of search from keywords to <strong>intent-driven intelligence</strong> marks a new era — one where technology meets human expression halfway.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=acf324ed9d02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improve LLM based response through parent-child retrieval strategy (Part 2)]]></title>
            <link>https://medium.com/@pinaki.brahma/improve-llm-based-response-through-parent-child-retrieval-strategy-part-2-04ef7982b9a3?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/04ef7982b9a3</guid>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[chunking-strategies]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Sun, 02 Mar 2025 06:34:27 GMT</pubDate>
            <atom:updated>2025-03-02T06:34:27.871Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>A larger context window wins over the smaller ones in most occassions. Especially more relevant context is more knowledge (read power) to the LLM.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IvkoI7GoII0w8mlxNW2szQ.png" /><figcaption>Image: A larger context window wins over the smaller ones in most occassions. Image created using Copilot based on prompt from author.</figcaption></figure><p>I will start this part with the assumption that you have read through <a href="https://medium.com/@pinaki.brahma/improve-llm-based-response-through-parent-child-retrieval-strategy-part-1-cde3b1493961">Part 1</a>. In a gist, part 1 introduced you to why we need a parent-child retrieval strategy. It also demonstrated a methodology to ingest documents with the needed metadata and structure to be useful for this strategy.</p><h3>Retrieval Process for Parent-Child Structure</h3><p>The retrieval process has been separated out from the data ingestion process, as promised. And, we will make sure that we don’t go for an in-memory storage, unlike what the <a href="https://python.langchain.com/v0.1/docs/modules/data_connection/retrievers/parent_document_retriever/">langchain documentation</a> has suggested. With these constraints, here are the FIVE steps to an effective parent-child retrieval:</p><h3>1. Instantiate a retriever object for our vector store</h3><pre>def get_retriever(path_milvus_config:str=None):<br>    &quot;&quot;&quot;initialize a retriever object. <br>    The config file has all the connection details to the vector store.<br>    This config also has the search parameters<br>    &quot;&quot;&quot;<br>    with open(path_milvus_config, &#39;r&#39;) as f:<br>        config = json.load(f)<br>    <br>    vector_store_multi = VectorStore.load_vector_store(<br>        config[&#39;collection_name&#39;], <br>        config[&#39;connection_args&#39;], <br>        config[&#39;model_name&#39;], <br>        )<br>    <br>    # An MMR based search<br>    retriever = vector_store_multi.as_retriever(<br>        search_type=&#39;mmr&#39;,  # top options: [&#39;mmr&#39;, &#39;similarity&#39;]<br>        search_kwargs=config[&#39;search_kwargs&#39;]<br>        )<br>    <br>  return retriever</pre><pre>// milvus_config.json<br>// this needs to be updated<br>{<br>    &quot;model_name&quot;: &quot;embedding-model&quot;,<br>    &quot;collection_name&quot;: &quot;DEMO_COLLECTION&quot;,<br>    &quot;connection_args&quot;: {<br>        &quot;host&quot;: &quot;hostname.com&quot;, // change this<br>        &quot;port&quot;: &quot;88888&quot;, // change this<br>        &quot;user&quot;: &quot;user&quot;, <br>        &quot;password&quot;: &quot;password&quot;<br>    },<br><br>    // search parameters<br>    &quot;search_kwargs&quot;: {<br>        &quot;k&quot;: 5,<br>        &quot;score_threshold&quot;: 0.5,<br>        &quot;fetch_k&quot;: 5<br>    }<br>}</pre><pre>retriever = get_retriever(path_milvus_config)</pre><h3>2. Similarity based Child-document Retriever</h3><p>This step is similar to any other RAG based flow, where an incoming query is matched with the documents in our vector store collection. The top_k documents are returned. There is no change at all. The readiness lies in the way we have ingested the documents to our vector store — alongwith the custom metadata.</p><pre># incoming user query<br>query = [&quot;How does the process X work&quot;]<br><br>retrieved_docs = retriever.invoke(query)</pre><p>Five most similar documents are fetched from vector store. An example document would look something like this:</p><pre>[<br>Document(<br>  page_content = &quot;Process X has ... &quot;, <br>  metadata = {<br>    source = &#39;filename.txt&#39;,<br>    parent_id = 23,<br>    child_id = 3,<br>    ...<br>  }<br>]</pre><p>The above block of code shows that one of the retrieved chunk is the 3rd amongst all the chunks present in the 23rd parent section.</p><p>In a typical RAG based system, we would have <em>augmented</em> this <em>context</em> to the user prompt and sent it to the LLM for it to generate a response. This could be a partial context to the LLM, depending on the chunk size. To improve the context, we will need to go through a few additional steps by implementing the parent-child retrieval strategy.</p><h3>3. Retrieve the Parent documents</h3><ul><li>Step 1: Get the list of source and parent_idfrom the child chunks that we have retrieved so far.</li><li>Step 2: Create the expression for metadata filter. This is a string that will be passed as a filter expression. Example: source==&#39;filename.tx&#39; . We will define a function that generates the expression automatically based on the list of source and parent_id found above.</li></ul><pre>def generate_metadata_filter_expression(dict_filenames_parents:dict)-&gt;str:<br>    expr_step1 = [f&quot;(filename==&#39;{k}&#39; &amp;&amp; parent_id in {v})&quot; <br>                  for k,v in dict_filenames_parents.items()]<br>    expr_step2 = &#39; || &#39;.join(expr_step1)<br>    return expr_step2<br><br><br>dict_filter_criteria = {<br>  &#39;filename.txt&#39;: [10, 23, 40],<br>  &#39;article.html&#39;: [3, 19, 12],<br>  ...<br>}<br><br>filter_expression = generate_metadata_filter_expression(dict_filter_criteria)</pre><ul><li>Step 3: Metadata Filter using the above expression</li></ul><pre>output_fields = [&quot;filename&quot;, &quot;parent_id&quot;, &quot;child_id&quot;, &quot;text&quot;]<br><br>result = retriever.vectorstore.col.query(expr=filter_expression, output_fields=output_fields)</pre><p>This expression makes sure that we just one extra call to the vector database, making this process efficient too.</p><pre>docs_after_filter = [<br>    Document(<br>        metadata={<br>            k:v for k, v in doc.items() if k in output_fields[:-1]<br>        }, page_content=doc.get(&#39;text&#39;)<br>    )<br>    for doc in result<br>]<br><br>len(docs_after_filter)</pre><p>The above block of code makes sure that the results are formatted correctly as list of Document .</p><h3>4. Stitching the child chunks to create the parent chunk</h3><p>The child chunks from each of the parent_id is sorted in the right order. The contents are stitched one after another to improve the context of chunks. Also, the metadata for the parent chunk is engineered to suit the downstream process. The end of this step leads to a list of parent chunks that are ready to passed onto the prompt augmentation layer.</p><pre>docs_retrieved_parents = []<br><br>for filename, parents in dict_filenames_parents.items():<br>    docs_onefilename = [doc for doc in docs_after_filter if doc.metadata.get(&#39;filename&#39;)==filename]<br>    <br>    for parent in parents:<br>        print(filename, &#39;\t&#39;, parent)<br>        docs_oneparent = [doc for doc in docs_onefilename if (doc.metadata.get(&#39;parent_id&#39;)==parent)]<br>        child_id = [doc.metadata.get(&#39;child_id&#39;) for doc in docs_oneparent]<br>        _ = child_id.sort()<br>        print(f&quot;child id:\t{child_seq_nbr}&quot;)<br><br>        l_txt_stitched = []        <br>        for seq_nbr in child_id:<br>            txt_temp = [doc.page_content <br>                        for doc in docs_oneparent <br>                        if doc.metadata.get(&#39;child_id&#39;)==seq_nbr][0]<br>            l_txt_stitched.append(txt_temp)<br>        <br>        if len(l_txt_stitched)&gt;1:<br>            txt_stitched = &#39;\n&#39;.join(l_txt_stitched)<br>        else:<br>            txt_stitched = l_txt_stitched[0]<br>        <br>        # check if parent doc already exists<br>        if txt_stitched in [doc.page_content for doc in docs_retrieved_parents]:<br>            continue<br><br>        dict_metadata_temp = docs_oneparent[0].metadata<br>        _ = dict_metadata_temp.update({&#39;child_id&#39;:0})<br>        doc_temp = Document(metadata=dict_metadata_temp, page_content=txt_stitched)<br>        docs_retrieved_parents.append(doc_temp)<br><br>len(docs_retrieved_parents)</pre><h3>Prompt Augmentation Layer</h3><p>The parent chunks created in the above process are optionally passed through a re-ranked layer. We may choose to only pass top_k parent chunks to the next layer, based on re-ranking score.</p><p>Finally, these parent chunks are together augmented to the user prompt as the much needed contexts. We can do this using a simple string concatenation, or go for a langchain recommended approach, like using ChatPromptTemplate to achieve the same.</p><pre>sys_prompt = &quot;&quot;&quot;You are a ... expert. <br>Your task is to answer questions in a concise manner, strictly based on the context provided. <br>If you dont see anything relevant in the context, say that you do not know.<br>Do not answer questions from any other area.<br><br>Question: {query}<br>====<br>Context: {context}<br><br>Reponse: <br>&quot;&quot;&quot;</pre><pre>prompt = ChatPromptTemplate([<br>    (&#39;system&#39;, sys_prompt),<br>    (&#39;human&#39;, &#39;{query}&#39;),<br>])<br><br>parser = StrOutputParser()<br>chain = prompt | llm | parser</pre><h3>LLM Call</h3><p>You can use any large language model based on your requirement. We have experimented with both OpenAIand open sourced models like llama , mistral-instruct , etc. In the above block of code, we have used a string parser to make sure that the LLM response is formatted as a plain string, without any distraction.</p><p>If need be, and this is highly recommended, go for a pydantic based JsonOutputParser. This helps you be absolutely certain about the format of the response. And this is critical when you want the response coming out of your GenAI based application to be talking to an engineering pipeline, that expects a certain input schema. If there is interest, I can certainly write another post on why (&amp; how) we need a pydantic based setup for llm generated response in our applications.</p><p>Coming back to our model chain that we have created, we can use this now to ask questions. And we can expect the application to give you answers solely based on the context that is getting provided alongwith.</p><pre>query = &#39;What is system X ... bla bla&#39;<br><br>response = chain.invoke({&#39;query&#39;:query, &#39;context&#39;:context})</pre><p>From the eperiments that we have conducted, we see that there is a sharp improvement in both the coverage as well as completeness of the questions that the application has answered. The comparison is between a simple RAG based approach and the one that we created here — a parent-child retrieval strategy.</p><p>You can definitely use this approach in your pet projects and see the difference. Please feel free to connect with me and discuss more.</p><p>Do not forget to hit the clap before you move to the next article ☺️. Your claps could help me understand if my articles add value. It also helps the Medium algorithm to broadcast my articles to a broader audience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=04ef7982b9a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Improve LLM based response through parent-child retrieval strategy (Part 1)]]></title>
            <link>https://medium.com/@pinaki.brahma/improve-llm-based-response-through-parent-child-retrieval-strategy-part-1-cde3b1493961?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/cde3b1493961</guid>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Fri, 28 Feb 2025 17:53:38 GMT</pubDate>
            <atom:updated>2025-03-01T08:02:50.919Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Trade off between large context that gets passed to LLM for a more complete answer, and the increase in cost due to increased token size.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cAJLmDUZPusLYI1rQ9-Krw.png" /><figcaption>Image: Combination of Parent and Child documents may be the best strategy to augment user queries with for the LLM to repsond accurately. Image generated by Copilot based on author’s prompt.</figcaption></figure><h3>Current Scenario</h3><p>Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing the accuracy and reliability of Large Language Models (LLMs). By retrieving relevant documents from an external knowledge base, RAG mitigates hallucinations and improves factual correctness. Traditional RAG implementations rely on chunking long documents into smaller sections for efficient retrieval. However, a naive chunking approach can lead to loss of context, making it challenging to retrieve meaningful information.</p><p>There is a retrieval library using which parent-child retrieval can be done. And the results are great to see. In fact, these results have inspired me to look into this strategy with more vigor.</p><pre># This text splitter is used to create the parent documents<br>parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000)<br># This text splitter is used to create the child documents<br># It should create documents smaller than the parent<br>child_splitter = RecursiveCharacterTextSplitter(chunk_size=400)<br><br># The vectorstore to use to index the child chunks<br>vectorstore = Chroma(<br>    collection_name=&quot;split_parents&quot;, embedding_function=OpenAIEmbeddings()<br>)<br><br># The storage layer for the parent documents<br>store = InMemoryStore()<br><br># instantiate a parent child retriever<br>retriever = ParentDocumentRetriever(<br>    vectorstore=vectorstore,<br>    docstore=store,<br>    child_splitter=child_splitter,<br>    parent_splitter=parent_splitter,<br>)<br><br># add documents using retriever<br>retriever.add_documents(docs)</pre><p>The above code snippet has been taken from <a href="https://python.langchain.com/v0.1/docs/modules/data_connection/retrievers/parent_document_retriever/">langchain’s official documentation</a>. Can you see where the problem lies in this process?</p><p>There are two big challenges here:</p><ol><li><strong>Data (knowledge) Ingestion becomes part of the retriever object</strong></li></ol><blockquote>Typically, we would need the data ingestion to be outside the inference pipeline. This would not only help us keep things modularized, but also, will enable us to have additional steps like security layer, etc. in place, necessary for a data storage layer.</blockquote><p><strong>2. An in-memory store for the document storage</strong></p><blockquote>An in-memory storage for a large document storage is potentially a bottle neck for a production grade system. And this is something we must avoid.</blockquote><h3>Gap That We Want to Fill</h3><p>The primary limitation of standard chunking in RAG is the fragmentation of meaningful relationships between sections of a document. When chunks are retrieved independently, the LLM may lack the necessary context to generate coherent and informative responses. The Parent-Child retrieval strategy addresses this issue by structuring documents into hierarchical chunks:</p><ul><li><strong>Parent chunks</strong> contain key sections that include multiple child chunks.</li><li><strong>Child chunks</strong> contain sub-segments of a section within a parent chunk.</li></ul><p>This structure ensures that relevant details are retrieved along with high-level summaries, improving the accuracy and coherence of the generated response.</p><h3>Data Ingestion Process for Parent-Child Structure</h3><p>Before we can use a parent-child relationship between collection of documents during the retrieval process, we will need to ingest the documents (along with the metadata) to support what we want to achieve.</p><h4>Page Content</h4><p>A document is split into sections based on topics or hierarchical headers. If the chunk size is very long, the idea is to split it further into smaller child chunks. A smaller chunk size will typically translates into better embeddings.</p><h4>Metadata</h4><p>Metadata includes filename as well as maintains relationship between different document chunks. This is the core of how we get back the parent documents, given a child document is retrieved natively.</p><pre>from typing import List<br>from langchain.docstore.document import Document<br><br>def create_docs_from_one_html(path_one_html:str)-&gt;List[Document]:<br>  &quot;&quot;&quot;Split document based on header 1. This is parent document.<br>  Split parent doc further into smaller child documents.<br>  &quot;&quot;&quot;<br>  ... <br><br>def add_headers_to_content(doc:Document)-&gt;Document:<br>  &quot;&quot;&quot;extract headers from default metadata of a document.<br>  Add these headers to the content of the document.<br>  &quot;&quot;&quot;<br>  ...<br><br>def get_metadata_from_one_html(path_one_html:str)-&gt;dict:<br>  &quot;&quot;&quot;get filename, parent_id, child_id<br>  &quot;&quot;&quot;<br>  ...</pre><h4>Ingesting documents to Vector Store</h4><p>Once all the documents have been chunked based on the above parent-child strategy, it is time to ingest these chunks into vector store. In this example, we have chosen Milvus as our vector store. However, you can use Chroma DB, Pinecone, etc. too just as easily.</p><p>First, we create a collection by connecting to Milvus.</p><pre>from langchain.vectorstores import Milvus<br><br>def create_vector_store(**kwargs)-&gt;Milvus:<br>    &#39;&#39;&#39;<br>    create a vector store with these initial documents; persist this db<br>    &#39;&#39;&#39;<br>    connection_args = kwargs.get(&#39;connection_args&#39;) # credentials are part of this<br>    collection_name = &#39;DEMO_COLLECTION&#39;<br>    model_name = kwargs.get(&#39;model_name&#39;)<br>    <br>    # custom index if required<br>    dict_index_params = {<br>        &#39;metric_type&#39;: &#39;COSINE&#39;, # alternate: &#39;L2&#39;<br>        &#39;index_type&#39;: &#39;HNSW&#39;,<br>        &#39;params&#39;: {&#39;M&#39;: 8, &#39;efConstruction&#39;: 64}<br>    }<br><br>    vector_store = Milvus(<br>        embedding_function=get_embedding_func(model_name),<br>        collection_name=collection_name,<br>        connection_args=connection_args, <br>        auto_id=True,<br>        index_params=dict_index_params,<br>    )<br>    <br>    return vector_store<br><br><br>vector_store = create_vector_store(**config_milvus)</pre><p>Next, all documents are added to new collection we created just now. If required, we can have multiple collections with different subsets of documents.</p><pre>_ = vector_store.add_documents(docs_all)</pre><p>To be continued … “Retrieval Process for Parent-Child Structure” in upcoming article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cde3b1493961" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A multi-agentic way of implementing complex LLM applications using LangGraph framework]]></title>
            <link>https://medium.com/@pinaki.brahma/a-multi-agentic-way-of-implementing-complex-llm-applications-using-langgraph-framework-2ac7572c422a?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ac7572c422a</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[generative-ai]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Wed, 18 Sep 2024 15:47:17 GMT</pubDate>
            <atom:updated>2024-09-18T15:47:17.825Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Add LangGraph to your repertoire of GenAI related skills. This one skill can straightaway enable you to create complex AI systems today. The pre-requisite of course is to have an understanding of how LangChain is used.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-jTqDmLTPnuXK-p2HrNgdw.png" /><figcaption>Image: Creating the process flow is fundamental to solving any problem. Image generated using <a href="https://openart.ai/">OpenArt</a> based on prompt given by Author.</figcaption></figure><h3>Introduction</h3><p>LangGraph is a robust tool designed for creating stateful, multi-actor applications using Large Language Models (LLMs). By extending the capabilities of the LangChain library, LangGraph enables the coordination of multiple chains (or actors) across various computational steps in a cyclic manner. In this blog post, we will introduce LangGraph, explore its fundamental concepts, and address common points of confusion for beginners.</p><h3>What is LangGraph?</h3><p>LangGraph is a library built on top of LangChain, specifically designed to incorporate cyclic computational capabilities into your LLM applications. While LangChain allows for the definition of chains of computation (Directed Acyclic Graphs or DAGs), LangGraph introduces the ability to add cycles. This enhancement facilitates more complex, agent-like behaviors, enabling you to call an LLM in a loop and determine the next action based on its responses.</p><h3>Key Concepts</h3><h4><strong>Stateful Graph:</strong></h4><p>At the core of LangGraph is the concept of a stateful graph. Each node within the graph represents a step in your computation, and the graph maintains a state that is continuously updated as the computation progresses.</p><p><strong>Nodes:</strong><br>Nodes are the fundamental building blocks of LangGraph. Each node represents a specific function or computation step. You can define nodes to perform various tasks, such as processing input, making decisions, or interacting with external APIs.</p><p><strong>Edges:</strong><br>Edges connect the nodes within your graph, defining the flow of computation. LangGraph supports conditional edges, allowing for dynamic determination of the next node to execute based on the current state of the graph.</p><h3>Code example</h3><h4>Installations</h4><p>I am expecting that you have already langchain and openai installed in your python environment. If not, create a new environment and install the following libraries.</p><pre>## general packages<br>numpy<br>pandas<br>matplotlib<br>pydantic<br>tqdm<br>python-dotenv<br><br>## llm &amp; other models<br>openai==1.12.0<br>sentence-transformers<br>faiss-cpu<br>chromadb<br><br>## frameworks<br>langchain<br>langchain-community<br>langchain-experimental<br>langchain-core<br>langchainhub<br>langgraph</pre><p>Install LangGraph using pip install langgraph.</p><h4>Imports</h4><pre># usual libraries<br>from typing import List<br>from typing_extensions import TypedDict<br>import re<br>from pydantic import BaseModel, Field<br><br># langchain related libraries<br>from langchain_core.prompts import ChatPromptTemplate<br>from langchain_core.output_parsers import JsonOutputParser, StrOutputParser<br><br># langgraph related libraries<br>from langgraph.graph import StateGraph, START, END</pre><h4>Define a state</h4><pre>class AgentState(TypedDict):<br>    question: str<br>    llm_output: str<br>    documents: list[str]<br>    cnt_retries: int</pre><h4>Methods for Nodes</h4><p><strong>Node — Question Scope Classifier</strong></p><pre>class QuestionScopeClass(BaseModel):<br>    &quot;&quot;&quot;Scope of the question&quot;&quot;&quot;<br>    score: str = Field(<br>        description=&quot;Boolean value to check if question is about what, where or comparison. If yes -&gt; &#39;Yes&#39;, else &#39;No&#39;&quot;<br>    )<br><br><br>def question_intent_classifier(state: AgentState):<br>    question = state[&quot;question&quot;]<br>    state[&#39;cnt_retries&#39;] = 0<br>    parser = JsonOutputParser(pydantic_object=QuestionScopeClass)<br>    output_format = parser.get_format_instructions()<br>    system = &quot;&quot;&quot;You are a question classifier. Check if the question is about one of the following topics: <br>        1. definition<br>        2. availability<br>        3. comparison<br>        If the question IS about these topics, respond with &quot;Yes&quot;, otherwise respond with &quot;No&quot;.<br>        <br>        Format output as: `{output_format}`<br>        &quot;&quot;&quot;<br><br>    intent_prompt = ChatPromptTemplate.from_messages(<br>        [<br>            (&quot;system&quot;, system),<br>            (&quot;human&quot;, &quot;User question: {question}&quot;),<br>        ]<br>    )<br><br>    llm = openAI()<br>    grader_llm = intent_prompt | llm | parser<br>    result = grader_llm.invoke({&quot;question&quot;: question, &#39;output_format&#39;: output_format})<br>    print(f&quot;to_proceed: {result[&#39;score&#39;]}&quot;)<br>    state[&quot;on_topic&quot;] = result[&#39;score&#39;]<br>    return state<br><br># router to enable conditional edges<br>def on_topic_router(state: AgentState):<br>    print(&#39;ontopic router ... &#39;)<br>    on_topic = state[&quot;on_topic&quot;]<br>    if on_topic.lower() == &quot;yes&quot;:<br>        return &quot;on_topic&quot;<br>    return &quot;off_topic&quot;</pre><p><strong>Node — Answer Grader</strong></p><pre>def grade_answer(state: AgentState):<br>    answer = state[&#39;llm_output&#39;]<br>    print(&#39;grading ...&#39;)    <br>    # print(&#39;generated answer type:&#39;, type(answer))<br>    # do we have &#39;do not know&#39; in answer<br>    pattern = r&#39;do not know|apolog|sorry&#39;<br>    is_answer = &#39;Yes&#39; if not re.search(pattern, answer.lower()) else &#39;No&#39;<br>    state[&#39;is_answer_ok&#39;] = is_answer<br>    print(f&quot;answer grade: {is_answer}&quot;)<br>    return state<br><br>def is_answer_router(state: AgentState):<br>    print(&#39;grading router ... &#39;)<br>    is_answer = state[&quot;is_answer_ok&quot;]<br>    if state[&#39;cnt_retries&#39;] &gt;2:  # max of 3 retries allowed (0 to 2)<br>        return &quot;hit_max_retries&quot;<br>    if is_answer.lower() == &quot;yes&quot;:<br>        return &quot;is_answer&quot;<br>    return &quot;is_not_answer&quot;</pre><p><strong>Node — Question Re-phraser</strong></p><pre>def question_rephraser(state: AgentState):<br>    print(&#39;rephrasing ...&#39;)<br>    question = state[&#39;question&#39;]<br>    print(f&quot;retrying: {state[&#39;cnt_retries&#39;]+1}&quot;)<br>    llm = openAI)<br><br>    template = &quot;&quot;&quot;<br>        You are an expert in rephrasing English questions. \<br>        You hav been tasked to rephrase question from the Retail and Supply Chain domain. \<br>        While rephrasing, you may do the following:<br>        1. Extract keywords from the original question<br>        2. Expand or create abbreviations of the question as needed<br>        3. Understand the intent of the question<br>        4. Include the above information to generate a rephrased version of the original question.\<br>        Do not output anything else apart from the rephrased question.\<br>        <br>        Question: {question}<br>        &quot;&quot;&quot;<br><br>    prompt = ChatPromptTemplate.from_template(<br>        template=template,<br>    )<br>    chain = prompt | llm | StrOutputParser()<br>    result = chain.invoke({&quot;question&quot;: question})<br>    # print(result)<br>    state[&#39;question&#39;] = result<br>    state[&#39;cnt_retries&#39;] +=1<br>    return state</pre><p><strong>Node — Retriever (Optional)</strong></p><pre># some function to create a retriever for a vector store<br>retriever = get_retriever(config_vector_store) <br><br>def retrieve_docs(state: AgentState):<br>    question = state[&quot;question&quot;]<br>    documents = retriever.invoke(input=question)<br>    state[&quot;documents&quot;] = documents<br>    print(f&quot;cnt of retrieved docs: {len(documents)}&quot;)<br>    return state</pre><p><strong>Node — Response Generation</strong></p><pre>def generate_answer(state: AgentState):<br>    question = state[&#39;question&#39;]<br>    context = [doc.page_content for doc in state[&#39;documents&#39;]]<br>    print(&#39;generating answer ... &#39;)<br>    llm = openAI()<br><br>    template = &quot;&quot;&quot;<br>        You are a Customer Support Chatbot aimed to answer the user&#39;s queries coming from Retail and Ecommerce industry.\<br>        Keep the tone conversational and professional.\<br>        Remember that abbreviations mentioned are related to these domains.\<br>        Answer the question strictly based on the context provided.\<br>        Avoid mentioning in the response that a context was referred.\<br>        Avoid using words like &#39;certainly&quot; and &quot;it looks like&quot; in the generated response.\<br>        Do not output anything else apart from the answer.\<br>        <br>        {context}<br><br>        Question: {question}<br>        &quot;&quot;&quot;<br><br>    prompt = ChatPromptTemplate.from_template(<br>        template=template,<br>    )<br>    chain = prompt | llm | StrOutputParser()<br>    result = chain.invoke({&quot;question&quot;: question, &quot;context&quot;: context})<br>    print(f&quot;result from generation: {result}&quot;)<br>    state[&#39;llm_output&#39;] = result<br>    return state</pre><p><strong>Node — Fall Back Response</strong></p><pre>def get_default_reply(state:AgentState):<br>    print(&#39;get the default answer ...&#39;)<br>    state[&#39;llm_output&#39;] = &#39;I do not have an answer.&#39;<br>    return state</pre><h4>Compose the Lang Graph</h4><p>This is the process flow that connects the different nodes that we have created above. This also enables conditional edges by utilizing the routers defined above.</p><pre>workflow = StateGraph(AgentState)<br><br># Add the Nodes<br>workflow.add_node(&#39;intent_classifier&#39;, question_intent_classifier)<br>workflow.add_node(&#39;retrieve_docs&#39;, retrieve_docs)<br>workflow.add_node(&#39;generate_answer&#39;, generate_answer)<br>workflow.add_node(&#39;grade_answer&#39;, grade_answer)<br>workflow.add_node(&#39;question_rephraser&#39;, question_rephraser)<br>workflow.add_node(&#39;default_reply&#39;, get_default_reply)<br><br># Add the Edges including the Conditional Edges<br>workflow.add_edge(&#39;intent_classifier&#39;, START)<br>workflow.add_conditional_edges(<br>    &#39;intent_classifier&#39;, on_topic_router, <br>    {<br>        &#39;on_topic&#39;: &#39;retrieve_docs&#39;, <br>        &#39;off_topic&#39;: &#39;default_reply&#39;<br>    }<br>)<br>workflow.add_edge(&#39;retrieve_docs&#39;, &#39;generate_answer&#39;)<br>workflow.add_edge(&#39;generate_answer&#39;, &#39;grade_answer&#39;)<br>workflow.add_conditional_edges(<br>    &#39;grade_answer&#39;, is_answer_router,<br>    {<br>        &#39;is_answer&#39;:END, <br>        &#39;is_not_answer&#39;:&#39;question_rephraser&#39;, <br>        &#39;hit_max_retries&#39;:&#39;default_reply&#39;<br>    }<br>)<br>workflow.add_edge(&#39;question_rephraser&#39;, &#39;retrieve_docs&#39;)<br>workflow.add_edge(&#39;default_reply&#39;, END)<br><br># compile the workflow<br>app = workflow.compile()</pre><p><strong>Visualize the workflow</strong></p><pre>try:<br>    display(Image(app.get_graph(xray=True).draw_mermaid_png()))<br>except:<br>    pass</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*0Ybvw--nCGhXvSDgLLED5Q.jpeg" /><figcaption>Image: Visual representation of the workflow we created</figcaption></figure><p><strong>Using the App</strong></p><pre>query = &quot;Capital of India&quot;<br>response = app.invoke(input={&quot;question&quot;: query})<br><br>print(result[&#39;llm_output&#39;])</pre><h3>Conclusion</h3><p>By understanding these foundational elements, users can harness LangGraph to design more complex, dynamic LLM applications that go beyond simple linear chains of tasks. Whether developing agent-based systems or managing multi-step workflows, LangGraph offers a robust framework to tackle challenges in the evolving field of generative AI.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ac7572c422a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learn about Consensus clustering — an ensemble for unsupervised learning]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@pinaki.brahma/learn-about-consensus-clustering-an-ensemble-for-unsupervised-learning-a1ab1b607669?source=rss-a9b1b8d48e87------2"><img src="https://cdn-images-1.medium.com/max/1792/1*3HFoLuvlHPYS4-XOaxBTgw.png" width="1792"></a></p><p class="medium-feed-snippet">It is common practice to use a set of week supervised learning algorithms to build a powerful ensemble model. However, the same principle&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@pinaki.brahma/learn-about-consensus-clustering-an-ensemble-for-unsupervised-learning-a1ab1b607669?source=rss-a9b1b8d48e87------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@pinaki.brahma/learn-about-consensus-clustering-an-ensemble-for-unsupervised-learning-a1ab1b607669?source=rss-a9b1b8d48e87------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1ab1b607669</guid>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[consensus]]></category>
            <category><![CDATA[clustering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Pinaki Brahma]]></dc:creator>
            <pubDate>Wed, 04 Sep 2024 03:31:32 GMT</pubDate>
            <atom:updated>2024-09-04T03:31:32.602Z</atom:updated>
        </item>
    </channel>
</rss>