<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[inspiringbrilliance - Medium]]></title>
        <description><![CDATA[Writings from Sahaj Software focusing on complex problems without complicating the solutions. - Medium]]></description>
        <link>https://medium.com/inspiredbrilliance?source=rss----5cdcaabf7d50---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>inspiringbrilliance - Medium</title>
            <link>https://medium.com/inspiredbrilliance?source=rss----5cdcaabf7d50---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 03:16:00 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/inspiredbrilliance" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Shift-Left for Agentic Systems]]></title>
            <link>https://medium.com/inspiredbrilliance/shift-left-for-agentic-systems-b247b1521392?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/b247b1521392</guid>
            <category><![CDATA[agentic-ai-architecture]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Karthika Vijayan]]></dc:creator>
            <pubDate>Sat, 07 Nov 2026 07:34:48 GMT</pubDate>
            <atom:updated>2026-11-10T06:05:53.897Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>A production-ready agentic system is shaped by decisions made before the first agent is built!</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cKym69Vj65p2mHmR.png" /></figure><p>Agentic systems are easy to demonstrate and difficult to operate. A prompt, an orchestrator, a few specialised agents and a vector store can produce an appealing prototype in an afternoon. However, production introduces a different set of conditions: variable traffic, incomplete data, ambiguous requests, latency expectations and a cost model that must survive procurement. Under these conditions, agents may start to retrieve the wrong facts, resolve the same entity differently, or incur a token bill that invalidates the business case. These failures are often attributed to the model. More often, they are architectural consequences of decisions made before the first agent even ran.</p><p>Software engineering calls the corrective principle <strong>shift-left. </strong>Address quality concerns during design when the system can <em>still</em> accommodate them, rather than discovering them in production, when the cost of change is highest. Agentic systems demand an especially strong version of this principle. Their critical properties are not features to add after deployment; they emerge from how the system is decomposed, evaluated, supplied with context, and grounded in enterprise data. Across enterprise AI engagements, four design commitments repeatedly determined whether a prototype could become a dependable product. These are:</p><ul><li><strong>Accuracy</strong>: behaviour is evaluated continually rather than inferred from a convincing demo.</li><li><strong>Reusability</strong>: capabilities can be adapted to new use cases without entirely rebuilding them.</li><li><strong>Performance</strong>: latency and cost constrain the architecture from the outset.</li><li><strong>Coherence</strong>: independent agents operate over the same grounded representation of reality.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZKh2Ac8YIf_b6rYB.png" /><figcaption><strong>Figure 1</strong>: <em>The four design commitments surrounding an enterprise agentic system</em></figcaption></figure><p>All these commitments rest on <strong>AI readiness of data</strong>. An agent cannot reason reliably over data that is poorly organised, ambiguously defined, or missing the metadata needed to interpret it. These considerations need to be addressed during design, rather than after the system begins to scale or struggles in production. The sections below examine each commitment through its mechanism and operational consequence.</p><h3>Accuracy</h3><p>Accuracy begins with a discipline that agentic systems make easy to neglect: do not trust a system just because it performs well on a small, hand-picked set of questions. For instance, a SQL agent we built for a client, who is an automobile manufacturer, translated natural-language questions into SQL queries convincingly in isolation. Broader testing with the large quantity of data ingested from automobile sensors, exposed a more serious behaviour: the agent generated syntactically valid SQL against tables that did not even exist. The queries looked plausible, but their answers had no grounding in the underlying schema. We added a schema validator and a correction loop that rewrites a failing query before execution. More importantly, we built the evaluation framework that made this failure visible.</p><p>That framework needs, at the very least, two complementary forms of test:</p><ul><li><strong>Component evaluations</strong> assess a single agent against a golden dataset of inputs and known-correct outputs. They provide the localisation benefits of unit tests: when behaviour degrades, the affected component is apparent.</li><li><strong>End-to-end evaluations</strong> assess the complete, often multi-turn flow. They must assert the path that an agentic system takes to get to an answer, as well as the final answer. A question about mileage of a vehicle should reach the API agent to call the mileage-API; a policy question should reach the policy agent to retrieve the vehicle insurance policy terms and reason over it. An orchestrator that obtains a correct answer through an accidental route has not demonstrated reliable behaviour.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*j6XzY3iR86M7UUiC.png" /><figcaption><strong>Figure 2</strong>: <em>Two complementary evaluation tiers: component evaluation for an individual agent and end-to-end evaluation for the entire system</em></figcaption></figure><p>Two practices make these evaluations durable. Use an LLM judge only where the output is inherently language-shaped and cannot be assessed programmatically. Deterministic behaviour deserves deterministic assertions, which are cheap, fast and easier to diagnose. Build an <strong>evaluation store</strong> early: a replayable set of representative cases large enough to expose regressions. Once that store exists, a model swap, prompt change, or routing change becomes a measured experiment rather than a leap of faith. Accuracy needs to be re-evaluated after every material change to the system.</p><h3>Reusability</h3><p>Enterprises rarely need a single-purpose agent. They need a capability, be it search, triage, query, or resolution, that can serve the next use case without a substantial rewrite. In a client engagement that I was part of, we built an AI-search framework for a vehicular fleet-management application. This app helped users with live tracking of vehicles, historical data-driven insights on vehicle health and operational efficiency of the fleet. The agentic system consisted of a SQL agent, API agent and a retrieval tool, among other use-case specific agents. Later, we were asked to support a sales assistant for small commercial vehicles and a resolution assistant for the finance team, banking on the same AI-search framework. That extension was possible because the original system was assembled from independently replaceable components, not wired around one application.</p><p>Reusability begins within the agent itself. Its <strong>prompt</strong>, <strong>model</strong>, and <strong>harness</strong> should each be engineered as modular components, because a new use case commonly requires changes to all three. The prompt should be a set of composable layers: role, domain expertise, persona, guardrails, and task instructions, rather than one indivisible block of text. The model should be plug-and-play, selected according to the capability, latency, and cost requirements of the task. The harness should separate pre-processing and post-processing so that input normalisation, retrieval, validation, and output formatting can be re-engineered without entangling the rest of the agent.</p><p>For instance, a new sales assistant may need different domain-prompt layers, a less expensive model for a bounded task, and different input and output processing from the AI-search assistant for fleet management. With a prompt library, a model interface, and a modular harness, these changes become deliberate substitutions rather than requiring reconstruction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ef8_REy9iPh0RCd_.png" /><figcaption><strong>Figure 3</strong>: Reuse depends on making the agent’s sources of variation explicit, so that new use cases require controlled adaptation rather than wholesale reconstruction.</figcaption></figure><p>At system scale, this means defining components by their responsibilities and the contracts between them, rather than building a single large prompt-driven application, a design approach I discuss in <a href="https://medium.com/inspiredbrilliance/agentic-ai-building-the-right-mental-model-for-system-design-269d85c1689f">my earlier piece on the mental model for agentic system design </a>. This makes it possible to add or replace a retrieval path, validator, domain agent, or model without redesigning the entire flow. It also keeps complexity proportional to the problem. Anthropic recommends simple, composable patterns over heavy frameworks ( <a href="https://www.anthropic.com/engineering/building-effective-agents">Building Effective Agents </a>, 2024), while OpenAI recommends starting with a single agent and adding further agents only when a specific bottleneck justifies them ( <a href="https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf">A Practical Guide to Building Agents </a>, 2025). The architecture should therefore remain modular enough to accommodate additional agents or capabilities without requiring a rebuild.</p><h3>Performance</h3><p>Latency and cost are often considered close to deployment, when they can expose architectural decisions that are difficult to change. We encountered this in a voice assistant, where even modest delay degrades the interaction. The first design placed every capability and tool description in one orchestrator. It was conceptually simple, but every call carried the full prompt and tool surface. Splitting responsibilities by capability reduced the prompt context each agent had to process. Additionally, we performed a usage data analysis from production to identify FAQs, for which we cached the SQL queries needed. In runtime, these were retrieved using a semantic matching between user questions and FAQs. These 2 strategies helped us bring down the latency by 10x.</p><p>The cost of agentic execution also depends on how work is allocated across the system. Anthropic reports that agents use roughly four times the tokens of a chat interaction, while multi-agent systems use about fifteen times as many ( <a href="https://www.anthropic.com/engineering/multi-agent-research-system">How we built our multi-agent research system </a>, 2025). On an early seven-agent design, a projection using a frontier model at prevailing prices reached several crores of Indian rupees annually before optimisation. That estimate changed the architecture before implementation, not because the model was unsuitable, but because the work had not been allocated according to its difficulty.</p><p>Two mechanisms address this. A <strong>mixture of models</strong> assigns a capable large model to orchestration and reserves smaller models for narrow, well-bounded tasks such as validation, formatting, and API invocation; escalation is justified only when evaluation shows that the smaller model is insufficient. <strong>Prompt caching</strong> reduces the repeated cost of stable prompt fragments, and is available across Anthropic, OpenAI, and Google. These mechanisms depend on the same component boundaries that enable reuse. A modular prompt can be cached; a bounded task can be assigned to an appropriately priced model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HUrwBEADnSVlqZ-u.png" /><figcaption><strong>Figure 4</strong>: <em>Model selection and prompt caching reduce the cost of execution by matching model capability to task complexity and reusing stable context.</em></figcaption></figure><p>A multi-agent design is justified only when its additional coordination cost is lower than the bottleneck it removes. Latency and spend should be modelled during design, when they can still influence the architecture, rather than treated as afterthoughts in production.</p><h3>Coherence</h3><p>Coherence is a failure mode we have observed that is difficult to detect at the component level. A system may be reusable, accurate at the component level, and performant, yet fail because its agents construct incompatible interpretations of the same request. We use <strong>coherence</strong> to mean that independent agents resolve shared entities consistently and act on a common representation of the world.</p><p>In the fleet-management AI assistant system the we built, two agents received the same vehicle registration number and resolved it independently. One associated it with one vehicle record; the other selected another. Neither response was obviously hallucinatory. Each agent had made a locally plausible choice. The system is now left with reasoning over incompatible facts, a defect that would not be exposed by testing either agent alone.</p><p>The cause is architectural. A conventional orchestrator is often asked to perform two distinct functions: understand the request by extracting and resolving its entities, and coordinate the work needed to answer it. These functions have different failure modes. In entity-heavy domains, like fleets, financial accounts, or parts catalogues, language understanding should be established once, before work is distributed, rather than reconstructed by every downstream agent.</p><p>Concretely, entity extraction and resolution become a separate upstream step. It produces a <strong>canonical state</strong>: a grounded set of entity references that the orchestrator and all downstream agents consume. If a registration number resolves to one vehicle at the head of the pipeline, downstream agents receive that resolution rather than independently attempting it again. This is a practical form of context engineering: curate the context that each agent receives instead of giving every agent all available information ( <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">Effective Context Engineering for AI Agents </a>, 2025). Coherence is not improved by tuning agents one at a time. It is created by establishing a shared boundary for interpretation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HFFysRkGh_gAp9bf.png" /><figcaption><strong>Figure 5</strong>: <em>Coherence is established by resolving shared entities once, before distributed work begins</em></figcaption></figure><h3>Data readiness</h3><p>The four commitments above assume that the system has reliable material to reason over. Enterprise tables, APIs, and documents are often inconsistently organised, lack useful metadata, or use conflicting terminology. A well-designed agentic system cannot compensate indefinitely for these data issues. Data readiness is therefore an architectural concern, not preliminary housekeeping; I examine it in more detail in <a href="https://medium.com/inspiredbrilliance/the-ai-ready-enterprise-preparing-data-for-the-age-of-ai-a4c12ec9716a">The AI-Ready Enterprise: Preparing Data for the Age of AI </a>.</p><h3>Conclusion</h3><p>Moving from an agentic demonstration to a dependable product requires architectural decisions around reusable boundaries, measurable correctness, performance constraints, and a canonical state for coherent reasoning. These decisions are only as strong as the data foundation beneath them.</p><p>Deferring these concerns transfers their cost to production, where dependencies are tighter, failures are harder to localise, and redesign is more expensive. Addressing them during design gives the system a better chance of meeting its operational requirements as it scales.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b247b1521392" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/shift-left-for-agentic-systems-b247b1521392">Shift-Left for Agentic Systems</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understand I/O Multiplexing and Asyncio Streams work in Python]]></title>
            <link>https://medium.com/inspiredbrilliance/understanding-how-i-o-multiplexing-and-asyncio-streams-work-in-python-11edee3ddcad?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/11edee3ddcad</guid>
            <category><![CDATA[io-multiplexing]]></category>
            <category><![CDATA[asyncio]]></category>
            <category><![CDATA[sockets]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Rohan Reddy Alleti]]></dc:creator>
            <pubDate>Mon, 26 Oct 2026 07:07:20 GMT</pubDate>
            <atom:updated>2026-10-26T07:07:19.080Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This post is a written companion to </em><a href="https://youtu.be/qDGI5G-C6AQ"><em>my talk at PyCon India 2025</em></a><em>. I could not complete my talk in time and hence this post is a complete version of it.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*t4jE04PUca6yezEa" /><figcaption>Photo by <a href="https://unsplash.com/@rubaitulazad?utm_source=medium&amp;utm_medium=referral">Rubaitul Azad</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What Is I/O Multiplexing?</h3><p>Multiplexing is handling multiple things over a single shared resource. In telecom, it’s multiple phone calls over one wire. In networking, it’s multiple data streams over one connection.</p><p><strong>I/O multiplexing</strong> applies this to a program: a single thread monitoring multiple I/O sources: sockets, files, pipes, and reacting to whichever one is ready. Instead of dedicating one thread to each connection and having most of them sit idle, you have one thread that says, “tell me when any of these has data,” and then handles whichever ones are ready.</p><p>The operating system provides the machinery for this: system calls like select, poll, epoll (Linux), and kqueue (macOS) that let your program hand the kernel a list of file descriptors and say, &quot;wake me up when something happens on any of these.&quot;</p><p><strong>Why is I/O multiplexing even relevant to us?</strong></p><p>Your socket server calls recv() and blocks: while it&#39;s waiting on one client, every other client is invisible. One fix is a thread per connection, but that does not scale; threads carry memory overhead (~8 MB stack each), OS context-switching costs, and Python&#39;s GIL restricts one thread to run Python bytecode at a time. With I/O Multiplexing, a single thread can handle multiple connections for a fraction of the cost. Asyncio is a popular library to do this in modern python, which this will post will dig into later.</p><h3>Sockets Are File Descriptors</h3><p>When your program opens a socket connection, it doesn’t interact with the network interface directly. Instead, the kernel gives it a file descriptor, and your program reads from and writes to that file descriptor while the kernel handles the networking underneath.</p><p>Similarly, file descriptors 0, 1, and 2 in any program are your standard input, standard output, and standard error. A socket is just another file descriptor, maybe number 5 or 6 or 42.</p><p>The kernel provides tools to monitor <em>file descriptors</em> for readiness and those tools work for sockets too.</p><h3>Monitoring File Descriptors with select</h3><p>The select syscall lets you hand the kernel a list of file descriptors and say, “tell me when any of these are ready to read from or write to.” Your program then blocks until something is ready (or until the timeout expires, if you set one) and handles whatever is ready.</p><p>Here’s an example using select: a non-blocking way to read from standard input:</p><pre>import select<br>import sys<br><br>while True:<br>    ready, _, _ = select.select([sys.stdin], [], [], 0)  # timeout=0, non-blocking<br>    if ready:<br>        data = sys.stdin.readline()<br>        print(f&quot;Got: {data.strip()}&quot;)<br>    # ... do other work here</pre><p>Compare this to input(), which blocks your program until the user types something. With select, your program can check whether stdin has data, and if it doesn’t, do something else.</p><p>select can monitor multiple file descriptors at once. Say you have a socket connection and standard input, you can tell select to watch both.(comment if you know of a different way to achieve the same :D)</p><pre>import sys<br>import select<br>import socket<br><br>client = socket.socket()<br>client.connect((&quot;localhost&quot;, 8888))<br><br>while True:<br>    readable, _, _ = select.select([sys.stdin, client], [], [])<br>    if readable:<br>        for reader in readable:<br>            if reader is client:<br>                data = client.recv(1024)<br>                print(f&quot;Server &gt;&gt;&gt; {data.decode()}&quot;)<br>            else:<br>                stdin_message = sys.stdin.readline()<br>                client.sendall(stdin_message.encode())</pre><p>This is a simple chat client, you can send multiple messages to the server <em>and</em> receive messages from it without each operation blocking each other, all in a single thread. If you’d used input(), you&#39;d be stuck waiting for keyboard input and would never receive incoming server messages until you typed something.</p><p>This is I/O multiplexing in its simplest form: one thread, multiple file descriptors, acting on whichever is ready.</p><p><em>(A quick note: </em><em>select can only monitor up to 1024 file descriptors. Python&#39;s </em><em>selectors module picks the best OS-level mechanism automatically:</em><em>epoll on Linux, </em><em>kqueue on macOS which scales much better)</em></p><h3>Building a Select-Based Server</h3><p>Let’s build a server using select that handles multiple clients:</p><pre>import socket<br>import select<br><br>server = socket.socket()<br>server.bind((&quot;localhost&quot;, 8888))<br>server.listen()<br><br>wait_read = [server]<br><br>while True:<br>    readers, _, _ = select.select(wait_read, [], [])<br>    for reader in readers:<br><br>        if reader is server:<br>            client_sock, addr = server.accept()<br>            print(f&quot;Got new connection from {addr}&quot;)<br>            wait_read.append(client_sock)<br><br>        else:<br>            data = reader.recv(1024)<br>            if not data:<br>                wait_read.remove(reader)<br>                reader.close()<br>            reader.send(large_data)  # send a large response</pre><p>Here’s how it works: we register the server socket with select and wait. When select returns, we loop over the ready sockets, and each one falls into one of two cases. If it’s the server socket, “ready” means a new client is trying to connect, so we accept() it and add the new client to our watch list. If it’s a client socket, “ready” means the client sent data, so we read it and send back a large response.</p><p>The problem: when we call reader.send(large_data), that call blocks until the entire payload is written. While the thread is busy sending data to client one, client two is waiting.</p><h3>The Problem</h3><p>Connect two clients to this server. Client one sends a message, triggering the server to stream the large payload back. Client two sends a message too. But client two gets <em>nothing</em>; it’s completely blocked while client one receives all the data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EZp4z87qft8wXE3Na2fJyQ.png" /></figure><p><em>The thread is stuck on Client A’s </em><em>send(). Client B doesn&#39;t exist until A is done.</em></p><p>This can be an undesirable effect and lets understand why this is happening.</p><p>The answer lies in <strong>socket buffers</strong>.</p><h3>How Socket Buffers Work</h3><p>send() doesn&#39;t write directly to the network, it writes into a kernel-managed buffer, which has a fixed size. When that buffer is full, send() blocks until the kernel flushes some of it onto the network. With 1 MB of data and a 10 KB buffer, the thread is stuck in a loop: write 10 KB, wait for flush, write 10 KB, wait for flush; completely occupied with one socket while every other client waits.</p><h3>Non-Blocking Sockets</h3><p>conn.setblocking(False) changes send() fundamentally: it writes as much as the buffer can hold and <strong>returns immediately</strong>, even if it didn&#39;t send all your data. It returns the number of bytes it actually wrote.</p><pre>n = conn.send(large_data)  # might only send 10KB out of 1MB<br># n tells you how many bytes were actually sent.<br># the rest? that needs to be tracked and re-sent.</pre><p>Same two clients, same large payload; but now both receive data, chunk by chunk:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*thjvlt0ueplz9AmDukEArw.png" /></figure><p><em>Both clients get data in chunks. Neither blocks the other.</em></p><p>With non-blocking sockets, one thread can serve many clients by reading and writing small chunks of data. While the kernel flushes Client A’s buffer to the network, the thread writes to Client B’s buffer. When A’s buffer has room again, the thread comes back and writes more. The rhythm is: read a little, write a little, buffer full, move on, come back later. No client gets the thread’s full attention, so each one’s throughput is lower, but every client makes progress and the server stays responsive to all of them. That’s the tradeoff, and for almost every network application, it’s exactly what you want.</p><p>This is also fundamentally how tools like Redis achieve their performance: a single thread, non-blocking I/O, never wasting a cycle waiting on a socket.</p><h3>The Developer’s Burden</h3><p>The tradeoff: you now have to track partial sends yourself. If send() only writes 2 KB out of 10 KB, those remaining 8 KB are your responsibility. Here&#39;s the non-blocking version of our server:</p><pre>import socket<br>import select<br>from pathlib import Path<br><br>file = Path(&quot;smallfile.txt&quot;)<br>large_data = file.read_bytes() * 1000      # big enough that send() can&#39;t push it all at once<br><br>server = socket.socket()<br>addr = (&quot;localhost&quot;, 8888)<br>server.bind(addr)<br>server.listen()<br><br>read_wait = {server}                       # sockets select() should watch for incoming data<br>write_wait = set()                         # sockets we still owe data to<br>send_buffers = {}                          # socket -&gt; bytes not yet sent (the bookkeeping)<br><br>while read_wait:<br>    # block until at least one socket is ready to read or write<br>    readers, writers, _ = select.select(read_wait, write_wait, [])<br><br>    for reader in readers:<br>        if reader is server:<br>            # new client knocking<br>            client_sock, addr = server.accept()<br>            client_sock.setblocking(False)  # send/recv return immediately, never stall<br>            print(f&quot;Got new connection from {addr}&quot;)<br>            read_wait.add(client_sock)<br>        else:<br>            data = reader.recv(1024)<br>            if not data:<br>                # empty recv = client closed the connection<br>                read_wait.remove(reader)<br>                reader.close()<br>            else:<br>                # queue the reply instead of sending it right now<br>                send_buffers[reader] = large_data<br>                write_wait.add(reader)<br><br>    for writer in writers:<br>        data_to_send = send_buffers.pop(writer)<br>        sent = writer.send(data_to_send)    # non-blocking: may send only PART of it<br>        if sent &lt; len(data_to_send):<br>            # put the leftover back; select() will tell us when to continue<br>            send_buffers[writer] = data_to_send[sent:]<br>        else:<br>            # all sent, stop watching this socket for writes<br>            write_wait.remove(writer)</pre><p>We now have send_buffers, write_wait, and we&#39;re tracking how many bytes were actually sent. If send() only sent a portion of the data, we slice off what was sent and store the remainder back in send_buffers so we can come back to it later. We&#39;re also using select with <em>both</em> read_wait and write_wait now: we need to know when a socket is ready for writing, not just reading.</p><p>It works, but it’s a lot of bookkeeping. And this is a simple echo server. Imagine doing this for a real application with multiple message types, connection state, timeouts, and error handling. This is where asyncio comes in.</p><h3>Asyncio Streams: The Whole Package</h3><p>Asyncio uses non-blocking sockets under the hood. It uses the same select/epoll/kqueue mechanisms we&#39;ve been talking about. But it handles all the painful parts for you:</p><pre>import asyncio<br>from asyncio import StreamWriter, StreamReader<br>from pathlib import Path<br><br>file = Path(&quot;smallfile.txt&quot;)<br>large_data = file.read_bytes() * 1000  # Simulating large data<br><br>async def handle_echo(reader: StreamReader, writer: StreamWriter):<br>    while True:<br>        data = await reader.read(1024)<br>        if not data:<br>            break<br>        writer.write(large_data)<br>        # await writer.drain()<br>    writer.close()<br>    await writer.wait_closed()<br><br>async def main():<br>    server = await asyncio.start_server(handle_echo, &#39;127.0.0.1&#39;, 8888)<br>    addr = server.sockets[0].getsockname()<br>    print(f&quot;Serving on {addr}&quot;)<br>    async with server:<br>        await server.serve_forever()<br><br>asyncio.run(main())</pre><p>Same large_data, same behavior, but no send_buffers dict, no write_wait set, no manual byte slicing. Asyncio&#39;s streams API handles all of that internally. It maintains an application-level buffer, keeps track of what&#39;s been sent and what hasn&#39;t, and automatically writes data into socket buffers when they&#39;re ready.</p><p>Notice that await writer.drain() is commented out. We&#39;ll come back to why.</p><h3>What await Actually Does</h3><p>When the coroutine hits await reader.read(1024), it doesn’t block the thread. Instead, it’s saying: “I have nothing to do until data arrives on this socket, so go run something else.” The asyncio event loop takes over, checks which other sockets are ready, runs the coroutines waiting on them, and returns to this coroutine once its data arrives.</p><p>This is cooperative scheduling: coroutines voluntarily give up control at await points. Threads work differently. The OS preemptively switches between them whenever it decides to, and each switch is a full context switch, which is far more expensive than resuming a coroutine. The flip side of cooperation is that a coroutine that never awaits, such as one stuck in a long computation, holds up every other coroutine on the loop.</p><h3>A Note on writer.drain()</h3><p>Remember the commented-out await writer.drain() in the code above? If you run the server without it, everything works and both clients receive their data. So why bother?</p><p>In our select server, we had to keep track of any data that wasn’t fully sent after a partial send() and try again later. Asyncio does that bookkeeping for you. writer.write() doesn’t write to the socket directly; it appends to asyncio’s internal buffer, and asyncio flushes that buffer into the kernel’s socket buffer in the background, handling partial sends and tracking what’s been sent. (In recent versions of CPython, when a send() only partially succeeds, asyncio holds onto the unsent portion as a memoryview, a lightweight reference to the remaining slice, rather than copying those bytes into a new buffer.)</p><p>The catch every write() call appends to the buffer, no matter how much is already waiting. If your code produces data faster than the network can send it, the buffer keeps growing, and your server’s memory grows silently with it until the process runs out.</p><p>That’s where drain() comes in. When you await writer.drain(), asyncio checks how much data is still waiting in the buffer. If it’s grown too large, your coroutine pauses until enough of it has been flushed to the network, then resumes. Instead of piling data into the buffer forever, your code slows down to match what the network can handle. This is called backpressure.</p><h3>What’s Next: Free-Threaded Python</h3><p>The free-threaded build of Python (no GIL) arrived experimentally in Python 3.13 and became officially supported in 3.14, though it’s still opt-in rather than the default. Without the GIL, threads can finally run Python code truly in parallel across multiple CPU cores.</p><p>That doesn’t mean you should switch from asyncio to one thread per connection. If your workload is I/O-bound, managing ten thousand connections with ten thousand threads, even without the GIL, still means ten thousand stack allocations, ten thousand entries for the OS scheduler to juggle, and a context switch every time a thread blocks on a socket. The overhead of managing the threads can dwarf the actual work being done.</p><p>Where free threading gets interesting is in combining the two approaches. Instead of one event loop on one core, you can run one thread per CPU core, each with its own event loop handling thousands of connections. Each thread does I/O multiplexing exactly as we’ve seen, but now they all run in parallel, so the server can use every core instead of just one.</p><h3>Wrapping Up</h3><p>If there’s one thing I hope you take away, it’s this: asyncio isn’t magic. It’s an event loop built on select/epoll/kqueue, non-blocking sockets, and careful buffer management.</p><p>The next time you write await writer.drain(), you’ll know exactly which buffer it’s waiting on, and why.</p><p><em>This post is based on my talk at PyCon India 2025, </em><a href="https://youtu.be/qDGI5G-C6AQ"><em>watch it here</em></a><em> if you want the live demos.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=11edee3ddcad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/understanding-how-i-o-multiplexing-and-asyncio-streams-work-in-python-11edee3ddcad">Understand I/O Multiplexing and Asyncio Streams work in Python</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Internals of MVCC in Postgres: Hidden costs of Updates vs Inserts]]></title>
            <link>https://medium.com/inspiredbrilliance/internals-of-mvcc-in-postgres-hidden-costs-of-updates-vs-inserts-381eadd35844?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/381eadd35844</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[oltp]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[postgres]]></category>
            <dc:creator><![CDATA[Rohan Reddy Alleti]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 04:35:33 GMT</pubDate>
            <atom:updated>2025-04-01T06:20:47.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tbZHHAUs6ZrN0W04" /><figcaption>Photo by <a href="https://unsplash.com/@abject?utm_source=medium&amp;utm_medium=referral">benjamin lehman</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Why do updates sometimes feel slower than inserts in Postgres? The answer lies deep in how Postgres manages data versions behind the scenes.</p><p>At the core of Postgres’s ability to handle concurrent transactions efficiently is Multiversion Concurrency Control (MVCC).</p><p>In this article, I’ll explore how MVCC works in Postgres and how it impacts both writes and reads, and the performance differences between inserts and updates.</p><h3>Setup</h3><pre>-- pageinspect extension to view underlying data storage, we will use it soon!<br>CREATE EXTENSION IF NOT EXISTS pageinspect;<br><br>-- Create table store<br>CREATE TABLE store (<br>    id SERIAL PRIMARY KEY,<br>    name TEXT NOT NULL,<br>    value INT NOT NULL<br>);<br>-- Disable autovacuum for the table<br>ALTER TABLE store SET (autovacuum_enabled = false);</pre><p>The setup involves creating a sample table called “store” and disabling autovacuum. This will let us observe how Postgres handles versioning (MVCC in action) without automatic cleanup, helping us better understand MVCC working.</p><h3>Overview of MVCC</h3><p>To understand why updates behave differently from inserts, let’s first look at how MVCC works in Postgres.</p><p>Multiversion Concurrency Control (MVCC) is the mechanism Postgres uses to handle concurrent transactions to the same row while maintaining data consistency and isolation.</p><p>Instead of locking rows during reads and writes, Postgres creates multiple versions of a row to allow transactions to operate without blocking each other.</p><p>These multiple versions of a row are called <strong>Tuples</strong>. If a row is inserted, it has 1 tuple associated with it. If the same row is updated, there exists 2 tuples, among which one of them is live and the other is dead.</p><p>Here’s how it works:</p><h4>Write operations</h4><p>Every insert or update query executes within a transaction, and each transaction is assigned a unique transaction ID. While the process also involves writing to the Write-Ahead Log (WAL) and checkpointing, we will not cover those details in this article.</p><p>You can check the current transaction ID in postgres in the following way:</p><pre>postgres=# SELECT txid_current();<br> txid_current<br>--------------<br>          753<br>(1 row)<br><br>postgres=# SELECT txid_current();<br> txid_current<br>--------------<br>          754<br>(1 row)</pre><p>Each row in Postgres has versioning information tied to transaction IDs, which track the row’s state over time. This versioning is managed through the xmin and xmax values, which can be directly queried.</p><pre>postgres=# INSERT INTO store(name, value) VALUES(&#39;score&#39;, 10);<br>INSERT 0 1<br><br>postgres=# SELECT xmin, xmax, * FROM store WHERE id=1;<br><br> xmin | xmax | id | name  | value<br>------+------+----+-------+-------<br>  755 |    0 |  1 | score |    10<br>(1 row)</pre><ul><li><em>xmin</em> represents the ID of the transaction that inserted the row (i.e., created the initial tuple).</li><li><em>xmax</em> represents the ID of the transaction that deleted or invalidated the row. Since xmax here is 0, it means the row is live and not deleted or dead.</li></ul><p>These values also influence row visibility, which we’ll explore in more detail later in the Read Operation section of the article below.</p><p>Let’s take a look at the actual tuple stored in the database to see how MVCC manages row versions internally. We can inspect tuples directly using the pageinspect extension, which allows us to view the raw contents of a table’s underlying pages.</p><p>Postgres stores data in the form of pages on disk, and each page contains multiple tuples. The tuple we’re interested in should ideally be in the first page (i.e., page 0). Let’s use pageinspect to query it and see what’s stored:</p><pre>postgres=# SELECT lp, t_ctid, t_xmin, t_xmax FROM heap_page_items(get_raw_page(&#39;store&#39;, 0));<br><br> lp | t_ctid | t_xmin | t_xmax<br>----+--------+--------+--------<br>  1 | (0,1)  |    755 |      0<br>(1 row)</pre><p>We can see that the xmin and xmax values match the values from a regular SELECT query on the table.</p><ul><li>lp – This is the line pointer, which acts as an index within the page, pointing to the actual location of the tuple. Since lp = 1, it means this is the first and only tuple in the page.</li><li>t_ctid – This is the tuple ID, which points to the physical location of the tuple in the format (page_number, tuple_number). If the row is updated, a new version will be created, and the ctid will point to the new location — more on this later.</li><li>xmin – Represents the transaction ID that inserted the row (in this case, transaction 755).</li><li>xmax – Represents the transaction ID that deleted or invalidated the row. Since xmax = 0, the row is still live.</li></ul><p>A tuple in Postgres consists of both system columns (like xmin, xmax, ctid, etc.) and the actual row data. The system columns help Postgres manage row versions and visibility, which we’ll explore in more detail later.</p><p>When a write (such as an update) occurs, the existing row isn’t modified directly. Instead, <strong>Postgres creates a new row version with the updated values, leaving the old row intact</strong>. The old tuple is then marked as dead but remains available for MVCC purposes (such as for supporting concurrent reads).</p><p>Let’s update the row and inspect the changes:</p><pre>postgres=# UPDATE store set value=20 where id=1;<br>UPDATE 1<br><br>postgres=# SELECT xmin, xmax, * FROM store WHERE id=1;<br><br> xmin | xmax | id | name  | value<br>------+------+----+-------+-------<br>  756 |    0 |  1 | score |    20<br>(1 row)</pre><p>The row now has a new xmin value (756), which represents the transaction ID that created this new tuple. Due to MVCC, we expect that the original row (now a dead tuple) still exists alongside the new one.</p><p>We can confirm this by checking the number of live and dead tuples using the pg_stat_all_tables view:</p><pre>postgres=# SELECT n_live_tup, n_dead_tup, relname FROM pg_stat_all_tables WHERE relname = &#39;store&#39;;<br><br> n_live_tup | n_dead_tup | relname<br>------------+------------+---------<br>          1 |          1 | store<br>(1 row)</pre><p>As expected, there is 1 live tuple (the updated row) and 1 dead tuple (the original row). Now let’s inspect the underlying page to see both tuples:</p><p>Lets query the page to see the tuples again.</p><pre>postgres=# SELECT lp, t_ctid, t_xmin, t_xmax FROM heap_page_items(get_raw_page(&#39;store&#39;, 0));<br> lp | t_ctid | t_xmin | t_xmax<br>----+--------+--------+--------<br>  1 | (0,2)  |    755 |    756<br>  2 | (0,2)  |    756 |      0<br>(2 rows)</pre><p>We now see two tuples as expected:</p><ul><li>The first tuple has xmin = 755 (from the original insert) and xmax = 756, indicating that it was invalidated by transaction 756 (the update).</li><li>The second tuple is the new version created by the update, with xmin = 756 and xmax = 0 (indicating it’s still valid).</li><li>Notice that the t_ctid of the original tuple has changed from (0,1) to (0,2). This is because t_ctid points to the most recent version of the row. The format (page_number, line_pointer) means that the latest version of the row is in page 0, line pointer 2.</li></ul><h4><strong>Heap Only Tuples (HOT)</strong></h4><p>This behavior is part of a Postgres optimization called Heap-Only Tuples (HOT), where on an UPDATE instead of immediately updating the index to point to the new tuple(which would be costly), Postgres updates the old tuple’s t_ctid to point to the new version. This creates a chain of tuples — the index leads to an old tuple, the old tuple points to the new one and so on.</p><p>Postgres relies on the vacuum process to clean up the dead tuple and update the index later.<br>You can read more about HOT in the official Postgres documentation:<br>👉 <a href="https://www.postgresql.org/docs/current/storage-hot.html">https://www.postgresql.org/docs/current/storage-hot.html</a></p><h3>Vacuum Process</h3><p>You may have noticed that vacuuming was disabled in the earlier example. This was intentional, as it allowed us to observe the dead tuples and the HOT chain, which would otherwise have been cleaned up by the vacuum process.</p><p>Vacuum plays a critical role in Postgres by permanently removing dead tuples, reclaiming storage, and updating indexes to eliminate HOT chains — all of which help improve read performance. It also helps prevent table bloat by freeing up storage space occupied by dead tuples.</p><p>Vacuum usually is configured to run automatically, but can also run manually. Lets try vacuuming our table which contains 1 dead tuple.</p><pre>postgres=# SELECT n_live_tup, n_dead_tup, relname FROM pg_stat_all_tables WHERE relname = &#39;store&#39;;<br><br>n_live_tup | n_dead_tup | relname<br>------------+------------+---------<br>          1 |          1 | store<br>(1 row)<br>--- We have 1 dead tuple<br>--- Manually vacuum &#39;store&#39; table<br>postgres=# VACUUM store;<br>VACUUM<br>--- We endup with 0 dead tuples<br>postgres=# SELECT n_live_tup, n_dead_tup, relname FROM pg_stat_all_tables WHERE relname = &#39;store&#39;;<br> n_live_tup | n_dead_tup | relname<br>------------+------------+---------<br>          1 |          0 | store<br>(1 row)<br>postgres=# SELECT lp, t_ctid, t_xmin, t_xmax FROM heap_page_items(get_raw_page(&#39;store&#39;, 0));<br> lp | t_ctid | t_xmin | t_xmax<br>----+--------+--------+--------<br>  1 |        |        |<br>  2 | (0,2)  |    756 |      0<br>(2 rows)</pre><ul><li>We can see only one tuple when inspecting the page, located at lp 2, as the old tuple at lp 1 has been removed.</li></ul><p>However, vacuuming a table comes with a cost. It consumes CPU and memory, which can create a performance bottleneck by using resources that could otherwise be dedicated to reads and writes. Additionally, the time required for vacuuming can vary from seconds to hours, depending on the volume of cleanup needed.</p><p>Vacuum is critical for Transaction Wraparound. You can read more about it here 👉 <a href="https://www.postgresql.org/docs/current/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND">https://www.postgresql.org/docs/current/routine-vacuuming.html#VACUUM-FOR-WRAPAROUND</a></p><h3>Impact on Read Performance</h3><p>MVCC affects not just writes but also read performance. When a query is executed, Postgres needs to determine which row version is valid, which adds some overhead:</p><h4>1. Row Visibility Checks</h4><p>Each time a row is read, Postgres checks its xmin and xmax values to determine if it’s the latest version. This adds some processing time, especially when there are multiple row versions due to frequent updates.</p><h4>2. Table Bloat</h4><p>When an update creates a new row version, the old version remains in the table until a VACUUM operation removes it. Over time, this increases table size and slows down reads, as Postgres may need to scan multiple row versions to find the valid one.</p><p>Frequent VACUUM and AUTOVACUUM help reduce bloat and improve read performance.</p><h4>3. Updates vs Inserts in Write-Heavy Workloads</h4><p>Read latencies can vary depending on whether you are updating existing rows or inserting new ones:</p><ul><li><strong>Inserts</strong> — Directly add a new row without affecting existing ones.</li><li><strong>Updates</strong> — Create a new row version and mark the old one as dead, increasing table size and requiring more work during reads to find the latest version.</li></ul><p>For updates, Postgres may need to adjust index pointers since a new tuple is created for each update. However, if the update qualifies for a <strong>HOT (Heap-Only Tuple) update</strong> — meaning the index does not need to be updated immediately, which improves efficiency.</p><p>However, HOT updates create chains of tuples in the heap, which increases the work for VACUUM. Postgres needs to follow the chain to find the latest valid version during reads, and once VACUUM removes old versions, the index may still need to be adjusted to reflect the latest state.</p><p>Frequent autovacuum helps clean up dead rows and maintain consistent performance. If you notice slower reads in a write-heavy workload, reducing update frequency or adjusting your data model might help.</p><p>This isn’t to suggest that you should avoid updates altogether. However, if you experience higher read latencies in a write-heavy workload, it’s worth considering whether high update frequency is contributing to the issue. Optimizing your data model to reduce or avoid excessive updates could help mitigate this problem and improve overall performance.</p><p>For extremely write-heavy cases, a wide-column store like Cassandra or ScyllaDB, which are optimized for high write throughput, could provide better performance.</p><h3>Impact on Storage space</h3><p>Due to MVCC, Postgres consume more disk space the more updates we do, and vacuum is responsible for reclaiming that space.</p><p>Lets demonstrate this with an example, I am going insert 1 Million records and perform a few updates. We will also look at the storage space taken by the table at each step.</p><pre>postgres=# truncate store;<br>TRUNCATE TABLE<br><br>--- Insert a Million records<br>postgres=# DO $$<br>BEGIN<br>    FOR i IN 1..1000000 LOOP<br>        INSERT INTO store (name, value)<br>        VALUES (&#39;Name_&#39; || i, i * 10);<br>    END LOOP;<br>END $$;<br>DO<br>postgres=# SELECT COUNT(*) FROM store;<br>  count<br>---------<br> 1000000<br>(1 row)</pre><p>Lets look at the space occupied by this table.</p><pre>postgres=# SELECT<br>    relname AS table_name,<br>    pg_size_pretty(pg_total_relation_size(relid)) AS total_size<br>FROM<br>    pg_catalog.pg_statio_user_tables<br>WHERE<br>    relname = &#39;store&#39;;<br><br>table_name | total_size<br>------------+------------<br> store      | 71 MB<br>(1 row)</pre><p>These 1 Million records take up about 71 MB on disk. Lets update all the rows and check the store space (we still have auto vacuum disabled).</p><pre>postgres=# UPDATE store SET value=2000;<br>UPDATE 1000000<br><br>postgres=# SELECT<br>    relname AS table_name,<br>    pg_size_pretty(pg_total_relation_size(relid)) AS total_size<br>FROM<br>    pg_catalog.pg_statio_user_tables<br>WHERE<br>    relname = &#39;store&#39;;<br> table_name | total_size<br>------------+------------<br> store      | 142 MB<br>(1 row)</pre><p>As expected, the size of the table as doubled due to each row having 2 tuples (1 dead and 1 live) after the update statement.</p><p>Lets manually vacuum the table and see if the storage space reduces.</p><pre>postgres=# VACUUM store;<br>VACUUM<br><br>postgres=# SELECT<br>    relname AS table_name,<br>    pg_size_pretty(pg_total_relation_size(relid)) AS total_size<br>FROM<br>    pg_catalog.pg_statio_user_tables<br>WHERE<br>    relname = &#39;store&#39;;<br><br>table_name | total_size<br>------------+------------<br> store      | 142 MB<br>(1 row)<br><br>postgres=# SELECT n_live_tup, n_dead_tup, relname FROM pg_stat_all_tables WHERE relname = &#39;store&#39;;<br><br>n_live_tup | n_dead_tup | relname<br>------------+------------+---------<br>    1000000 |          0 | store<br>(1 row)</pre><p>Interestingly, after vacuuming the table, the storage space remains at 142 MB, but the number of dead tuples has dropped to zero, confirming that vacuum successfully cleaned up the tuples.</p><p>This is the expected behavior of vacuum — it doesn’t reduce the total storage space but instead marks the freed space as reusable. This means that any new data will be written into the existing storage space rather than increasing the overall disk usage.</p><p>However, in some cases, this might not be ideal. If you want to actually reclaim the disk space, you can run a VACUUM FULL operation:</p><pre>postgres=# VACUUM FULL store;<br>VACUUM<br><br>postgres=# SELECT<br>    relname AS table_name,<br>    pg_size_pretty(pg_total_relation_size(relid)) AS total_size<br>FROM<br>    pg_catalog.pg_statio_user_tables<br>WHERE<br>    relname = &#39;store&#39;;<br> table_name | total_size<br>------------+------------<br> store      | 71 MB<br>(1 row)</pre><p>In this example, VACUUM FULL has reclaimed the physical disk space that was previously marked as reusable or unused, reducing the overall storage size.</p><h3>Conclusion</h3><p>We have covered how MVCC works in detail as well as the importance of the vacuum process for performance.</p><p>There are still a lot more things that I haven’t touched in this article, and I would encourage you to read about some of these topics (or unless I decide to write about them sometime in the future):</p><ul><li>VACUUM ANALYSE operation</li><li>Re-index operation</li><li>Transaction wraparound</li><li>Tuning auto vacuum</li></ul><p><strong>Takeaway</strong>: MVCC allows Postgres to handle concurrent transactions efficiently but introduces trade-offs. Inserts typically lead to faster reads/performance than updates because updates create new row versions, which can lead to bloat unless properly managed by vacuuming. Understanding MVCC and HOT helps you fine-tune your database for better performance.</p><h3>Resources</h3><ul><li><a href="https://www.notion.com/blog/sharding-postgres-at-notion">https://www.notion.com/blog/sharding-postgres-at-notion</a></li><li><a href="https://blog.sentry.io/transaction-id-wraparound-in-postgres/">https://blog.sentry.io/transaction-id-wraparound-in-postgres/</a></li><li><a href="https://www.youtube.com/watch?v=GtQueJe6xRQ">https://www.youtube.com/watch?v=GtQueJe6xRQ</a></li><li><a href="https://www.interdb.jp/pg/pgsql07/01.html">https://www.interdb.jp/pg/pgsql07/01.html</a></li><li><a href="https://www.postgresql.org/docs/current/storage-hot.html">https://www.postgresql.org/docs/current/storage-hot.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=381eadd35844" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/internals-of-mvcc-in-postgres-hidden-costs-of-updates-vs-inserts-381eadd35844">Internals of MVCC in Postgres: Hidden costs of Updates vs Inserts</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The AI -Ready Enterprise: Preparing Data for the Age of AI]]></title>
            <link>https://medium.com/inspiredbrilliance/the-ai-ready-enterprise-preparing-data-for-the-age-of-ai-a4c12ec9716a?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/a4c12ec9716a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Karthika Vijayan]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 04:35:27 GMT</pubDate>
            <atom:updated>2026-09-11T04:35:26.067Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A practical framework for building data systems that AI can find, understand, trust, and use.</em></p><p>Co-authored with <a href="https://medium.com/u/67a8c14d1eff">Sujit Kamthe</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IJhjKO2UNR9QcbOdt0bMHw.png" /></figure><p>Conversations around enterprise AI are moving rapidly from models to applications. Foundational models are becoming increasingly capable, and organisations are experimenting with retrieval augmented generation (RAG), tool calling, AI agents and multi-agent systems to bring intelligence into their business processes. Yet, as these systems move closer to production, a familiar constraint emerges: the data they need was rarely designed for AI consumption.</p><p>Enterprise data has traditionally been organized around applications, reporting, transactions and human decision-making. AI introduces a different set of requirements on data. It needs to find relevant information, understand meaning, establish connections across data sources, assess whether it can trust the information at hand, and consume information in a form suitable for reasoning and action. This leads to a fundamental question, what does it actually mean for enterprise data to be AI-ready?</p><h3>A layered view of AI-ready data</h3><p>AI readiness is not a binary state. Connecting a large language model (LLM) to an enterprise database, or adding a vector database to an existing data platform, does not make the underlying data AI-ready. Readiness develops through a set of capabilities, each addressing the requirements for AI systems. AI-ready enterprise data can be understood as a layered set of capabilities, as shown in the figure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7vT2EX2ysIj97bfVa7zAFw.png" /><figcaption>The layered framework: each layer makes enterprise data more usable by AI.</figcaption></figure><p>The foundation is an <strong>enterprise data organization </strong>layer, where structured and unstructured data are maintained at appropriate granularity, with consistency, quality, temporal fidelity and governance. This is followed by machine-consumable <strong>metadata and semantic context</strong> layer, that allows AI systems to understand what data represents, where it comes from and whether it is appropriate for a given task. The unstructured enterprise data including documents, reports, policies, etc. are contained in the <strong>retrieval-ready knowledge </strong>layer so that this content can be found by AI using their meaning rather than by exact filenames or identifiers.</p><p><strong>Data governance and control</strong> layer provide the policies, lineage, access controls, consent, and compliance context that allow AI systems to use information responsibly. <strong>Enterprise knowledge integration</strong> layer connects shared entities and relationships across structured data, unstructured content, metadata, and governance. Finally, <strong>insights and signals</strong> layer makes frequently used business logic, metrics, features, and predictions directly reusable by AI applications.</p><p>Together, these layers make data progressively more machine-processable, machine-understandable, discoverable, trustworthy, connected, and ready for AI consumption.</p><h3>Four systems that operationalise the layered framework</h3><p>The layered framework becomes actionable through four complementary systems: the <strong>System of Record</strong>, the <strong>System of Understanding</strong>, the <strong>System of Discovery</strong>, and the <strong>System of Action</strong>. A <strong>unified discovery plane</strong> provides an interface for AI systems to consume capabilities exposed by these four systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cqBdiTR6dQGMRFyblY34lA.png" /><figcaption>The four systems: governed data, understanding, discovery, and action work together through a unified discovery plane.</figcaption></figure><h4><strong>System of Record</strong></h4><p>The System of Record provides the governed foundation for structured and unstructured enterprise data. A <a href="https://en.wikipedia.org/wiki/Lakehouse">lakehouse </a>can support progressive refinement, schema evolution, transactional guarantees, versioning, and time travel for structured data. Additionally, governed ‘<em>object and document storage</em>’ provide equivalent foundations for unstructured content. This system can also host feature stores, metric stores, and derived signals. That lets an organization compute valuable business insights once and reuse it across AI applications. A time-tagged training dataset, a governed customer risk score, or a derived business metric becomes a reusable enterprise asset instead of something that every application has to recompute each time.</p><h4>System of Understanding</h4><p>Data is useful to AI only when its meaning is associated with it. The System of Understanding provides machine-consumable descriptions of schemas, business definitions, ownership, quality, freshness, lineage, and policies. It may include a metadata catalog and an enterprise knowledge graph that captures the entities and relationships connecting information across domains. For example, an enterprise knowledge graph can connect a customer record in a database, a mention in a contract document, related transactions in a lakehouse, and current state available through an API. Organizations can start with semantic linking and entity resolution, then evolve toward richer graph representations. <a href="https://github.com/microsoft/graphrag">GraphRAG </a>can complement semantic retrieval when an AI system must reason across explicit relationships.</p><h4>System of Discovery</h4><p>Traditional discovery assumes that the consumer already knows which dataset, schema, or column contains the answer. AI systems cannot make that assumption. The System of Discovery provides an AI-facing retrieval surface over enterprise knowledge. It uses vector indexes and semantic retrieval to find relevant documents, metadata, reports, knowledge-graph context, and other information based on their meanings.</p><p>RAG is especially effective when knowledge is distributed across a large collection of unstructured content. Graph traversal provides a complementary path when the question depends on explicit relationships. The discovery layer is not the authoritative source of enterprise data; it is a retrieval and access surface over governed assets maintained by the Systems of Record and Understanding.</p><p><em>At these three systems, data quality becomes actionable for AI consumption. Experienced teams often know, through tribal knowledge, which data sources are complete, current, or reliable enough for a particular decision. AI systems cannot infer that reliably from a table name or a retrieved document. Quality metrics, validation status, freshness, lineage, and ownership must be available as governed metadata or retrieval-ready information, so that the AI can assess whether a source is fit for the task. Otherwise, weak data is likely to produce weak responses.</em></p><h4>System of Action</h4><p>The System of Action is where AI systems turn the capabilities from data into decisions and actions. An AI agent should be able to query structured data, retrieve documents, traverse enterprise knowledge graphs, access reusable signals, and invoke real-time APIs without needing to understand the implementation details of every backend system.</p><p>The unified discovery plane can bring these capabilities together through a consistent, AI-oriented interface. It should also carry the context needed for responsible use: access controls, policies, quality signals, and provenance should travel with the data and tools that an agent uses.</p><h3>Data for AI and AI for Data</h3><p>The architecture has a second direction. AI does not only consume AI-ready data; it can help create and maintain it. This is the <strong>AI for Data</strong> dimension, which complements the <strong>Data for AI</strong> foundation.</p><p>AI can help classify sensitive information, enrich metadata, process documents, extract and resolve entities, evolve knowledge graphs, perform advanced data-quality checks, and generate reusable predictive signals. Feedback from agent usage, failures, and quality signals can then inform upstream improvements to data preparation and curation.</p><p>That does not mean every data-preparation task should invoke an LLM. Smaller domain-specific models, traditional machine learning, deterministic rules, asynchronous processing, batching, and event-driven workflows are often better suited to high-volume or well-defined tasks. <a href="https://sahaj.ai/agentic-ai-building-the-right-mental-model-for-system-design/">LLMs should be reserved for work where semantic reasoning adds meaningful value </a>.</p><h4>The framework in practice</h4><p>Consider an out-of-home media-planning AI assistant asked to recommend high-performing advertising frames in Chicago for young professionals, subject to inventory availability in Q3 of the year, and to compare the recommendation with similar campaigns from the previous year.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h4e47vdnBXZ9mmig7k-vLw.png" /><figcaption>A practical example: the System of Action combines governed data, connected context, and retrieval-ready knowledge to produce a grounded recommendation.</figcaption></figure><p>The System of Record supplies inventory, audience scores, frame-performance metrics, and historical campaign results. The System of Understanding connects audiences, locations, frames, campaigns, and the business meaning of performance measures such as reach, coverage, and frequency. It also exposes the quality and freshness information the AI assistant needs to judge whether the data is suitable for the recommendation. The System of Discovery retrieves historical campaign briefs, planning documents, audience research, and market reports. The System of Action brings that evidence together: it applies the current inventory constraints, identifies comparable campaigns, and generates a recommendation with supporting rationale and historical context.</p><p>The quality of the answer depends on more than the model. It depends on whether the enterprise has prepared the data, context, governance, and reusable signals that the model needs to reason well.</p><h4>Preparing for durable enterprise AI</h4><p>AI readiness is a data-engineering challenge as much as an AI challenge. More sophisticated agents can improve an application only to the extent that they can work with data that is discoverable, interpretable, trustworthy, connected, and usable.</p><p>The durable path is to build those capabilities progressively. Start by organizing and governing enterprise data. Make its meaning, quality, freshness, lineage, and policies visible to machines. Prepare knowledge for retrieval, connect entities across domains, and materialize high-value signals for reuse. Then let AI systems consume those foundations and help improve them over time.</p><p>As models become more capable, the differentiator will increasingly be the quality of the enterprise knowledge they can safely use. AI-ready data turns that knowledge from a passive asset into an active participant in reasoning and action.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a4c12ec9716a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/the-ai-ready-enterprise-preparing-data-for-the-age-of-ai-a4c12ec9716a">The AI -Ready Enterprise: Preparing Data for the Age of AI</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI: Building the Right Mental Model for System Design]]></title>
            <link>https://medium.com/inspiredbrilliance/agentic-ai-building-the-right-mental-model-for-system-design-269d85c1689f?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/269d85c1689f</guid>
            <dc:creator><![CDATA[Karthika Vijayan]]></dc:creator>
            <pubDate>Fri, 14 Aug 2026 16:52:41 GMT</pubDate>
            <atom:updated>2026-08-14T16:52:40.195Z</atom:updated>
            <content:encoded><![CDATA[<p><strong><em>The complexity of the agentic system should emerge from the complexity of the problem, rather than the other way around.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WCaFX4hSd40_qF5IoiBXFg.png" /></figure><p>AI agents have evolved from Large Language Models (LLMs) that primarily generate responses to systems capable of reasoning and dynamic decision-making. By interpreting goals, inferring what is needed, and integrating external capabilities such as APIs and databases, the LLM serves as the decision-making core of an agent. Agentic AI refers to systems built around these intelligent agents.</p><p><strong>AI Agents</strong></p><p>An AI agent extends beyond a simple prompt by combining an LLM’s reasoning capability with functional layers such as tools, memory and orchestration. Tools provide additional functionality and programmatic access, memory maintains relevant context, and planning and orchestration coordinate execution. These functional capabilities that assist an LLM to be an AI agent can be visualised as layers as shown in the figure below [1]. The complexity of an agent can be built progressively by adding the layers required by the task. With addition of more layers, the agency of the system over tasks increases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B5tI2FcZkqJ2M9c3Q-wL2A.png" /></figure><p>The building blocks of the agent stack come together through a workflow, where tools, deterministic components and intelligent components interact to achieve a goal. In a conventional workflow, the sequence of steps is predefined; in an agentic workflow, some of those steps can be determined dynamically based on the situation. Within this framework, we can define an agent.</p><p>In my opinion, an agent is an LLM-driven component that integrates <strong>at least</strong> two layers of the agent stack shown above, distinguishing it from a simple LLM call with a prompt. True agency emerges when these functional layers such as tools, memory, or planning enable the component to navigate parts of the workflow dynamically. The LLM provides the reasoning capability, while these additional layers enable it to act beyond generating a response.</p><p><strong>From Agents to Applications</strong></p><p>With the distinction between workflows and agents established, the important question for any project is not “<strong>How do I build an agent?</strong>” but “<strong>Does this problem need an agent at all?</strong>” This is where the right mental model becomes important. Start with the business application and its objectives, and decompose what the system needs to accomplish. Is it retrieving information, making a recommendation, scheduling something, taking an action, or combining several of these? For each capability, determine whether it can be implemented deterministically or requires the system to reason and dynamically decide what to do next. Some tasks may need conventional software, some an LLM within a predefined workflow and only some may warrant an agent. This naturally leads to a deterministic-first approach: use conventional software where the behaviour is known, and introduce intelligence where it adds value.</p><p>The next step is not necessarily to introduce an agent. An LLM can first be used as a capability within the application, for example, to summarise information, extract entities, classify a request, or generate response to a question. Multiple such capabilities can then be composed into a workflow, where a predefined sequence of steps is used to accomplish a task. The steps may involve LLM calls, deterministic code, tools, or other application components, and may execute sequentially or in parallel, but the application determines the overall flow. For example, one LLM call may extract information, two calls may independently analyse different aspects, and a second one may combine their outputs. This remains a workflow because the sequence and coordination are predetermined.</p><p><strong>Scoping the Agentic Application</strong></p><p>Agentic behaviour enters when the system needs to dynamically determine a part of that workflow. To make this decision, we first need to scope the application. Scoping can be approached from two complementary perspectives, the <strong>consumer</strong> and the <strong>source</strong>. From the consumer perspective, identify what the application needs to provide; what features users need, what decisions it must support and what actions it should perform. From the source perspective, identify the data and functional sources required to support those features, such as databases, APIs, documents, search systems, or other applications. Together, these perspectives map what the application needs to do to what it needs to know or access.</p><p>Once the scope is established, the first question is whether the identified capabilities can be brought together within a <strong>single agent</strong>. An agent can combine the required tools, memory, and planning capabilities and coordinate the workflow to achieve the application objective. This is often the simplest starting point, particularly when the capabilities are closely related and can share the same context. The agent can reason over the task, select the appropriate tools, and determine the sequence of actions required, keeping the overall architecture relatively simple. For example, an agent tasked with generating a business report may query a database to retrieve relevant data, call an API to obtain additional information, analyse the results, and then produce the report, coordinating these steps toward a single capability.</p><p><strong>Pivot to Multi-Agent System</strong></p><p>As the scope of the application becomes broader, different capabilities may require different forms of reasoning, tools, knowledge, or context. At this point, combining everything into a single agent can make its responsibilities increasingly difficult to define and maintain. The question then changes from <em>“What can this agent do?”</em> to <em>“Should these responsibilities belong to the same agent?”</em> This provides a natural point to consider a <strong>multi-agent system</strong>, where specialised agents are assigned distinct capabilities and collaborate through a workflow. The objective is not to increase the number of agents, but to establish meaningful boundaries between responsibilities.</p><p>The same consumer and source perspectives used during scoping can help determine these boundaries. From the <strong>consumer perspective</strong>, capabilities that have distinct objectives or require different reasoning approaches may be separated into specialised agents. From the <strong>source perspective</strong>, different data or functional sources may require specialised capabilities. For example, a SQL agent for database access when the number of tables increases and table-schema becomes complex, an API agent for application services, or a RAG-based component for documents. These agents can then be coordinated through a sequential, hierarchical or dynamic workflow, allowing each agent to focus on a well-defined responsibility while the overall system works toward the application objective.</p><p><strong>The Agentic AI Mental Model</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OS4a7ghWkGBhlTO-LfT8vw.png" /></figure><p>This mental models aids us with a useful progression for designing agentic systems. We begin with the business objective, decompose it into capabilities, determine whether each capability requires deterministic logic, an LLM, a workflow, or an agent, and then decide whether those agents should operate independently or collaborate as a multi-agent system. A single agent should therefore be the starting point rather than an assumption that the system must eventually become multi-agent. As the problem demands greater separation of capabilities, specialised knowledge, or distinct decision-making responsibilities, multiple agents may provide a more appropriate architecture. Thus, the complexity of the agentic system should emerge from the complexity of the problem, rather than the other way around.</p><p>The importance of this mental model lies in recognising that greater intelligence does not necessarily mean a better system. Introducing an LLM, a workflow, or an agent adds computational cost, latency, and architectural complexity, and these should be justified by the value they bring to the application. A deterministic component may be more reliable and efficient for a well-defined task, while an LLM or an agent becomes valuable when the problem requires interpretation, reasoning, or dynamic decision-making. Thus, the progression from deterministic components to LLM capabilities, workflows, and agents provides a way to introduce intelligence deliberately, avoiding unnecessary complexity and ensuring that the use of agentic AI is driven by the problem it solves rather than by the technology trend itself.</p><p><strong>References</strong></p><p><a href="https://livebook.manning.com/book/multi-agent-systems-with-autogen-cx/welcome/v-3#6">[1]</a> Victor Dibia, “Multi-Agent Systems with AutoGen”.</p><p>[2] Anand, Oshin and Vijayan, Karthika, A Framework for Building Robust AI Agents (July 01, 2026). Available at SSRN: <a href="https://ssrn.com/abstract=7261000">https://ssrn.com/abstract=7261000</a> or <a href="https://dx.doi.org/10.2139/ssrn.7261000">http://dx.doi.org/10.2139/ssrn.7261000</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=269d85c1689f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/agentic-ai-building-the-right-mental-model-for-system-design-269d85c1689f">Agentic AI: Building the Right Mental Model for System Design</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering the AI Partnership: Making AI-Accelerated Delivery Reliable]]></title>
            <link>https://medium.com/inspiredbrilliance/engineering-the-ai-partnership-making-ai-accelerated-delivery-reliable-bd982cf3e509?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/bd982cf3e509</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[intelligent-engineering]]></category>
            <category><![CDATA[claude]]></category>
            <dc:creator><![CDATA[Sarulatha Shanmugam]]></dc:creator>
            <pubDate>Thu, 05 Feb 2026 05:49:43 GMT</pubDate>
            <atom:updated>2026-02-05T05:49:41.911Z</atom:updated>
            <content:encoded><![CDATA[<p>We rebuilt an existing website to be CMS-driven: Strapi owned content and page structure, while a modularised Vue frontend handled rendering and behaviour. Over an 8‑week delivery window, we went from <em>“AI is exciting but unpredictable”</em> to shipping <strong>1–2 pages per day</strong> — by treating AI like a teammate with a clear process, not a magic button.</p><p>This post is a <em>standalone case study</em> of that evolution: the workflow we designed, the mistakes we hit early, and the system that kept <strong><em>decisions and quality in human hands while AI accelerated execution</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qMy1-ftyLmEvaT4jyxGKMA.png" /></figure><h3>The learning curve is real</h3><p>When we first got access to Claude Code, using an AI tool at that scale felt new and exciting. It also came with a <em>learning curve</em>: working effectively with AI takes time, energy, and deliberate effort — much like any collaboration.</p><blockquote>The core shift was simple: you can’t delegate unclear thinking.</blockquote><p>AI becomes most effective when you first do the ideation — clarify the intended architecture, the constraints, and what “good” looks like — and then let it execute inside those <strong>guardrails</strong>.</p><h3>Start with a repeatable pattern</h3><p>Once the initial setup was complete, the work became <em>predictable</em>: every page followed the same underlying workflow, even when the UI details differed.<br>Here’s the pattern we stabilised (this became the backbone for everything that came after):</p><ol><li>Take a page and split it into sections.</li><li>For each section, check if an existing component matches exactly.</li><li>If it matches, reuse it.</li><li>If it almost matches, decide whether the change is safe to fold into the existing component without breaking other usages; otherwise create a new component.</li><li>If nothing matches, create the new Strapi component and the corresponding Vue component.</li><li>Map Strapi ↔ frontend via API calls for that page.</li><li>Validate the page loads correctly with CMS data.</li><li>Add tests for the page and any new components.</li></ol><p>This looks obvious written down, but the power came from agreeing that this was the “<strong>default loop</strong>” and treating exceptions as explicit decisions, not improvisation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hLU2Yo4PGjAxFucSYi-law.png" /></figure><h3>Context Engineering, Not Prompting</h3><p>My first scaling attempt was <em>documentation-driven</em>. I used Claude to create a set of documents (a main guide plus supporting notes for existing patterns and test guidelines), then asked it to follow them when recreating each section.</p><p>It worked — until output quality started drifting. As the context grew, it became easier for earlier constraints and decisions to fall out of the model’s working context. I had to do a lot of corrections as it was “forgetting” my earlier instructions.</p><p>So I condensed. Instead of a sprawling doc set, I maintained a single <strong>“source of truth”</strong> file designed to be model-friendly: context, objective, steps to follow, and the gotchas to watch out for. That also reduced the split between “docs for humans” and “context for the model,” because one artifact served both.</p><p>One practical note I learned quickly: <strong><em>these tools love to write</em></strong>. If you don’t explicitly tell them to be brief, you can end up fatigued from reading rather than shipping, so I started instructing the model to keep outputs short unless asked otherwise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_knBDIGt7vxnQxEnFUjNfw.png" /></figure><h3>Chunk It. Command It. Ship It.</h3><p>Even with condensed context, I found that I needed a more reliable <em>abstraction</em> than “here’s a big file, now do the thing.”</p><blockquote>So we chunked the work into repeatable actions and turned them into standalone commands.</blockquote><p>I split the work into two repeatable command shapes:</p><ol><li>New page setup: CMS flow for a new page (routes, splitting sections, extracting constants, wiring the page skeleton).</li><li>Section implementation: CMS + frontend creation/update for one section (including mapping and validation steps).</li></ol><p>Then I used Claude to draft these commands with strict rules:</p><ul><li>Each command was constrained to a specific set of steps.</li><li>Each prompt was standalone and didn’t depend on previous chats.</li><li>The command contained explicit <strong>“decision checkpoints”</strong> where Claude had to ask us what to do, rather than guessing.</li><li>The command follows Anthropic’s guidelines for a Claude command.</li></ul><p>This preserved the right division of labour:</p><blockquote>We owned the decisions; the model owned the execution.</blockquote><p>It also made the workflow shareable — commands lived inside the project and could be reused by others on the team.</p><p><strong><em>A further optimisation: </em></strong>we started running each command in a fresh chat. Because the commands were designed to be self-contained, a new chat gave us a cleaner, larger effective context budget and reduced accidental drift from earlier conversations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*USj1M9tS9EXnIQEFLRZkpw.png" /></figure><h3>Skills: The Unsung Productivity Win</h3><p>The last set of improvements wasn’t about code — it was about everything around it.</p><p>Because we shipped in <em>vertical slices </em>(page-by-page), we had frequent demos and needed a user manual that explained our page/section/component model for the client team.</p><p>Instead of rebuilding decks and docs from scratch each week, I used <strong>Claude Skills</strong> — reusable, shareable workflows — to generate slides from notes and produce documentation artifacts quickly.​</p><p>The most valuable skill was a document wizard for the user manual. The workflow was: generate Markdown, apply a known HTML template for styling, then export a PDF. We used Claude to research options and bootstrap the implementation, then reused that capability throughout the project.</p><blockquote>We didn’t just accelerate code — we accelerated everything around delivery.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j-QR5T6dTV2uiubuoAHBvw.png" /></figure><h3>Habits that made AI reliable</h3><p><em>Living documentation</em> stays valuable when it evolves alongside the work rather than being written once and abandoned.​</p><p>We treated the AI workflow the same way: iteratively improved, lightweight, and embedded in the day-to-day loop.</p><p>The practices that made the biggest difference:</p><ul><li>Review everything: the <strong>AI output is still your responsibility</strong>.</li><li>Keep the model’s output brief by default; ask for depth only when you need it.</li><li>Convert repetitive work into standalone commands with complete context.</li><li>Insert <strong>decision checkpoints</strong> so the model asks rather than assumes.</li><li>Start fresh chats for standalone <strong>commands/prompts</strong> to reduce drift.</li><li>Create <strong>skills</strong> for recurring non-development tasks (slides, manuals, release notes), not just code.</li><li>Ask for a <strong>plan before implementation</strong>, and confirm it before the model starts editing.</li><li>Update your “source of truth” and commands as you learn, so the system stays current.</li></ul><p>AI-accelerated delivery isn’t about finding the perfect prompt — it’s about designing a repeatable way of working that the model can reliably follow. Start small, make the process explicit, and keep humans responsible for decisions and quality while AI handles the execution..</p><p>Done well, AI becomes a force multiplier across discovery, build, and delivery — not a source of risk or rework. Enjoy your journey with this new and exciting technology!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd982cf3e509" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/engineering-the-ai-partnership-making-ai-accelerated-delivery-reliable-bd982cf3e509">Engineering the AI Partnership: Making AI-Accelerated Delivery Reliable</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Naive Automation to Intentional Collaboration with AI]]></title>
            <link>https://medium.com/inspiredbrilliance/from-naive-automation-to-intentional-collaboration-with-ai-c9d61e7b7fc2?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/c9d61e7b7fc2</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[intelligent-engineering]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Sarulatha Shanmugam]]></dc:creator>
            <pubDate>Tue, 03 Feb 2026 13:42:09 GMT</pubDate>
            <atom:updated>2026-02-03T13:42:08.491Z</atom:updated>
            <content:encoded><![CDATA[<p>When Claude Code landed in our team, we dived right in and jumped on the AI train: we asked it to rebuild the site end-to-end with a CMS behind it. We had <strong>20 pages</strong> to recreate in<strong> ~6 weeks</strong> with <strong>two developers</strong>, so <strong><em>“let the model handle it”</em></strong> felt like the obvious move.</p><p>It produced code fast — and we spent the next two days discovering why “fast output” isn’t the same as a system you can extend, test, and hand over.</p><p>This post is the story of that failure, and the pivot that made AI genuinely useful: <em>design first, then let AI fill in the pieces.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8F2jSu3TcwGhPi-qMlmWyA.png" /></figure><h3>The automation plan that backfired</h3><p>The existing codebase was messy — multiple iterations, multiple contributors, no consistent patterns. Recreating static pages using Strapi CMS meant reverse-engineering layouts, rebuilding components, mapping APIs, and preserving styling across two websites.</p><p>Our first attempt: One comprehensive prompt to Claude.</p><ul><li>Analyse the page → break it into sections/components.</li><li>Build Strapi component + matching Vue.js frontend component with the same styling.</li><li>Create Strapi single type for a page → add components → map Strapi → Vue.js components.</li><li>Set up frontend API → render data.</li></ul><p>It worked… sort of. Claude delivered a complete build. But the output was unusable:</p><ul><li>No extension/reuse/modification flexibility.</li><li>Frontend components drifted from the existing design.</li><li>Complex Strapi components with excessive styling inputs (<em>non-technical marketing team nightmare</em>).</li></ul><h3>Two days of salvage attempt</h3><p>We paired for two days on the AI-generated output, trying to salvage something production-ready. Mental exhaustion hit hard and we made essentially no progress — the components were unreadable, the UI felt locked-in, and even small changes became battles.</p><p>A big part of the frustration was that <strong><em>Claude had extracted the page sections brilliantly</em></strong>, but it had built the wrong architecture. It didn’t have the judgment to design the system and data flow we desired.</p><p>The root problems became pretty clear:</p><ul><li>The Strapi components were not reusable and hard to read.</li><li>The UI was inflexible; small changes were painful.</li><li>There were no safeguards against API contract drift.</li></ul><blockquote>That’s when it clicked: <br>AI isn’t a magic wand. It needs to be given specific instructions to do a particular task — the better the instruction, the better the execution — and it needs a way to retain the right context as the work evolves.</blockquote><h3>The pivot: manual baseline first</h3><blockquote>We stopped trying to fix Claude’s output and started trying to fix our process.</blockquote><p>Once we came to that realisation, our approach changed. Instead of telling Claude to “do this,” we focused on designing a system that could stay flexible as requirements changed — and then using AI inside those boundaries.</p><p>We paired on the CMS-to-frontend workflow and used Claude in a narrower role: not “<em>build everything</em>,” but “<em>analyse this page/component and help us reason about patterns.</em>”</p><p>Why the second approach worked:</p><ul><li>We designed for <em>extensibility</em> and <em>flexibility</em> first, so new requirements didn’t force rewrites.</li><li>We picked one page and one component and built the end-to-end data flow (CMS owns content/structure; frontend owns rendering/behaviour).</li><li>We extracted reusable sub-components early, so we weren’t duplicating UI patterns.</li><li>We introduced typed API contracts using an OpenAPI spec, so frontend/back-end drift surfaced immediately.</li><li>We added a CMS page-loading workflow that could handle real needs (routing, feature flags, APIs) without turning every page into a special case.</li><li>We kept existing frontend components mostly intact and modularised cautiously, because the codebase already had similar sections reused with small variations — and shipping pages mattered more than perfect structure.</li></ul><p>After about 2–2.5 days, we had one section built properly end-to-end, and we could finally see the shape of a system we could extend. At that point, we knew what we wanted Claude to do — and just as importantly, what we didn’t want it to do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0OK-v1VAiQz5aaewEWkuzw.png" /></figure><h3>Claude as precise executor</h3><p>With the framework in place, the next challenge wasn’t “can Claude write code?” — it was: <strong><em>can we get Claude to repeat our approach reliably, section after section, without drifting into a new architecture every time?</em></strong></p><p>We started cautiously. For the first Claude-assisted component, I didn’t hand over the whole problem; I fed instructions one step at a time in the order we’d actually build it, correcting mistakes and adding pointers as we went. That first component wasn’t about speed — it was about calibrating Claude to our expectations around flexibility, editor experience, and API boundaries.</p><p>Once it was working, I asked Claude to do something more valuable than generating the next chunk of code: <em>document the process it had just followed</em> — what it did, in what order, and which constraints mattered. We reviewed that write-up, fixed anything ambiguous, and treated it as a living set of instructions that would evolve alongside the implementation.​</p><p><strong>From there, the repeatable loop emerged:</strong></p><ul><li><strong>Next component</strong>: “Follow the documented approach.”</li><li><strong>If it drifted</strong>: Correct it inline.</li><li><strong>Extract learnings</strong>: Ask Claude to summarise anything new not captured.</li><li><strong>Update the document</strong>: Feed verified learnings back in.</li><li><strong>Repeat</strong>: Quality stayed consistent as volume increased.</li></ul><p>That simple loop (<strong>build → review → update the instructions</strong>) turned Claude into a reliable executor while we scaled from one component to a full page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hLU2Yo4PGjAxFucSYi-law.png" /></figure><p>While that was happening, we also parallelised the work: one of us focused on using Claude to finish the remaining components for the page, while the other worked through tests and end-to-end validation. Over the next 2.5–3 days, we finished the full page with the rest of the sections and tests integrated.</p><h3>The real lesson</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ZOF9JfERGc7jFvKiuxgLg.png" /></figure><p>At the end, we realised AI isn’t a developer — and that’s not a criticism, it’s the unlock. It can follow instructions and produce output quickly, but it won’t consistently give you what you expect unless you already know what “right” looks like and can steer it there.</p><p>The good news is: <em>AI really can make our lives easier</em>. There’s just a learning curve, and a big part of that curve is learning how to ideate properly, break work into the right shapes, and prompt with enough clarity that the model can execute instead of improvising.</p><p>For us, the shift was simple: stop asking for “the whole solution,” and start giving it a flexible baseline to operate within. Once we did that, Claude stopped being a source of complexity and became what it’s best at — an accelerator that helps you move faster inside a system you understand and control.</p><p>If you get that relationship right, you don’t just have AI-generated code — you end up with a truly <strong>AI-accelerated</strong> development workflow.​</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9d61e7b7fc2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/from-naive-automation-to-intentional-collaboration-with-ai-c9d61e7b7fc2">From Naive Automation to Intentional Collaboration with AI</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI in Technical Decision-Making: A Field Guide]]></title>
            <link>https://medium.com/inspiredbrilliance/ai-in-technical-decision-making-a-field-guide-122ebdb3fe67?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/122ebdb3fe67</guid>
            <category><![CDATA[ai-engineerig]]></category>
            <category><![CDATA[perplexity-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Sarulatha Shanmugam]]></dc:creator>
            <pubDate>Tue, 03 Feb 2026 13:41:43 GMT</pubDate>
            <atom:updated>2026-02-03T13:41:42.147Z</atom:updated>
            <content:encoded><![CDATA[<p>We had <strong>one week</strong> to run discovery, <strong>two people</strong> to do it, and <strong>eight weeks</strong> to deliver — so instead of debating CMS options in theory, we used AI to <strong><em>rapidly prototype multiple paths</em></strong> and let the client make an evidence-based choice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-9uf7wSHFlfwaigZh5hcJg.png" /></figure><h3>One-week discovery, real stakes</h3><p>We were brought into a project to migrate parts of an existing web application into a Content Management System (CMS) so the marketing team could ship content without constant developer involvement.</p><p>The constraint that shaped everything: we had less than a week to complete discovery (including presenting working prototypes), and then eight weeks for actual development — while maintaining two independently deployed websites inside a single codebase.</p><p>There was also a genuine dilemma from day one: we initially leaned toward a headless approach because it better matched the architecture and long-term workflow, but the client strongly preferred a full WordPress solution — mainly because WordPress is popular, widely supported, and feels like <em>the safe default</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Mmly9MB7PT0mpmuaJ8EjA.png" /></figure><h3>Constraints</h3><p>At first glance, the scope sounded clean: migrate “static” pages into a CMS. In reality, some “static-looking” pages contained dynamic behaviour, which meant we needed strict boundaries: CMS owns content and structure; the application still owns logic and interactivity.</p><p>Here’s the situation we were optimising for:</p><ul><li><strong>Requirements</strong>: Marketing can edit pages and publish blogs without engineering.</li><li><strong>Non‑negotiables</strong>: Preserve the recent visual refresh; keep UI consistent.</li><li><strong>Reality check</strong>: Mixed legacy patterns/templates in the codebase; “static” pages with hidden dynamic edge cases.</li><li><strong>Human constraint</strong>: Marketing team had little/no CMS experience, so workflow simplicity mattered as much as raw flexibility.</li><li><strong>Exit strategy</strong>: Because CMS was new to the project and stakeholders, we needed a realistic ability to roll back or switch direction later without turning it into a rewrite.</li></ul><p><strong><em>That’s a lot to be confident about in under a week.</em></strong></p><p><strong>What the client used to evaluate options</strong></p><p>To avoid a tool debate, we aligned early on the editor workflows the client actually cared about.</p><ol><li>Enabling non-technical teams to update content.</li><li>Rearranging and managing components on a page.</li><li>Adding/removing sections and controlling ordering.</li><li>Independently creating new pages and blog content.</li></ol><h3>What we did differently: parallel prototyping</h3><blockquote>The real problem wasn’t “pick a CMS.” It was: reduce uncertainty fast enough to make a decision we could stand behind .</blockquote><p>So we split the work in parallel as a two-person team:</p><ul><li>One of us took WordPress: research, local setup, and a small prototype that mirrored the kind of pages we actually needed to migrate.</li><li>The other took Strapi: research, local setup, and a comparable prototype aimed at the same content and workflow goals.</li><li>We compared findings daily and used our knowledge of the existing codebase to sanity-check every conclusion against real integration constraints.</li></ul><p>AI helped most when it pulled us out of <em>“documentation land”</em> and into <em>“working baseline”</em> territory — fast. We used it to:</p><ul><li>Accelerate setup and configuration when we got blocked — so we didn’t burn half of discovery on environment friction.</li><li>Turn vague requirements into concrete evaluation questions — <em>“What would content modelling look like for these page types?” or “What would editors actually do day-to-day?”</em>.</li><li>Surface common integration pitfalls early enough to test them in the prototype, not discover them mid-build.</li></ul><p><strong><em>The key is that AI didn’t make the decision. It reduced the cost of testing assumptions.</em></strong></p><h3>The discovery loop that changed the decision</h3><p>By the end of discovery, we weren’t presenting an opinion — we were presenting options the client could interact with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GHSahYrQm34ztb6rT9SYoA.png" /><figcaption>Discovery wasn’t linear — we iterated: prototype → align → client feedback → iterate.</figcaption></figure><p>We showed three paths, not one:</p><ul><li>Full WordPress.</li><li>Headless WordPress.</li><li>Strapi (headless).</li></ul><p>This shifted the conversation from <em>“what’s popular” </em>to <strong><em>“what workflow will we actually run for years.”</em></strong> The client could see what editors could do, what felt confusing, and what would create ongoing engineering overhead.</p><p>We also made the ecosystem tradeoff explicit: staying in the same JavaScript/TypeScript world vs introducing a separate CMS-side ecosystem (PHP + WordPress conventions/plugins) and owning that long-term.</p><p>The most important outcome: we didn’t have to “convince” the client that headless + Strapi was better. We gave them enough concrete evidence — pros/cons plus working prototypes — that they could decide themselves. That decision also surfaced questions that often appear later as scope creep, but here they came up early while change was still cheap.</p><h3>What AI changed (and what it didn’t)</h3><p>AI accelerated discovery, but it never owned decision-making.</p><blockquote>AI is an amplifier, not an authority.</blockquote><p>Where AI helped:</p><ul><li>Getting to <strong>working prototypes quickly</strong>, which let us evaluate real workflows instead of debating features.</li><li>Narrowing the space faster, so our proofs-of-concept were targeted at the <strong>highest-risk unknowns</strong>.</li><li>Making it <strong>realistic</strong> to explore multiple options deeply enough to be honest about tradeoffs under a hard timeline.</li></ul><p>Where human judgement took over:</p><ul><li>Understanding our codebase’s historical quirks and the real cost of “static-looking” pages with behaviour.</li><li>Balancing delivery risk against long-term ownership for both engineering and marketing.</li><li>Managing client dynamics: respecting the WordPress preference, while still creating a process where the client could evaluate alternatives without feeling pushed.</li><li>Protecting an exit strategy: ensuring the approach didn’t lock us into a corner if adoption or constraints changed.</li></ul><h3>Reflection</h3><p>The biggest lesson wasn’t that AI produced better answers — it made it feasible to produce better evidence inside an unrealistic timeline.</p><p>By using AI to compress time-to-prototype, we gave the client what discovery often fails to deliver under pressure: a chance to evaluate real workflows (page composition, section ordering, new page/blog creation) before committing. That’s why the client could choose Strapi confidently — because they experienced the tradeoffs, instead of being sold a conclusion.</p><p>That’s the approach I’d reuse: define workflow-first criteria (including an exit strategy), parallelise prototypes, and treat AI as leverage — not judgement.</p><blockquote>Use AI to accelerate building — not deciding — so the final decision is owned by humans and backed by something real.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=122ebdb3fe67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/ai-in-technical-decision-making-a-field-guide-122ebdb3fe67">AI in Technical Decision-Making: A Field Guide</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[intelligent Engineering: Principles for Building With AI]]></title>
            <link>https://medium.com/inspiredbrilliance/intelligent-engineering-principles-for-building-with-ai-28e935c66f18?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/28e935c66f18</guid>
            <category><![CDATA[intelligent-engineering]]></category>
            <dc:creator><![CDATA[Karun Japhet]]></dc:creator>
            <pubDate>Thu, 20 Nov 2025 12:55:50 GMT</pubDate>
            <atom:updated>2026-01-06T16:57:40.653Z</atom:updated>
            <content:encoded><![CDATA[<h4>Principles required for engineering with AI for end to end software development</h4><h3>Software engineering is changing. Again.</h3><p>I’ve spent the last two years applying AI across prototyping, internal tools, production systems, and team workflows. I’ve watched it produce an elegant solution in seconds, then confidently generate code calling APIs that don’t exist. I’ve seen it save hours on boilerplate and cost hours debugging hallucinated dependencies.</p><p>One thing has become clear: AI doesn’t make engineering easier. It shifts where the hard parts are.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EllKQNevAzm4IpdpE58GHA.jpeg" /></figure><p>The teams I’ve seen succeed with AI aren’t the ones using it everywhere. They’re the ones using it deliberately, knowing when to trust it, when to verify, and when to ignore it entirely.</p><p>Here’s a working set of principles I’ve found useful. They aren’t finished and will evolve with the tools. But they help keep me grounded in what actually matters.</p><h3>intelligent Engineering Principles</h3><p>These principles fall into two buckets: what is new, and what remains timeless but more important than ever.</p><h3>AI-Native Principles</h3><p>These principles exist because of AI. They address challenges that didn’t matter before.</p><h4>AI augments, humans stay accountable.</h4><p>AI can help you move faster and see options you’d miss on your own. But it can’t own the outcome. Engineering judgment stays with you. When something breaks in production, “the AI suggested it” isn’t an acceptable answer.</p><h4>Context is everything.</h4><p>AI output reflects what you put in. Vague requests get vague results. Bring useful context: project constraints, coding standards, relevant examples, what you’ve already tried.</p><p>As systems grow, context management becomes a discipline of its own. How do new teammates get AI tools primed with the right information? How do you keep that context current? When context exceeds what fits in a prompt, you’ll need solutions like modular documentation.</p><h4>Smarter AI needs smarter guardrails.</h4><p>Faster generation demands sharper review. AI-produced code still needs validation: Is it correct? Secure? Does it solve the right problem?</p><h4>Shape AI deliberately.</h4><p>I’ve seen teams adopt whatever AI tools are trending without asking whether they fit. Six months later, half the codebase assumed Copilot’s import ordering, onboarding docs referenced prompts that no longer worked, and no one remembered why. Decide upfront: where does AI help us? Where does it not? What happens when we switch tools?</p><h4>Learning never stops.</h4><p>At the start of 2025, AI practices evolved weekly. By year’s end, monthly. That’s still faster than most teams are used to. What didn’t work three months ago might work now. The only way to know is to keep experimenting.</p><p>I’ve settled on 90% getting work done, 10% experimenting. Try new ways to solve the same problem. Revisit old problems to see if there’s a simpler solution now. Check if techniques you learned last quarter still make sense.</p><h3>Timeless Foundations</h3><p>These aren’t new, but AI makes them more important.</p><h4>Learn fast, adapt continuously.</h4><p>Start small, validate often, and shorten feedback loops. If an AI-assisted workflow isn’t helping, change it. Don’t let sunk cost keep you on a bad path.</p><h4>Fast doesn’t mean good.</h4><p>AI makes it easy to generate code fast. That doesn’t mean the code is worth keeping. Unmaintainable, insecure, or rigid solutions cost more than they save. Build the right thing, not just the quick thing.</p><h3>What This Looks Like in Practice</h3><p>Here’s what this means day-to-day:</p><ul><li>I use AI to draft implementations, then spend more time reviewing than I saved generating. The review is where the real work happens.</li><li>When AI suggests an approach, I ask “why?” If I can’t explain the choice to a teammate, I don’t use it.</li><li>I’ve learned to be specific. “Write a function to parse dates” gets garbage. “Write unit tests for an ISO 8601 date parser, including edge cases for timezone offsets, leap seconds, and malformed input — then implement a function that passes them” gets something I can actually trust.</li><li>I treat AI output like code from a confident junior developer: often correct, sometimes subtly wrong, occasionally completely off base.</li></ul><p>The craft hasn’t changed. I still need to understand the problem, reason about edge cases, and take responsibility for what ships.</p><h3>Skills Worth Building</h3><p>Principles guide decisions. Skills make them possible.</p><p>Here’s what I’ve found worth investing in:</p><p><strong>Context engineering matters more than prompt engineering.</strong> A clever prompt won’t fix bad context. I spend more time curating what information the model sees than crafting how I ask for things. Project documentation, coding standards, relevant examples. These matter more than prompt tricks.</p><p><strong>Understanding tokens and context windows helps.</strong> You don’t need to become an ML engineer. But it helps to know why your 50-file codebase overwhelms the model, or why it “forgets” earlier instructions.</p><p><strong>Agentic workflow primitives matter more than AI theory.</strong> You won’t build RAG systems from scratch. You’ll use tools with these built in. What matters is configuring them: hooks that customize behavior, skills that extend capabilities, context management that keeps information relevant. I spend more time learning how my tools’ hooks work or how to structure context files than reading ML papers.</p><h3>Why This Matters</h3><p>I’ve seen what happens when teams adopt AI without thinking it through. Prototypes that demo well but collapse under real load. Codebases where no one understands why decisions were made because “the AI suggested it.” Bugs that take days to track down because the generated code looked plausible but handled edge cases incorrectly.</p><p>The failure mode isn’t dramatic. It’s slow erosion: teams that gradually stop reasoning deeply because the model provides answers quickly.</p><p>The alternative isn’t avoiding AI. It’s using it with intention. The engineers I’ve seen do this well have gotten faster <em>and</em> more thoughtful. They use AI to handle the routine and focus on the hard problems.</p><h3>What’s Next</h3><p>These principles aren’t final. I expect to revise them as tools improve and as I learn what actually works versus what sounds good in theory.</p><p>If you’re experimenting with AI in your engineering work, I’d be curious to hear what’s working for you. What would you add? What would you challenge?</p><h3>Credits</h3><p><em>This blog would not have been possible without the review and feedback from</em> <a href="https://www.linkedin.com/in/greg-reiser-6910462/"><em>Greg Reiser</em></a><em>,</em> <a href="https://www.linkedin.com/in/gsong/"><em>George Song</em></a> <em>and</em> <a href="https://www.linkedin.com/in/karthika-vijayan/"><em>Karthika Vijayan</em></a> <em>for reviewing multiple versions of this post and providing patient feedback 😀.</em></p><p><em>This content has been written on the shoulders of giants (at and outside</em> <a href="https://sahaj.ai/"><em>Sahaj</em></a><em>).</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28e935c66f18" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/intelligent-engineering-principles-for-building-with-ai-28e935c66f18">intelligent Engineering: Principles for Building With AI</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When Training Takes Too Long: Distributed Data Parallelism]]></title>
            <link>https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?source=rss----5cdcaabf7d50---4</link>
            <guid isPermaLink="false">https://medium.com/p/f8ee13381799</guid>
            <category><![CDATA[system-scaling]]></category>
            <category><![CDATA[llm-training]]></category>
            <category><![CDATA[data-parallelism]]></category>
            <category><![CDATA[distributed-training]]></category>
            <category><![CDATA[ddp]]></category>
            <dc:creator><![CDATA[3pi]]></dc:creator>
            <pubDate>Mon, 03 Nov 2025 15:27:21 GMT</pubDate>
            <atom:updated>2026-04-20T08:46:17.968Z</atom:updated>
            <content:encoded><![CDATA[<h4>“When training speed limits progress, distribute the data”</h4><h3>Table of Contents</h3><ul><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#4422">Architectural vs. System-Level Scaling</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#e7ba">From Sequential to Parallel Training</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#effc">The Core Idea of Distributed Data Parallelism (DDP)</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#f4ca">Synchronizing Gradients with All-Reduce</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#b9f7">How DDP Overlaps Communication and Computation</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#f639">Why DDP became the standard</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#7dd0">The Limitations of DDP</a></li><li><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799?postPublishedType=repub#6f46">What if the model itself is too large to fit on one GPU?</a></li></ul><h3>Architectural vs. System-Level Scaling</h3><p>Architectural tricks such as LoRA and QLoRA reduce the cost of training, but they do not make training fast enough once data becomes massive. At that point, the problem shifts from model efficiency to system-level scaling: how do we distribute work across GPUs so training finishes in practical time? In this article, we focus on the most important answer to that question: Distributed Data Parallelism <strong>(</strong><a href="https://docs.pytorch.org/tutorials/intermediate/ddp_tutorial.html"><strong>DDP</strong></a>).</p><ol><li><strong>Vertical Scaling:</strong> Designing smarter architectures and fine-tuning strategies that reduce computation without sacrificing performance — topics explored in my earlier blogs (<a href="https://medium.com/inspiredbrilliance/exploring-lora-part-1-the-idea-behind-parameter-efficient-fine-tuning-and-lora-ec469d176c26">LoRA</a>, <a href="https://medium.com/inspiredbrilliance/why-qlora-isnt-enough-for-true-model-scaling-d0d9c013df01">QLoRA</a> etc,).</li><li><strong>Horizontal Scaling:</strong> Distributing the workload across multiple GPUs or nodes. This distribution can parallelize both <strong>data </strong>and <strong>model</strong> computations.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/648/1*zvUbTwW9uS1IL1DfSCxMIQ.png" /><figcaption><strong>Left:</strong> Techniques that improve computational efficiency within the model’s architecture.<br> <strong>Right:</strong> Scaling models and data across multiple nodes for distributed training and inference.</figcaption></figure><h3>From Sequential to Parallel Training</h3><p>A neural network can be viewed as a function of its <strong>weights</strong>, say <strong><em>f(w₁, w₂, w₃, …)</em></strong>. During training, the dataset — for example, 100 samples — is not processed all at once, but in smaller batches. For each batch, the model makes predictions, computes a scalar <strong>loss</strong> that measures how far those predictions are from the target, and then calculates <strong>gradients</strong> <strong><em>(g₁, g₂, g₃, …)</em></strong> for each weight. The optimizer uses these gradients to update the weights, and by repeating this process batch after batch, the model gradually adjusts itself to minimize the overall loss.</p><blockquote>For a deeper exploration of objective landscapes, model dimensions, loss surfaces, and solution spaces, check out my earlier blog on <a href="https://medium.com/inspiredbrilliance/intrinsic-dimension-part-1-why-large-models-can-be-fine-tuned-with-fewer-parameters-d15702d2943e"><strong>Intrinsic Dimension</strong></a>.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/385/1*UFjXCpiz9xKpWjV78l-ISw.png" /><figcaption>The model, represented as a function of learnable weights, processes all the data samples sequentially to compute a loss. Gradients are then used by the optimizer to update the weights accordingly.</figcaption></figure><p>If you want to speed up training, the next intuitive step is to use multiple GPUs. You can create a copy of the model <em>f(w₁, w₂, w₃, …)</em> on each GPU and split the dataset into two parts — for instance, one GPU handles the <em>odd-numbered</em> samples D(<em>1,3,5,7,…)</em> and the other processes the <em>even-numbered</em> ones D<em>(2,4,6,8,…)</em>. Each GPU independently performs forward and backward passes on its data shard, producing distinct losses <em>(Loss₁, Loss₂)</em> and gradients.</p><p>While this parallelizes the workload, the two models drift apart without synchronization — each learning its own version of the weights. To maintain coherence, gradients must be synchronized after every step so all GPUs learn as one unified model. This concept forms the foundation of data-distributed training.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*0Wu9dNXlptQH5fQDxzmd_Q.png" /><figcaption>Loss₁ and Loss₂​, along with their corresponding gradients (shown in maroon and yellow) on GPUs 1 and 2, differ — causing each optimizer to update its model weights independently.</figcaption></figure><h3>The Core Idea of Distributed Data Parallelism (DDP)</h3><p>Data Parallelism accelerates training by distributing data across GPUs while keeping identical model replicas.</p><p>The idea is the following:</p><ul><li>You split the dataset into multiple shards.</li><li>Replicate the same model across multiple GPUs.</li><li>Each GPU processes a different shard of data <em>in parallel</em> during a training step.</li><li>During the forward pass, each GPU computes a loss using its local data.</li><li>During the backward pass, it calculates gradients for its weights based on that loss for that particular data shard.</li></ul><p>To keep models consistent, these gradients must be synchronized across GPUs — a process which:</p><ol><li><strong>Aggregate gradients:</strong> Gradients from all GPUs are summed (or averaged) into one unified gradient — typically done on a primary GPU.</li><li><strong>Broadcast results:</strong> The averaged gradients are then shared back with all GPUs so that every replica updates weights identically.</li></ol><p>This process keeps all model copies in sync, enabling them to learn as if they were one unified model — just much faster.</p><p><strong>Note:</strong> Communication efficiency depends on the interconnect. High-speed links like <strong>NVLink</strong> or <strong>RoCE</strong> (RDMA over Converged Ethernet) reduce latency and improve synchronization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*Ydxe4sOTHczbe9r6w2vhcA.png" /><figcaption>Gradients computed on different GPUs (yellow and maroon) must be synchronized and unified (green), then shared back to all GPUs via a network interface (in orange).</figcaption></figure><h3>Synchronizing Gradients with All-Reduce</h3><p>To visualize how synchronization works, imagine three data chunks — <strong>D₁</strong>, <strong>D₂</strong>, and <strong>D₃</strong> — being processed simultaneously by <strong>GPU 1</strong>, <strong>GPU 2</strong>, and <strong>GPU 3</strong>.<br>Each GPU holds an identical copy of the model layers <em>(</em><strong><em>L₁</em></strong><em>, </em><strong><em>L₂</em></strong><em>, </em><strong><em>L₃</em></strong><em>)</em> but processes a different shard of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*BwgAkjaw2M6sMc4M9Acy5A.png" /><figcaption>Block Diagram of DDP Framework</figcaption></figure><p>During the forward pass, each computes its own loss <em>(</em><strong><em>Loss₁</em></strong><em>, </em><strong><em>Loss₂</em></strong><em>, </em><strong><em>Loss₃</em></strong><em>)</em>; during the backward pass, it calculates gradients specific to its data shard <em>(</em><strong><em>g_D₁</em></strong><em>, </em><strong><em>g_D₂</em></strong><em>, </em><strong><em>g_D₃</em></strong><em>)</em>. <br>Before updates, gradients are synchronized via <strong>All-Reduce</strong>:</p><ul><li>Gradients are aggregated (e.g., averaged by dividing by 3 for three GPUs).</li><li>The result is broadcast back to all GPUs.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/485/1*csElnyJP3bM3sQjF3iRNOQ.png" /><figcaption>Each GPU processes a different data shard and computes its gradients (g_D₁, g_D₂, g_D₃). Before updates, gradients are averaged (e.g., divided by 3 for three GPUs) and broadcast back so all models stay identical.</figcaption></figure><p>Once synchronized, all GPUs update their weights using the same unified gradients (<strong>g</strong>), ensuring that every model replica stays in perfect sync for the next iteration.</p><p>Under the hood, <strong>NCCL (NVIDIA Collective Communications Library)</strong> performs these operations efficiently on GPUs, while <strong>Gloo</strong> handles equivalent communication across CPU-based systems.</p><h3>How DDP Overlaps Communication and Computation</h3><p>DDP’s real strength lies in overlapping computation and communication to maximize efficiency.</p><p><strong>Step-by-step flow:</strong></p><ol><li><strong>Forward Pass (Layers L₁ → L₂ → L₃):</strong><br>The input data flows through all layers of the model. Each GPU computes its own forward pass and obtains a loss value — <strong>Loss₁</strong>, <strong>Loss₂</strong>, and <strong>Loss₃</strong>, respectively, for each data shard.</li><li><strong>Backward Pass — Layer 3:</strong><br>Backpropagation begins from the final layer and proceeds backward. Gradients (<strong>∂L/∂w</strong>) for Layer 3 are computed first, since gradients for earlier layers depend on those from later ones through the <strong>chain rule</strong>. In other words, Layer 3 must complete its gradient computation before Layer 2 can begin.</li><li><strong>Backward Pass — Layer 2 (with Overlapping Communication):</strong><br>While the gradients from <strong>Layer 3</strong> are being synchronized across GPUs, the backward computation for <strong>Layer 2</strong> starts simultaneously.<br>This overlap between communication (gradient sync) and computation (gradient calculation) is what makes DDP far more efficient than a naïve distributed sampler, where these steps would happen sequentially.</li><li><strong>Backward Pass — Layer 1:</strong><br>Similarly, while gradients from Layer 2 are being synchronized, the computation for Layer 1 begins. This pipelined execution continues until all layers have finished computing and synchronizing their gradients.</li></ol><p>By the end of this process, every GPU has synchronized gradients across all layers — ready to update weights in perfect unison.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*KaOvwT3VgqiT2fNGKE1YVA.png" /><figcaption>Step-by-step flow of DDP’s forward and backward passes. Gradients are computed layer by layer (L₃ → L₂ → L₁), with communication and computation overlapped for maximum efficiency.</figcaption></figure><h3>Why DDP became the standard</h3><ul><li><strong>Speedup:</strong> Training can scale almost linearly with the number of GPUs — in ideal conditions, 4 GPUs ≈ 4× faster training.</li><li><strong>Ease of use:</strong> PyTorch’s DistributedDataParallel (DDP) framework makes distributed training highly accessible with minimal code changes.</li><li><strong>Efficiency: </strong>DDP overlaps gradient computation and communication (via All-Reduce) across layers, significantly reducing idle GPU time and improving overall throughput.</li><li><strong>Scalability (to a point):</strong> Works well for medium-to-large models that fit on individual GPUs but benefit from faster data throughput.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/923/1*vBUX82yzEEVx24pUe1zpuA.png" /><figcaption>Timeline of DDP’s overlapping operations: At <em>t₁</em>, only the backward pass for Layer 3 runs. At <em>t₂</em>, Layer 2’s backward computation overlaps with Layer 3’s All-Reduce — independent tasks that make DDP efficient</figcaption></figure><h3>The Limitations of DDP</h3><p>While DDP accelerates training, it doesn’t solve every problem —</p><ul><li><strong>Communication Overhead:</strong> All-Reduce becomes expensive as GPU count increases.</li><li><strong>Synchronization Delays:</strong> The entire system waits for the slowest GPU to finish its batch.</li><li><strong>Memory Duplication:</strong> Each GPU holds a complete copy of the model.</li></ul><p>So while DDP reduces training time, it doesn’t reduce memory usage — and for truly massive models, that’s the real bottleneck.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*ciEQypS1H6a6abZhmoDwyg.jpeg" /><figcaption>When the model (the elephant) outgrows the GPU’s memory (the room), it simply doesn’t fit — highlighting DDP’s key limitation: every GPU must store a complete copy of the model.</figcaption></figure><h3><em>What if the model itself is too large to fit on one GPU?</em></h3><p>When a model grows too large to fit on even one GPU, techniques like Model Parallelism, Pipeline Parallelism, Hybrid Parallelism, and Fully Sharded Data Parallel (FSDP) come into play.<br>Unlike DDP, which duplicates the full model across devices, these methods split the model itself — across layers, tensors, or even optimizer states — enabling true large-scale distributed training.</p><p>We’ll explore these techniques in detail in the forthcoming articles.</p><p>All images are created by the author using <strong>draw.io</strong> and <strong>Excalidraw</strong>, unless specified otherwise.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8ee13381799" width="1" height="1" alt=""><hr><p><a href="https://medium.com/inspiredbrilliance/when-there-is-not-enough-time-distributed-data-parallelism-f8ee13381799">When Training Takes Too Long: Distributed Data Parallelism</a> was originally published in <a href="https://medium.com/inspiredbrilliance">inspiringbrilliance</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>