<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ramakrishna Sanikommu on Medium]]></title>
        <description><![CDATA[Stories by Ramakrishna Sanikommu on Medium]]></description>
        <link>https://medium.com/@ramakrishna.sanikommu?source=rss-c317b65ef00a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*b6c9bAB1CJ8IfymX</url>
            <title>Stories by Ramakrishna Sanikommu on Medium</title>
            <link>https://medium.com/@ramakrishna.sanikommu?source=rss-c317b65ef00a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 05:17:29 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ramakrishna.sanikommu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Agentify Your Enterprise: Two New Layers, Not a New Enterprise]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/agentify-your-enterprise-two-new-layers-not-a-new-enterprise-1b21899f120e?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b21899f120e</guid>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[antrhopic]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 03:30:32 GMT</pubDate>
            <atom:updated>2026-09-25T17:07:46.329Z</atom:updated>
            <content:encoded><![CDATA[<h3>Your ERP, CRM, data platform, Analytics and dashboards already work. Here’s how to put agents on top of them without rebuilding what isn’t broken.</h3><p>Most large organisations look like this. Systems of record such as ERP, CRM, HCM and SCM hold the business. A data platform moves that data into a lakehouse and warehouse. Business logic and ML models hold the enterprise’s judgment. An integration layer connects it all, and portals and dashboards put it in front of people. Data governance runs through every layer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WA2gPSvRgH31Hj0Wo7YbGA.png" /></figure><h3>What most people think “agentify” means</h3><p>For most leadership teams, “agentic enterprise” means starting again: a new AI platform, a rebuilt data foundation, legacy systems retired, eighteen months, eight figures.</p><p>But look at the diagram above and ask which layer you would rebuild. None of them is the problem. The problem is the human glue between them, and it isn’t inside any system.</p><p>Two related assumptions make it worse. The first is that you need an agent for every system: an SAP agent, a Salesforce agent, a Workday agent. The second is that you can’t start until your data is unified. Both lead to large, slow programmes, and the results show it. Only 2% of organisations have deployed agents at full scale, and Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027.</p><h3>What agentifying actually means: two new layers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P2hLyt5f-XapMBjDH8mbtQ.png" /></figure><p>Six layers are identical and marked unchanged. No migration, no replatform. The dashboard doesn’t go away; a person can still open it, and now an agent can read it too.</p><p>Inside the dashed boundary are the only two layers you build.</p><p><strong>The capability and context layer</strong> makes the existing estate machine-readable. It exposes what your systems already do, such as looking up stock, creating a requisition or fetching a policy, as tools an agent can discover and call, with the meaning of the data attached. This is infrastructure: built once, shared by every agent, owned by a platform team, and slow to change.</p><p><strong>The agent layer</strong> is where the work between systems happens. An orchestrator plans each task, routes it to the right agents and combines the result. Beneath it, domain agents each own a business outcome: supply shortages, financial close, onboarding, service triage, cross-system analysis. This is application logic: built per use case, owned by domain teams, and cheap to rewrite. Your first supply agent will be wrong, and you’ll rewrite it several times. That’s fine, as long as rewriting it never means touching the layer beneath.</p><p>Between them sits the <strong>human in the loop</strong>. The planner didn’t disappear. She moved from the middle of the stack to the approval step. She decides instead of integrating.</p><p>Down the right side are two rails outside the request path. The <strong>agent workbench</strong> is new tooling for designing, testing, versioning and monitoring agents. <strong>Governance and trust</strong> is not new. It is your existing IAM, RBAC, catalog, lineage and audit, extended to cover non-human actors. If your agentic budget has a line item called “build governance,” something has gone wrong.</p><h3>You don’t need an agent for every system</h3><p>This is where most programmes go wrong, and where the two-layer design earns its keep.</p><p>Imagine ten systems and eight business processes you want to automate. If every agent integrates directly with every system it needs, you’re heading toward eighty point-to-point integrations, each one breaking when a schema changes. This is the classic N×M problem, and MCP was built specifically to solve it, by giving models a standard way to discover and call capabilities across many data sources.</p><p>With a capability layer in between, the maths changes. Each system is exposed once, as a set of tools. Each agent picks the tools it needs. Ten systems plus eight agents means eighteen things to maintain instead of eighty.</p><h3>Five ways to agentify a system</h3><p>Not every system should be treated the same way. Three options get most of the attention. Two more matter just as much in real programmes.</p><p><strong>1. Wrap it.</strong> Leave the application exactly as it is and expose its existing APIs through MCP servers. The agent works through the system, not inside it. This should cover most of your estate.</p><p><strong>2. Redesign it.</strong> Keep the system of record and its data, but redesign the workflow so the agent becomes the primary operator and the human becomes the approver. The application stays; its interaction model changes.</p><p><strong>3. Rebuild it, agent-native.</strong> Build new, from scratch, with agents at the core. Use this only where the old application’s basic assumption, that a human reads a screen and clicks, is itself the bottleneck, and there’s no deep system of record to preserve.</p><p><strong>4. Adopt the vendor’s agents.</strong> For work that lives entirely inside one vendor’s platform, adopting the vendor’s own agent is often faster and cheaper than building your own. What matters is connecting it to your wider agent layer through A2A.</p><p><strong>5. Leave it alone.</strong> A fast, deterministic, well-behaved service gains nothing from an agent in front of it. If a process always gives the same output for the same input, that’s a tool, not an agent. Being willing to say “this one doesn’t need it” is what makes the rest of your recommendations credible.</p><p>In practice, a typical enterprise ends up with most systems wrapped, a handful redesigned, one or two rebuilt, several vendor agents adopted, and a lot left alone.</p><h3>How to build it: the technology</h3><p>Here’s how each part of the agentified architecture maps to real technology. The architecture holds whichever cloud you’re on.</p><h4>The capability and context layer</h4><p><strong>MCP servers.</strong> The Model Context Protocol is now the de facto open standard for connecting agents to tools. An MCP server sits in front of an existing interface, such as a REST API, SAP OData, a SOAP service or a SQL query, and exposes it as tools.</p><p>The unit of work is the tool definition: a name, a description and an input schema. The agent decides what to call by reading the description, so the description is effectively production code.</p><pre>{<br>  &quot;name&quot;: &quot;get_onhand_stock&quot;,<br>  &quot;description&quot;: &quot;Returns on-hand and committed stock for ONE SKU at ONE plant. Does not include in-transit stock; use get_intransit_stock for that. Read-only.&quot;,<br>  &quot;inputSchema&quot;: {<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;properties&quot;: {<br>      &quot;sku&quot;:   { &quot;type&quot;: &quot;string&quot;, &quot;description&quot;: &quot;Material number, e.g. MAT-100245&quot; },<br>      &quot;plant&quot;: { &quot;type&quot;: &quot;string&quot;, &quot;description&quot;: &quot;Four-character plant code, e.g. US01&quot; }<br>    },<br>    &quot;required&quot;: [&quot;sku&quot;, &quot;plant&quot;]<br>  }<br>}</pre><p>A tool called get_inv described as &quot;gets inventory&quot; will be called at the wrong moments with the wrong arguments. Most enterprises have never reviewed prose as a deployable artifact. They need to start.</p><p>Five engineering rules separate a real capability layer from a thin wrapper:</p><p><strong>Idempotent writes.</strong> Agents retry. The write tool accepts a caller-generated key and returns the original result on a repeat call, so the agent can never create two requisitions.</p><p><strong>Traffic shaping.</strong> Your APIs were sized for a few hundred people clicking, not an agent making hundreds of calls a second. Put a cache, a queue and a circuit breaker in front of fragile backends.</p><p><strong>Structured errors.</strong> Return {&quot;error&quot;: &quot;sku_not_found&quot;, &quot;retryable&quot;: false}, not an HTML error page, so the agent knows whether to retry, change approach or stop.</p><p><strong>Shaped responses.</strong> Don’t return 40,000 rows. Return a summary and a way to drill in.</p><p><strong>Separate read and write servers.</strong> Reads are broad and low-risk. Writes are few, narrow and gated. This single decision removes most of your risk.</p><p><strong>Tool registry.</strong> Beyond fifty tools, discovery becomes the problem: what exists, who owns it, which agent may use it. A tool registry or gateway answers that, and each major cloud agent platform now includes one.</p><p><strong>Semantic and RAG context.</strong> The semantic model you already built for BI is exactly what an agent needs to understand that STAT_CD_3 means order status. Your existing vector store supplies unstructured context such as policies, contracts and procedures.</p><p><strong>Event triggers.</strong> Agents shouldn’t only wait to be asked. Your existing event bus, whether Kafka, Pub/Sub, EventBridge or CDC streams, can wake an agent when a shipment is delayed or an invoice fails matching.</p><p><strong>Scoped credentials.</strong> The agent carries the requesting user’s identity through to the backend, typically via OAuth token exchange. Agents should act with the permissions of whoever triggered them, never with an admin account.</p><h4>The agent layer: frameworks</h4><p>This is where LangChain, ADK and the rest come in. They are the code you write agents in.</p><p><strong>LangGraph</strong> (from LangChain) suits explicit, stateful workflows with loops and approvals, and is the most widely deployed in production. Since its 1.0 release it has focused on durable execution that resumes after failure, human-in-the-loop interrupts, and memory.</p><p><strong>Google ADK</strong> suits Google Cloud and Gemini shops and has strong A2A support. It reached stable v1.0 across Python, Go, Java and TypeScript, and its graph-based engine lets you slide from open-ended reasoning to strictly deterministic workflows.</p><p><strong>Microsoft Agent Framework</strong> suits .NET and Azure shops. It went GA in April 2026 and supports non-Microsoft model providers.</p><p><strong>OpenAI Agents SDK and Claude Agent SDK</strong> suit simpler, model-driven agents from those providers.</p><p><strong>CrewAI</strong> suits role-based teams of agents, and <strong>Strands Agents</strong> suits AWS shops.</p><p>How to choose: first match your cloud and your team’s language. Then decide how much control you want. Graph-based frameworks such as LangGraph give precise control, while model-driven loops trade some control for simplicity. For work that touches systems of record, lean toward explicit graphs. You want to be able to point at the exact step where an approval happens.</p><p>The dominant pattern is <strong>supervisor and specialists</strong>: an orchestrator receives the request, plans it, and delegates to domain agents, each with a narrow set of tools.</p><h4>Agent to agent: A2A</h4><p>MCP connects an agent to a tool. A2A connects an agent to another agent. Confusing the two is one of the most common architecture mistakes.</p><p>A2A matters most when you adopt vendor agents. When your CRM vendor’s agent, your ITSM vendor’s agent and your own supply agent need to cooperate, A2A is how they discover each other and hand off work without you rebuilding any of them.</p><p>One caution: don’t reach for A2A when one agent with tools will do. Most use cases start as a single agent calling MCP tools, and should stay that way until they genuinely need to split.</p><h4>Human in the loop</h4><p>Frameworks provide the mechanics. LangGraph and ADK can both pause an agent mid-workflow to wait for a person. The design question is which actions need it.</p><p>The simplest rule is reversibility. If an action is easy to undo, such as summarising an incident or drafting a reconciliation, let the agent act and log it. If it can be undone with some effort, such as rescheduling a production run, let it act and notify someone. If it can’t be undone, such as releasing a payment, the agent proposes and a person approves.</p><p>Route those approvals into your existing approval workflows. Don’t build a new inbox.</p><h4>Where agents run</h4><p>Each major cloud now offers a managed runtime with scaling, identity, memory and monitoring.</p><p><strong>Google:</strong> Vertex AI was rebranded as the Gemini Enterprise Agent Platform, bringing together ADK, the low-code Agent Studio and the managed Agent Engine runtime.</p><p><strong>AWS:</strong> Amazon Bedrock AgentCore runs agents built with Strands, LangGraph, CrewAI or custom code, with services for runtime, identity, memory, tool gateways, observability and evaluation.</p><p><strong>Microsoft:</strong> Azure AI Foundry is now Microsoft Foundry, and its Agent Service is the managed runtime.</p><p>These runtimes are largely framework-agnostic. Choosing a runtime doesn’t choose your framework; treat them as two separate decisions.</p><h4>The agent workbench</h4><p>This is the rail most teams skip, then regret around agent number four, when nobody can prove a prompt change helped or explain a spend spike.</p><p><strong>Tracing</strong> records every model call, tool call and handoff, end to end. Prefer frameworks that emit OpenTelemetry-compatible traces so you aren’t locked into one vendor’s view. LangSmith, Langfuse and Arize Phoenix are common choices.</p><p><strong>Evaluation</strong> is a fixed set of real test cases that every change must pass before it ships.</p><p><strong>Versioning</strong> keeps prompts and tool descriptions under version control and deploys them like code.</p><p><strong>Cost tracking</strong> shows spend per agent, per run and per business outcome.</p><h4>Governance and trust</h4><p>Extend your existing governance; don’t rebuild it. One practice matters more than most: generate a run ID at the start of every agent execution, carry it through every tool call, and write it into the database row the agent changes. When a request passes through MCP, a queue and another agent, traces fragment. The teams that handle this well keep the system of record in the database, not in protocol message history. When an auditor asks which agent changed which record and when, the answer should be a SQL query.</p><h3>Why this costs less, and the better reason to do it</h3><p>The cost argument is straightforward. You build two layers instead of rebuilding six. Years of investment in systems of record, data platforms, business logic and ML models keep paying off instead of being written off. And because the capability layer is shared, each new agent is cheaper than the last.</p><p>But the stronger reason is reversibility. When you add layers rather than replace them, the old system keeps running. If an agent gets something wrong, you switch it off and the business carries on exactly as it did last month. When a better model arrives, you swap it without touching the estate. If a regulator asks a question you can’t yet answer, the human process is still there.</p><p><strong>The measure of a good agentic programme isn’t what it automates. It’s what it can undo.</strong></p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b21899f120e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI on Google Cloud, With Nothing But Google Cloud]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/agentic-ai-on-google-cloud-with-nothing-but-google-cloud-cac3a86c1120?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/cac3a86c1120</guid>
            <category><![CDATA[gemini-enterprise]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[ai-governance]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 13:57:30 GMT</pubDate>
            <atom:updated>2026-09-25T17:08:17.200Z</atom:updated>
            <content:encoded><![CDATA[<h3>The agent was never the hard part. A complete native reference architecture — ADK, Agent Runtime, Knowledge Catalog, Model Armor — including the two places it still falls short.</h3><p>Nothing about the agent was wrong. What was missing was everything around it: a way to prove which of forty candidate tables it should have trusted, a way to stop it answering a question the asker had no rights to ask, a way to attribute $40,000 of monthly token spend to a workflow, and a way to answer an auditor asking what the system did last Tuesday at 14:05. That gap — between an agent that works and an agent that ships — is the subject of this post.</p><p>I’ve written before about <a href="https://medium.com/@ramakrishna.sanikommu/agentic-ai-orchestration-fb4b8e8be8a7">agentic orchestration</a>, <a href="https://medium.com/@ramakrishna.sanikommu/context-engineering-rag-2-0-the-next-chapter-in-genai-4e53c0382bf4">context engineering and RAG 2.0</a>, <a href="https://medium.com/@ramakrishna.sanikommu/one-llm-was-never-the-endgame-the-inevitable-rise-of-ai-gateway-architecture-c6393ce4478f">AI gateway architecture</a> and <a href="https://medium.com/@ramakrishna.sanikommu/agentic-mesh-the-new-kid-on-the-ai-block-innovation-or-just-more-buzzwords-fd4efeb4f76b">agentic mesh</a> as patterns. This is the implementation. One cloud, one opinionated stack, every box a service you can turn on today.</p><h3>The architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zv6uzXUrxxnUd81JUn6C1g.png" /></figure><p><strong>Six planes and two cross-cutting rails.</strong> The rails matter more than they look: security and observability aren’t a layer, they’re a property of every hop. Any diagram that puts governance in a box at the bottom describes a system that will fail its first audit.</p><p>The diagram reads top-down, the way a request travels. The sections below walk bottom-up, the way you build.</p><h3>1. The data foundation you already have</h3><p>The most common mistake in agentic programmes is treating them as greenfield. You have a lakehouse. Use it.</p><p>What you already run What it becomes BigQuery marts and governed views The analytics agent’s only legal query surface Cloud Storage document lake The retrieval corpus, chunked and embedded Dataflow / Dataproc Also the incremental embedding pipeline Datastream CDC Freshness triggers for index invalidation Pub/Sub Event backbone for re-index and agent triggers Cloud Composer Orchestration for evaluation and re-index runs</p><p><strong>Agents read governed views, never base tables.</strong> A view is a contract — where you pin join logic, row filters and column masking. If your agent can SELECT * from a raw table, you have no story for the auditor and no way to change the schema without breaking it.</p><p><strong>Embedding is a pipeline, not a script.</strong> Chunking, embedding and index refresh belong in Dataflow, on a schedule and on CDC triggers, with the same monitoring and lineage as everything else. The number of production RAG systems quietly serving three-week-old vectors is higher than anyone admits. Publish index lag as a metric and let the agent state its own currency — “as of 14:05” — rather than answering confidently from yesterday.</p><h3>2. Context, semantics and the knowledge graph</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u8OOQKoDhudpZe2lamqLQQ.png" /></figure><p>This is where I’d spend a disproportionate share of the budget, and it’s the section most architectures skip.</p><p><strong>Knowledge Catalog</strong> is a genuinely different product from the catalogue you may remember. It automatically extracts semantics from structured <em>and</em> unstructured data and builds a dynamic context graph explicitly intended to ground agents and reduce hallucination — catalogue as agent substrate rather than compliance artefact. It holds auto-extracted semantics, the business glossary and data products, aspects (sensitivity, ownership, SLA, freshness), lineage, and the context graph joining them.</p><h3>Semantic layer vs. context layer</h3><p>These get used interchangeably. They answer different questions.</p><p>The <strong>semantic layer</strong> is <em>definitional</em>, resolved once for everyone: what does this metric mean, at what grain, which joins are legal, what’s the agreed name. Stable, human-curated, changes on a release cycle. This is the dbt/LookML/governed-view work data teams have done for twenty years. Necessary, not sufficient.</p><p>The <strong>context layer</strong> is <em>situational</em>, resolved per request: of the 400 assets matching this query, which may <em>this agent</em> read, for <em>this user</em>, at <em>this freshness</em>, with what lineage and sensitivity? Volatile, machine-assembled, alive for one turn.</p><p>A semantic layer without a context layer produces an agent that knows what “revenue” means and cheerfully reports it to someone who shouldn’t see it, from a table that broke last night. Knowledge Catalog stores both, which is why it sits <em>under</em> the agent rather than beside it. The pattern: on every turn, the orchestrator queries the catalogue for candidate assets filtered by principal, sensitivity and freshness — <em>before</em> routing to any retrieval surface. Single-digit milliseconds, and it eliminates a whole class of failure.</p><h3>When the answer is a path, not a passage</h3><p>Vector retrieval is good at “find me text like this” and bad at “which suppliers, two hops out, have an open quality incident?” <strong>Spanner Graph</strong> is the native answer — property graph with GQL on Spanner’s consistency guarantees.</p><pre>GRAPH EnterpriseGraph<br>MATCH (c:Component {id: @component_id})<br>      -[:SUPPLIED_BY]-&gt;(s:Supplier)-[:HAS_INCIDENT]-&gt;(i:Incident)<br>WHERE i.status = &#39;OPEN&#39;<br>  AND i.opened_at &gt; TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)<br>RETURN s.name, i.severity, i.summary<br>ORDER BY i.severity DESC</pre><p><strong>Entity resolution is the actual project</strong> — the graph is only as useful as its ability to say “these seven records are one customer,” and that’s a data engineering effort, not a database one. Give the graph its own sub-agent; relationship questions have a different latency profile and failure mode from passage retrieval.</p><h3>3. Retrieval</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MtOkrdbyxsA0Ak6SUpZJXw.png" /></figure><p>Five native surfaces. The mistake is picking one as doctrine.</p><p>Surface Use when Trade-off <strong>RAG Engine</strong> You want managed RAG over your corpora — ETL, OCR, chunking and embedding handled Least control over the pipeline <strong>Agent Search</strong> Google-quality retrieval and grounding; can also serve as RAG Engine’s backend Least control over ranking internals <strong>Vector Search</strong> High QPS, low latency, large index You own embedding, chunking, refresh <strong>BigQuery vector index</strong> Recall must join to governed tables Higher latency than dedicated ANN <strong>Spanner Graph</strong> The answer is a relationship path Not a text retrieval engine</p><p>Most systems end up with two or three, routed per query — and that routing is a catalogue lookup, not a hardcoded if.</p><p>The pipeline that holds up: <strong>query understanding</strong> (rewrite, decompose, classify — Gemini Flash, the highest-leverage cheap call in the system) → <strong>source routing</strong> → <strong>parallel retrieval</strong> (never serialise this) → <strong>fusion and reranking</strong> (RRF across surfaces, then a cross-encoder; a hundred candidates in, eight out — consistently the highest-ROI step teams skip) → <strong>context assembly</strong> → <strong>generation with grounding metadata</strong>.</p><p>A BigQuery leg, for when recall must respect row-level policy:</p><pre>WITH q AS (<br>  SELECT ml_generate_embedding_result AS embedding<br>  FROM ML.GENERATE_EMBEDDING(<br>    MODEL `proj.models.text_embedding`,<br>    (SELECT @user_query AS content),<br>    STRUCT(TRUE AS flatten_json_output, &#39;RETRIEVAL_QUERY&#39; AS task_type))<br>)<br>SELECT base.doc_id, base.chunk_text, base.source_uri, distance<br>FROM VECTOR_SEARCH(<br>  TABLE `proj.kb.chunk_embeddings`, &#39;embedding&#39;, TABLE q, &#39;embedding&#39;,<br>  top_k =&gt; 40, distance_type =&gt; &#39;COSINE&#39;,<br>  options =&gt; &#39;{&quot;fraction_lists_to_search&quot;: 0.02}&#39;)<br>WHERE base.sensitivity_tag IN UNNEST(@permitted_tags)<br>ORDER BY distance</pre><p>Note the WHERE. <strong>Access control belongs inside retrieval, not after generation.</strong> Filter the answer after the model has seen the data and you&#39;ve already leaked — its reasoning was shaped by content the user had no right to.</p><h3>4. Models</h3><p><strong>Model Garden</strong> serves 200+ models through one API — Gemini, Gemma, Claude, Llama. Anthropic’s models are first-class alongside Gemini, which materially changes the lock-in conversation.</p><p>Tier deliberately: <strong>Flash-Lite</strong> for query rewriting, routing and classification (called constantly, latency dominates); <strong>Flash</strong> for tool selection and most sub-agents; <strong>Pro</strong> for planning and final synthesis; <strong>Gemma on GKE</strong> for regulated or air-gapped serving; a second vendor for eval-judging, where independent failure modes matter.</p><p>Three cost levers teams leave unused: <strong>provisioned throughput</strong> once traffic is predictable, converting a variable bill to a fixed one and removing tail latency from quota contention; <strong>context caching</strong> for large static prefixes — re-send 8,000 tokens of unchanging system instruction every turn and you pay for it every turn; and <strong>batch prediction</strong> for anything not user-facing.</p><h3>5. Building agents: ADK</h3><p><strong>Agent Development Kit</strong> is the code-first path — stable v1.0 across Python, Go, Java and TypeScript, with a graph-based workflow API in 2.0. Agent Studio is the low-code canvas over the same runtime; <strong>Agent Garden</strong> is the template library. My rule: prototype in Agent Studio, ship in ADK. Anything versioned, tested and reviewed belongs in a repository.</p><pre>from google.adk.agents import LlmAgent<br>from google.adk.tools.agent_tool import AgentTool</pre><pre>analytics = LlmAgent(<br>    name=&quot;analytics_agent&quot;, model=&quot;gemini-flash&quot;,<br>    instruction=(&quot;Answer quantitative questions using ONLY the governed views &quot;<br>                 &quot;in your tool schema. If it cannot be answered from those &quot;<br>                 &quot;views, say so and name what is missing. Never guess a join.&quot;),<br>    tools=[...])</pre><pre>root = LlmAgent(<br>    name=&quot;orchestrator&quot;, model=&quot;gemini-pro&quot;,<br>    instruction=(&quot;Plan, delegate, synthesise. You do not query datastores &quot;<br>                 &quot;directly. Reconcile conflicting answers by preferring the &quot;<br>                 &quot;more recent, better-cited one.&quot;),<br>    tools=[AgentTool(analytics), AgentTool(knowledge)],<br>    before_model_callback=redact_pii,<br>    after_tool_callback=log_tool_provenance)</pre><pre>adk deploy agent_engine --project $PROJECT --region us-central1 ./app</pre><p>The CLI target still reads agent_engine even though the service is now Agent Runtime. It yields a Reasoning Engine resource — projects/…/locations/…/reasoningEngines/… — the agent&#39;s global address for gateway routing, registry entry, IAM binding and trace attribution.</p><p>Three things I’d insist on in review. <strong>The orchestrator never touches a datastore</strong> — it plans and delegates, keeping its context small and its blast radius smaller. <strong>Callbacks are your enforcement seam</strong>: before_model_callback and after_tool_callback are where redaction and provenance logging live in code, complementing Model Armor at the perimeter. <strong>Use the built-in eval framework from day one</strong> — a golden set of 100 questions written in week two saves you a quarter.</p><h3>6. Interop: MCP downward, A2A sideways</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F64Dxo_p9r5654FwZH6Kww.png" /></figure><p><strong>MCP</strong> is vertical — agent reaching a tool. Google ships native MCP servers including BigQuery and Maps, and Model Armor sanitises MCP tool calls and responses via floor settings, mitigating prompt injection arriving <em>through tool output</em>. That’s the attack most teams haven’t modelled.</p><p><strong>A2A</strong> is horizontal — agent reaching a peer. agents advertise capability through agent cards and negotiate across frameworks and vendors.</p><p>The distinction that stops this becoming spaghetti: <strong>if you can redeploy it, use MCP. If you can only negotiate with it, use A2A.</strong> A peer you cannot debug or version-pin needs timeouts, circuit breakers and a degraded-mode answer.</p><p>When you attach a toolset, always constrain it:</p><pre>bq_tools = MCPToolset(<br>    connection_params=...,                    # the native BigQuery MCP server<br>    tool_filter=[&quot;list_tables&quot;, &quot;describe_table&quot;, &quot;execute_query&quot;])</pre><p>tool_filter is not optional in production. An MCP server exposes what it exposes; the agent should see only the subset you approved.</p><h3>7. Runtime, sessions and memory</h3><p><strong>Agent Runtime</strong> (formerly Agent Engine) handles deployment, autoscaling, sessions, memory and observability. It’s framework-agnostic, but the native path gets managed session and memory services without wiring.</p><p>Lifetime Contents <strong>Session state</strong> One conversation Turn history, intermediate results <strong>Long-term memory</strong> Across conversations Durable facts, preferences, outcomes <strong>Context window</strong> One model call Whatever you chose to send</p><p>ADK exposes <strong>configurable context layers — Static, Turn, User and Cache</strong> — specifically to control what enters the window and cut token usage. This is the most direct cost lever in the architecture: a four-agent system that naively forwards the full session to every sub-agent burns several times the tokens of one passing scoped slices.</p><p><strong>Sub-agents get a slice, not the session</strong> — a cost decision and a blast-radius decision at once. And <strong>write memory deliberately, not continuously</strong>: extract durable facts on session close, filter for relevance, store the distillate. Agents writing every turn build a corpus of noise that degrades every later retrieval.</p><h3>8. The experience layer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FH71kiCtCcP1wmTvDv2Bug.png" /></figure><p>Top of the diagram, near the end of the build order, deliberately. The most common UI mistake in agentic programmes is building a bespoke chat front end for an internal audience who’d have been better served inside a tool they already have open.</p><p><strong>Gemini Enterprise</strong> is the packaged surface — the app employees open. Custom agents built on the platform appear in its dashboard. Zero front-end code, and for most internal knowledge-work cases it’s the right answer; you won’t out-design Google’s search-plus-agents experience on a project budget.</p><p><strong>Chat apps and Workspace add-ons</strong> meet users where they already are. An agent that answers in Google Chat or inside Gmail and Docs gets adopted; a separate portal with a separate login does not. Agents can be published to Cloud Marketplace for wider distribution.</p><p><strong>CX Agent Studio</strong>, part of Gemini Enterprise for Customer Experience, is the external-customer and contact-centre path — the evolution of Dialogflow CX, built on ADK, with a visual builder, multimodal simulator, integrated evaluation and tracing, and human-like voice in 40+ languages. It’s priced per session with voice billed by the second past a threshold. Model the unit economics before committing; per-session behaves very differently from per-token at volume.</p><p><strong>A custom front end</strong> on Cloud Run or Firebase Hosting, behind IAP or Cloud Armor, covers embedded and external web experiences. The recent addition worth knowing is <strong>A2UI (Agent-to-UI)</strong>: the agent renders real interface components — layouts, charts, interactive menus — that update live as the conversation moves, assembling the interface the user needs rather than emitting a wall of prose. For anything data-shaped, a substantial upgrade over chat.</p><p>Two surfaces people forget: <strong>AppSheet</strong>, when the job is forms, approvals and workflow rather than conversation; and <strong>Looker conversational analytics</strong>, when the question is really a metric question and a governed semantic model already exists.</p><p>For development, <strong>adk web</strong> gives a local loop with trace and eval views. It is not a product surface. I&#39;ve seen the ADK dev UI demoed to executives more than once; it always ends the same way.</p><h3>9. Gateway, registry and prompt management</h3><p><strong>Agent Gateway</strong> manages the fleet from one control point — unified connectivity between agents and tools across environments, enforcing consistent policy and Model Armor protection against prompt injection and data leakage. It also gives an authorisation-debugging view over agent egress: which agent, which host, which registered path, authorised or not.</p><p><strong>Agent Registry</strong> indexes every internal agent, tool and skill so only approved assets are discoverable and callable — the answer to the question every platform team eventually asks: <em>how many agents do we actually have?</em></p><p><strong>Prompt management</strong> is a first-class capability and GA. Prompts are versioned resources in your project, created and edited in Studio or programmatically through the SDK, with full enterprise support including CMEK and VPC Service Controls.</p><pre>import vertexai<br>client = vertexai.Client(project=PROJECT, location=LOCATION)</pre><pre># create_version / get / get_version / list / list_versions / delete / restore<br>pinned = client.prompts.get_version(prompt_id=PROMPT_ID, version_id=VERSION_ID)</pre><pre>response = genai_client.models.generate_content(<br>    model=pinned.prompt_data.model,<br>    contents=pinned.assemble_contents())</pre><p>The pattern that matters: <strong>pin an explicit version in production, never float on latest.</strong> Design in Studio, promote through the SDK, and tag the active version on every trace span so a quality regression is traceable to a prompt change rather than guessed at.</p><p><strong>Apigee</strong> remains the model gateway — the LLM-facing concern rather than the agent-facing one. Model Armor integrates into the proxy layer via SanitizeUserPrompt on the request flow and SanitizeModelResponse on the response. Apigee also carries the cost controls: LLMTokenQuota and PromptTokenLimit cap a user or application at a maximum token count per day.</p><p><strong>Do they overlap?</strong> Slightly. Apigee governs traffic to <em>models and APIs</em> — token quotas, rate limits, routing, failover. Agent Gateway governs traffic to and from <em>agents</em> — fleet policy, egress authorisation, tool permissions. Most enterprises run both; starting with one, pick Apigee if your pain is model spend, Agent Gateway if it’s agent sprawl.</p><h3>10. Guardrails and governance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y1a6taJTK5CaQz5vF_o4Jg.png" /></figure><p><strong>Model Armor</strong> is the core primitive — inline inspection on ingress <em>and</em> egress, integrated across Gemini Enterprise, Agent Gateway, Apigee, GKE inference gateway and Google Cloud MCP servers, without application code changes. Coverage: prompt injection and jailbreak detection, DLP infoTypes, malicious URL detection, and responsible-AI filters.</p><p>The rest: <strong>native agent identities</strong> — the agent is a principal, not the user, with its own permissions; <strong>VPC Service Controls</strong>; <strong>Sensitive Data Protection</strong> for de-identification before data crosses a boundary, noting the default Gemini Enterprise template blocks FINANCIAL_ACCOUNT_NUMBER and will silently break finance workflows until tuned; <strong>Agent Threat and Anomaly Detection</strong>, which uses statistical models and LLM-as-a-judge to flag unusual <em>reasoning</em> in real time, catching an agent reasoning its way toward an unusual action before the action; the <strong>Agent Security dashboard</strong>; and <strong>Cloud Audit Logs</strong>, worth exporting to BigQuery for joins against token spend.</p><p><strong>On latency:</strong> the enforcement hops add roughly 110 ms of fixed overhead against 0.4–6 seconds of inference. A rounding error. Teams shipping slow agents aren’t slow because of guardrails — they’re slow because they chained four sequential model calls before the first token reached the user.</p><h3>11. Observability and FinOps</h3><p>With a normal API, the same request costs the same every time. With an agent, it doesn’t. One question might use 800 tokens; the same question tomorrow might use 40,000, because the agent decided to call more tools. That makes budgeting hard unless you measure it.</p><p>Three things to put in place:</p><p><strong>Trace every step.</strong> Cloud Trace records the full call tree — orchestrator, each sub-agent, each tool call, each model call. When an agent is slow, this is the only view that tells you <em>which</em> step was slow.</p><p><strong>Tag every model call</strong> with the agent name, session, tenant and use case. Then export your billing data to BigQuery and join the two. Once that’s in place, “what does one conversation cost us?” is a SQL query instead of a guess.</p><p><strong>Test agents like code.</strong> Keep a fixed set of questions with known-good answers, score every release against it automatically, and block the deploy if quality drops. Agents get worse for reasons that have nothing to do with your code — a table changed upstream, the document set grew, the model version rolled forward. Without a regression gate you find out from a user.</p><p>One thing that surprises most teams the first time they measure it: the biggest cost driver is almost never which model you picked. It’s how many turns a conversation takes, and how much unchanging context you resend on every one of them.</p><h3>Conclusion</h3><p>If you take five things from this:</p><ol><li><strong>The agent is the small part.</strong> Semantics, access control, freshness, cost attribution and evaluation are where the work is — and every one of them is a data engineering problem wearing a new hat.</li><li><strong>Build on the estate you have.</strong> Governed views, existing pipelines, existing lineage. Greenfield is the slowest path.</li><li><strong>Ground before you retrieve.</strong> Knowledge Catalog decides what this agent may use, for this user, right now. Skip it and you will ship an agent that is confidently wrong or quietly over-permissioned.</li><li><strong>Enforce inside, not after.</strong> Access control belongs in the retrieval query; guardrails belong at every hop. Roughly 110 ms of governance around a 2-second answer is a trade worth making.</li><li><strong>Ship into Gemini Enterprise first.</strong> Build a custom UI only when the packaged surface genuinely doesn’t fit.</li></ol><p>Google now ships the runtime, the registry, the gateway and the guardrails. That’s the part you no longer have to build — which means the quality of what you ship comes down to the layer underneath.</p><p>Build that first.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cac3a86c1120" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Semantic / Context Layer: Grounding Agentic AI in Enterprise Truth]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/the-semantic-context-layer-grounding-agentic-ai-in-enterprise-truth-6c31226b227c?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c31226b227c</guid>
            <category><![CDATA[semantic-layer]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[anthropics]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[google-gemini]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Wed, 24 Jun 2026 22:43:23 GMT</pubDate>
            <atom:updated>2026-09-25T17:08:50.155Z</atom:updated>
            <content:encoded><![CDATA[<h4>If the LLM Is the Brain, the Semantic Layer Is the Heart of Agentic AI, <strong>Building the context layer your agents are missing</strong></h4><p><em>Every enterprise is building AI agents. Most are building them on a shaky foundation.</em></p><p>Walk into any large company and you’ll find the same shopping list checked off: a frontier LLM’s (Claude, GPT, Gemini), an enterprise data platform, a lake full of unstructured documents, a vector database, a clutter of agents, an orchestration framework (LangGraph, CrewAI), a pile of tools, and some flavor of memory. The architecture diagrams look impressive. The demos work.</p><p>Then the agents meet production, and something strange happens. Two agents answer the same revenue question with two different numbers. A “simple” question about last quarter returns SQL against the wrong table. Costs balloon. Trust evaporates. The pilot that was going to transform self-serve analytics gets quietly shelved.</p><blockquote>The missing piece isn’t a better model. It’s the layer almost nobody put on the diagram: the <strong>semantic / context layer</strong>.</blockquote><p>If the LLM is the brain of an agentic system, the semantic layer is the heart. The brain can be brilliant and still useless if nothing is pumping clean, governed, consistent meaning to it. This post is about that heart — what breaks without it, what it actually is, why it matters, and how the major platforms (Google, Microsoft, Databricks, Snowflake, AWS, Anthropic) let you build one today.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XzY1xRoqBydIvkO06TYTDg.png" /><figcaption>The LLM is the brain (reasoning, agents, orchestration, MCP). The semantic / context layer is the heart — governed metrics, ontology, knowledge graph, memory, lineage. The enterprise data and tools are the body. Strip out the heart and the brain still “works,” but it hallucinates business logic and metrics drift.</figcaption></figure><ol><li><strong>The Real Problem: A Brain With No Heart</strong></li></ol><p>An LLM is, at its core, a probabilistic reasoning engine trained on the public internet. It has never seen your FCT_ORDERS table. It does not know that your company defines <em>net revenue</em> as gross revenue after discounts <strong>and</strong> returns, that &quot;last quarter&quot; means the most recently <em>completed</em> fiscal quarter, or that the column actually called amt_ttl_pre_dsc is the thing finance calls &quot;topline.&quot; Hand a brilliant model your raw schema and ask a business question, and it will confidently invent the business logic it&#39;s missing. That&#39;s not a bug in the model. That&#39;s the model doing exactly what it was built to do with insufficient context.</p><p><strong>Metric drift.</strong> Ask three agents — or three dashboards — for “total sales” and you get three numbers, because “total sales” means retail-only to one team, retail-plus-wholesale to another, and retail-plus-wholesale-minus-returns to finance. Without a single governed definition, every agent re-derives business logic on the fly. Multiply that across thousands of agents and you don’t have analytics, you have an argument.</p><p><strong>RAG is the wrong shape for the question.</strong> Retrieval-Augmented Generation was designed to find the relevant <em>passage</em> in a pile of <em>unstructured text</em>. But most enterprise questions aren’t retrieval-shaped — they’re computational. “Net revenue by region last quarter” isn’t sitting in a paragraph somewhere waiting to be retrieved; it has to be <em>computed</em>, correctly, across structured systems, using a definition. RAG handles none of that natively.</p><p><strong>MCP is plumbing, not meaning.</strong> The Model Context Protocol has rightly become the “USB-C port for AI” — the standard wire for connecting agents to tools and data. But a wire only carries context; it doesn’t create it. MCP without a semantic layer underneath just opens every valve to your warehouse with no map and no filter.</p><p><strong><em>The common thread</em></strong>: all the components in your stack are necessary, and none of them — not the model, not the vector DB, not MCP, not the orchestration framework — supplies <em>governed business meaning</em>. That’s the gap. The market is catching up to a simple truth: <strong>AI without governed semantics cannot scale.</strong></p><h3>2. What the Semantic / Context Layer Actually Is</h3><p>Let’s be precise, because three terms get blurred together — and the difference matters when you start building.</p><p><strong>The semantic layer</strong> is the governed <em>vocabulary</em> of your business. It’s where “net revenue,” “active customer,” “churn,” and “region” are each defined exactly once — as metrics, dimensions, and relationships mapped onto your physical tables — so that every consumer, human or machine, gets the same answer. The classic BI semantic layer (think Power BI’s tabular model, or Looker’s LookML) was built for dashboards: passive, read-only, designed for a human clicking around.</p><p><strong>The context layer</strong> is bigger than the semantic layer. There are really two things people mean by “context,” at two different levels. The first is <em>runtime context</em> — what you feed the model on each individual call: the current conversation, what it remembers from this session, what it remembers long-term, and which tools it can use. Anthropic calls assembling this “context engineering.” The second is the <em>enterprise context layer</em>, which sits one level above that: the permanent, shared foundation every agent pulls from — your governed definitions, who’s allowed to see what, and where each number came from — spanning <em>all</em> your systems, not just one data warehouse. Put simply: runtime context is what one agent sees in the moment; the enterprise context layer is the single source of truth that <em>every</em> agent draws on, everywhere.</p><p><strong>MCP</strong> is neither of those. It’s the transport — the protocol that lets the agent <em>reach</em> the semantic/context layer. Connectivity, not intelligence.</p><h3>So what’s the actual difference between the semantic layer and the context layer?</h3><p>Short version: <strong>the semantic layer is one part of the context layer.</strong></p><p>The <strong>semantic layer</strong> answers a narrow, precise question: <em>what do our business terms mean and how are they calculated?</em> It’s the dictionary of metrics, dimensions, and relationships — “net revenue = gross minus discounts and returns,” defined once. It’s the heart of the heart, but it’s mostly about <em>numbers and definitions</em>.</p><p>The <strong>context layer</strong> answers a much wider question: <em>everything an agent needs to know to use our data correctly and safely.</em> That includes the semantic layer’s definitions <strong>plus</strong> the surrounding fabric — which data is trustworthy and certified, where it came from (lineage), who’s allowed to see it (permissions), what it looks like (schema and data types), how the tables relate, and the unwritten “house rules” an agent should follow (the caveats and join paths that live in nobody’s schema).</p><h3>Context is richer than just metrics</h3><p>,which describes context as a multi-layered fabric of six metadata types:</p><ul><li><strong>Technical metadata</strong> — schema, data types, security tags (IAM), straight from source systems.</li><li><strong>Semantic metadata</strong> — business ontologies, glossaries, classifications.</li><li><strong>Operational metadata</strong> — lineage, data profiles, usage stats, quality scores.</li><li><strong>Relationships</strong> — the connections between physical assets and business concepts.</li><li><strong>Trust metadata</strong> — signals indicating whether data comes from a certified, authoritative source.</li><li><strong>Agent guidance</strong> — unstructured “READMEs for agents”: the filters to apply before aggregating, the join paths to follow, the caveats invisible in the schema.</li></ul><p>The new definition, increasingly converged across the industry, looks like this:</p><blockquote><strong><em>Contextual semantic layer = governed metrics + ontology + knowledge graph + memory + LLM orchestration.</em></strong></blockquote><h3>3. Why It’s Worth the Effort: The Advantages</h3><ol><li><strong>Accuracy that’s actually deployable.</strong> This is the headline. Studies show LLM accuracy on data questions improving by as much as <strong>3x</strong> when the model queries through a semantic layer instead of raw tables. Databricks reported its Genie Ontology context engine lifting answer accuracy from ~50% to <strong>84.5%</strong> versus naive text-to-SQL. And Bloomberg Media’s internal “Knowledge Gap Agent” improved its Data Access Agent’s SQL accuracy by <strong>63%</strong> simply by capturing missing institutional knowledge as governed context. The model stops guessing business logic because the business logic is handed to it.</li><li><strong>Consistency — the end of metric drift.</strong> Define net_revenue once; every agent, dashboard, and notebook inherits it. The number finance signs off on is the number your customer-facing agent quotes. No more reconciling three &quot;truths&quot;.</li><li><strong>Governance, security, and lineage by construction.</strong> A good semantic layer enforces RBAC at query time and carries lineage with every result. Snowflake’s Cortex Analyst respects role-based access; Microsoft’s Fabric Data Agents honor Purview policies and row/column-level security; AWS Context makes every agent query identity-aware via IAM and Lake Formation. An agent can’t accidentally leak a column a user isn’t entitled to, and every answer comes with a “here’s where this came from” trail. This is the difference between a fun demo and something legal and compliance will let you ship.</li><li><strong>Cost control.</strong> When definitions live in one governed place, you stop repeating them in every prompt and you keep more questions inside governed business logic instead of degrading into expensive, unpredictable raw-table behavior. Databricks now wraps this in a Unity AI Gateway with hard spend caps; Better semantic coverage is, very directly, a lower and more stable bill.</li><li><strong>Explainability and trust.</strong> Because metrics, relationships, and verified queries are explicit, the system can <em>show its work</em>. That’s what turns a “confident” agent into a <em>trusted</em> one, especially as agents move from answering questions to taking actions (adjusting pricing, placing orders, triaging tickets).</li><li><strong>Interoperability and future-proofing.</strong> Define a metric once and serve it to Claude, to ChatGPT, to your in-house agent, and to Power BI — without bespoke glue for each.</li></ol><h3>4. How to Build One: The Platform Playbook</h3><p>There’s no single product called “the semantic layer” — you assemble it.</p><h4>Google Cloud —<strong> Knowledge Catalog (the universal context engine)</strong></h4><p>The most complete “context layer as a product” play. <strong>Knowledge Catalog</strong>, a Gemini-powered, always-on context engine that builds a dynamic context graph to ground agents and reduce hallucinations. It runs on three pillars: <strong>Aggregation</strong> (unify technical metadata across BigQuery, AlloyDB, Spanner, Looker, and third-party catalogs like Atlan/Collibra into one governed source of truth), <strong>Enrichment</strong> (Gemini auto-generates descriptions, business glossaries, relationships, and verified “golden” SQL patterns), and <strong>Search/Retrieval</strong> (semantic search plus a Context Retrieval API and MCP tools like lookupContext). Looker/LookML supplies the metric semantics; the Deep Research Agent in Gemini Enterprise consumes it. If you&#39;re GCP-centric — like a Vertex/Gemini agentic build — this is your heart.</p><h4>Microsoft / Azure — Power BI semantic models + Fabric IQ + Purview</h4><p>Microsoft’s edge is that millions of businesses already use Power BI semantic models, and it has turned those into the grounding layer for AI under <strong>Fabric IQ</strong> — pairing them with ontologies and Purview governance so Fabric Data Agents answer questions in plain English while respecting each user’s permissions.</p><h4>AWS — AWS Context + SageMaker Catalog + Bedrock</h4><p><strong>AWS Context</strong>: a service that automatically maps your data into a knowledge graph and serves governed relationships, business rules, and domain knowledge to agents at runtime via agentic-search APIs and <strong>MCP tools</strong>. Every query is identity-aware (inheriting IAM and Lake Formation permissions) and auditable, with metadata published in open Apache Iceberg on S3. It’s fed by <strong>Glue Data Catalog skill assets</strong> (join paths, query patterns, usage rules — the “agent guidance” layer), <strong>S3 Annotations</strong>, and <strong>SageMaker Catalog</strong> (built on DataZone: semantic search, GenAI-generated metadata, data products, lineage, Amazon Q). <strong>Bedrock AgentCore</strong> runs the agents (with memory, gateway, and identity). If you already run S3/Glue/Lake Formation, the pitch is zero data movement.</p><h4>Databricks — Unity Catalog Semantics +Genie Ontology</h4><p>A clean separation of the two jobs. <strong>Unity Catalog</strong> is the unified governance plane for data, AI assets, agents, and MCPs . On top sits <strong>Unity Catalog Business Semantics / Metric Views</strong> — define KPIs like revenue, churn, and margin once in SQL/YAML (with synonyms, formats, relationships), governed and <strong>open-sourced / OSI-compliant</strong>, queryable from SQL, APIs, BI, and agents via MCP. Feeding the agents is <strong>Genie Ontology</strong> : a self-improving context engine that ranks the authoritative definition via “ontorank” . <strong>Genie One</strong> (the agentic coworker) and <strong>Agent Bricks</strong> consume it.</p><h4>Snowflake — Cortex Analyst + Semantic Views</h4><p>Warehouse-native and tightly governed. <strong>Semantic Views</strong> are schema-level objects that encode logical tables, dimensions, facts, metrics, relationships, synonyms, verified queries, and custom instructions — with full RBAC and sharing. <strong>Cortex Analyst</strong> is the LLM-powered text-to-SQL engine that reads the semantic view and generates governed SQL that runs in your warehouse; <strong>Cortex Agents</strong> orchestrate it alongside <strong>Cortex Search</strong> for unstructured data. Snowflake’s newer <strong>Horizon Context / Cortex Sense</strong> extends this toward a cross-system context approach. Because semantic views are warehouse-bounded, pair them with a cross-system context layer when meaning spans beyond Snowflake.</p><h4>Anthropic — the brain and the wire (MCP)</h4><p>Anthropic doesn’t sell a semantic layer; it standardized how every agent <em>reaches</em> one. <strong>MCP</strong> is now the de-facto protocol for connecting agents to governed context — Google’s Knowledge Catalog, Databricks’ Unity Catalog, Snowflake, and AWS Context all expose themselves to Claude through it. Pair that with Anthropic’s <strong>context-engineering</strong> discipline (working/session/long-term memory + tool context per call) and Claude as the reasoning brain, and you have the top and the spine of the diagram. The pattern: Anthropic provides the brain and the nervous system; you bring the heart.</p><h4>OpenAI — a brain that consumes a layer, not one that provides it</h4><p>Worth stating plainly for architects: OpenAI is a model/brain provider, not a semantic-layer vendor. ChatGPT Enterprise offers connectors and tools, and OpenAI adopted MCP — so GPT agents <em>consume</em> a governed semantic layer (yours, on one of the platforms above) rather than supplying one. Like Anthropic and Google, OpenAI sits outside the OSI standard. Plan to build your context layer on your data platform and let the model — GPT, Claude, or Gemini — reach it over MCP.</p><h3>The two standards that tie it together: OSI and MCP</h3><p>Two open standards make this portable. <strong>OSI (Open Semantic Interchange)</strong> — v1.0 shipped on GitHub in January 2026, built on MetricFlow — lets you define a metric once in vendor-neutral YAML and have any compliant tool consume it. Snowflake, Databricks, dbt, Cube, AtScale, Salesforce, Tableau, BlackRock and 40+ partners are in; notably absent are Microsoft, SAP, IBM, Oracle, and the frontier labs (OpenAI, Anthropic, Google). <strong>MCP</strong> is the transport that carries it to the agent. The irony worth noting: the labs that build the brains (Anthropic, OpenAI, Google) and the wire (MCP) largely sit <em>outside</em> the meaning standard (OSI) — so the semantic layer remains the piece you, the enterprise, own.</p><h3>A concrete example: a governed semantic model</h3><p>Theory is easy; let’s make it real. Here’s a trimmed Snowflake-style semantic view for a retail revenue domain — the kind of thing that answers the exact question in the diagram. Notice the messy physical column amt_ttl_pre_dsc never reaches the agent; net_revenue is defined exactly once, with the discount-and-returns logic baked in.</p><blockquote>Store semantic files(YAML) into a Snowflake Semantic View or Google’s catalog etc → the agent reads it from there over MCP</blockquote><pre>name: retail_revenue<br>description: Governed revenue model for the retail analytics domain.<br><br>tables:<br>  - name: orders<br>    base_table: { database: SALES, schema: PUBLIC, table: FCT_ORDERS }<br>    primary_key: { columns: [order_id] }<br><br>    dimensions:<br>      - name: region<br>        expr: region<br>        synonyms: [&quot;area&quot;, &quot;geo&quot;, &quot;territory&quot;]<br>        description: Sales region the order was booked in.<br>      - name: channel<br>        expr: sales_channel<br>        synonyms: [&quot;retail vs wholesale&quot;]<br><br>    time_dimensions:<br>      - name: order_date<br>        expr: order_date<br>        data_type: date<br><br>    facts:<br>      - name: gross_revenue<br>        expr: amt_ttl_pre_dsc            # physical column no human can read<br>        description: Order revenue before discounts.<br>      - name: discount_pct<br>        expr: discount_pct<br>        access_modifier: private_access  # used in math, never exposed alone<br><br>    metrics:<br>      - name: net_revenue<br>        description: Gross revenue after discounts and returns. THE company definition.<br>        expr: SUM(gross_revenue * (1 - discount_pct))<br>        synonyms: [&quot;revenue&quot;, &quot;net sales&quot;, &quot;topline&quot;]<br><br>  - name: customers<br>    base_table: { database: SALES, schema: PUBLIC, table: DIM_CUSTOMER }<br>    primary_key: { columns: [customer_id] }<br><br>relationships:<br>  - name: orders_to_customers<br>    left_table: orders<br>    right_table: customers<br>    expr: orders.customer_id = customers.customer_id<br><br># Few-shot patterns the model trusts and reuses (Google calls these &quot;golden queries&quot;)<br>verified_queries:<br>  - name: net_revenue_by_region_last_quarter<br>    question: &quot;What was net revenue by region last quarter?&quot;<br>    sql: &gt;<br>      SELECT region,<br>             SUM(amt_ttl_pre_dsc * (1 - discount_pct)) AS net_revenue<br>      FROM SALES.PUBLIC.FCT_ORDERS<br>      WHERE order_date &gt;= DATE_TRUNC(&#39;quarter&#39;, DATEADD(&#39;quarter&#39;, -1, CURRENT_DATE()))<br>        AND order_date &lt;  DATE_TRUNC(&#39;quarter&#39;, CURRENT_DATE())<br>      GROUP BY region<br>      ORDER BY net_revenue DESC<br><br>custom_instructions: &gt;<br>  &quot;Last quarter&quot; always means the most recently completed calendar quarter.<br>  Never aggregate gross_revenue directly; &quot;revenue&quot; always resolves to net_revenue.</pre><p>This file <em>is</em> the heart for this domain: synonyms (so “topline” and “geo” resolve), the one true metric definition, join paths (so the model doesn’t guess), verified queries (high-trust few-shot examples), and custom instructions (defaults and guardrails). Keep it in Git, review it like code, run it through CI/CD, and track usage — the modern semantic layer is owned and released like a product, not maintained like a wiki.</p><h3>Wire it to the agent as the only door to data</h3><p>The move that actually delivers the safety: the agent doesn’t get raw warehouse access. It gets <strong>one governed tool</strong> — the semantic layer — exposed over MCP and dropped into your orchestration graph. The model still reasons; it just can’t reach the data except through the heart.</p><pre># Illustrative LangGraph wiring. The agent&#39;s ONLY data tool is the semantic layer.<br>from langgraph.prebuilt import create_react_agent<br><br>def query_semantic_layer(question: str) -&gt; dict:<br>    &quot;&quot;&quot;Answer a business question against the governed semantic model.<br>    Returns rows + the exact SQL + lineage. Metric definitions (e.g. net_revenue)<br>    are fixed in the model — the LLM does not invent them.&quot;&quot;&quot;<br>    return cortex_analyst.ask(semantic_view=&quot;retail_revenue&quot;, question=question)<br><br>agent = create_react_agent(<br>    model=&quot;claude-...&quot;,            # the brain: reasoning + planning<br>    tools=[query_semantic_layer],  # the heart: the single, governed door to data<br>)</pre><h3>5. Conclusion: Build the Heart First</h3><p>Don’t treat the context layer as something to add later. So build the heart first. Ask one question: <em>where does our governed business meaning live, and how does every agent reach it?</em> If the answer is “in the prompt” or “the model figures it out,” that’s your highest-leverage work. And you don’t have to build from scratch — Google Knowledge Catalog, Databricks Unity Catalog, Snowflake Cortex, Microsoft Fabric IQ, and AWS Context are all ready to use, reachable by any model over MCP.</p><p>Models keep getting cheaper and smarter. The lasting advantage is the heart — build that, and everything above it gets smarter, safer, and cheaper at once.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c31226b227c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing Claude Code for Cost, Speed, and Productivity]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/optimizing-claude-code-for-cost-speed-and-productivity-5122fb9ce1de?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5122fb9ce1de</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-coding]]></category>
            <category><![CDATA[anthropic-claude]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Wed, 03 Jun 2026 01:25:10 GMT</pubDate>
            <atom:updated>2026-09-25T17:11:33.066Z</atom:updated>
            <content:encoded><![CDATA[<p>There’s a moment every team hits with Claude Code. The pilot goes great — two engineers ship a week’s work in a day — and then the first real invoice lands and someone in finance asks a very fair question: <em>why are we paying this much?</em></p><p>Here’s the thing most teams miss while staring at that bill: <strong>the sloppy habits that inflate your cost are the exact same habits that make Claude slower and worse at its job.</strong> Dumping your whole repo into context doesn’t just cost more — it buries the signal, drags response times, and degrades the model’s reasoning. So optimization isn’t a tax you pay to save money. It’s a three-for-one: tune Claude Code well and it gets <em>cheaper, faster, and sharper</em> at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4VIZdFC4zG7CW52YNeQ80g.png" /></figure><h3>The one idea behind all three wins</h3><p>Claude Code charges by tokens, and tokens scale with <strong>context size</strong> — not with how hard you’re working. Every turn, Claude re-processes the earlier messages, the files it already read, the tool outputs, and your memory files. That single fact explains the whole playbook:</p><ul><li><strong>Cost:</strong> more context = more tokens = bigger bill.</li><li><strong>Performance:</strong> more context = slower responses <em>and</em> “context rot,” where accumulated noise crowds out the signal and the agent starts making mistakes it wasn’t making an hour ago.</li><li><strong>Productivity:</strong> a focused agent finishes the task on the first try instead of the third.</li></ul><p>So the north star is simple: <strong>send the model the least context that still lets it do great work.</strong> Everything below is a way to do that, organized by the goal it serves most — though you’ll notice almost every tactic quietly helps all three.</p><h3>Part 1 — Spend less</h3><h3>Right-size the model</h3><p>Start every session on Sonnet; it handles most coding well. Reserve Opus (roughly 5× the per-token cost) for genuine architecture and multi-system debugging. Drop to Haiku for the trivial stuff.</p><pre>/model sonnet   # default: tests, edits, refactors<br>/model opus     # reserve: cross-system debugging, architecture<br>/model haiku    # cheap: lookups, renaming, formatting</pre><p>The biggest lever here is pinning cheap models to your <strong>subagents</strong>, so high-volume grunt work never touches Opus:</p><pre># .claude/agents/test-runner.md<br>---<br>name: test-runner<br>description: Runs the suite and reports only failures<br>model: haiku<br>tools: Read, Bash<br>---<br>Run the project&#39;s tests. Return a one-paragraph summary of failures only.</pre><blockquote><strong><em>Note:</em></strong><em> model selection is the highest-ROI, lowest-friction policy you can set. A team default of Sonnet, with Opus treated as “ask first,” moves the cost curve more than anything else.</em></blockquote><h3>Keep context lean</h3><p>Context is the meter — watch it and trim it.</p><ul><li><strong>/clear</strong> between unrelated tasks. Stale context taxes every later message. Use /rename first so you can /resume it.</li><li><strong>/compact with intent:</strong> /compact Focus on the API changes and test output. You can standardize this in CLAUDE.md:</li></ul><pre># Compact instructions<br>When summarizing this conversation:<br>- Preserve all API changes and their rationale<br>- Keep error messages and their fixes<br>- Summarize exploration attempts briefly</pre><ul><li><strong>/context</strong> shows what&#39;s eating space right now.</li><li><strong>.claudeignore</strong> keeps node_modules, build output, and lockfiles out of view every session.</li></ul><p>Claude Code also helps automatically via <strong>prompt caching</strong> (repeated content like system prompts is cheaper) and <strong>auto-compaction</strong> near the limit — but those are safety nets, not a strategy.</p><h3>Turn down thinking you don’t need</h3><p>Extended thinking is on by default and earns its keep on hard problems, but thinking tokens bill as <em>output</em>, and the budget can hit tens of thousands per request. For routine work:</p><pre>/effort low<br>export MAX_THINKING_TOKENS=8000</pre><h3>Don’t send everything — preprocess locally</h3><p>The cheapest token is the one you never send. A huge share of what gets fed to an LLM is deterministic work a five-line script does better, faster, and free.</p><p><strong>Hooks</strong> intercept and shrink data <em>before</em> the model sees it. Instead of letting Claude read a 10,000-line log to find errors, a PreToolUse hook greps for failures and returns only those lines — tens of thousands of tokens down to a few hundred:</p><pre>// settings.json<br>{<br>  &quot;hooks&quot;: {<br>    &quot;PreToolUse&quot;: [<br>      { &quot;matcher&quot;: &quot;Bash&quot;,<br>        &quot;hooks&quot;: [ { &quot;type&quot;: &quot;command&quot;, &quot;command&quot;: &quot;~/.claude/hooks/filter-test-output.sh&quot; } ] }<br>    ]<br>  }<br>}</pre><pre>#!/bin/bash<br># ~/.claude/hooks/filter-test-output.sh<br>input=$(cat); cmd=$(echo &quot;$input&quot; | jq -r &#39;.tool_input.command&#39;)<br>if [[ &quot;$cmd&quot; =~ ^(npm test|pytest|go test) ]]; then<br>  filtered=&quot;$cmd 2&gt;&amp;1 | grep -A 5 -E &#39;(FAIL|ERROR|error:)&#39; | head -100&quot;<br>  echo &quot;{\&quot;hookSpecificOutput\&quot;:{\&quot;hookEventName\&quot;:\&quot;PreToolUse\&quot;,\&quot;permissionDecision\&quot;:\&quot;allow\&quot;,\&quot;updatedInput\&quot;:{\&quot;command\&quot;:\&quot;$filtered\&quot;}}}&quot;<br>else echo &quot;{}&quot;; fi</pre><p>Same philosophy, more places: run <strong>linting, formatting, type-checking, and validation as scripts or in CI</strong> — don’t pay a model to do what eslint or mypy already does. Install <strong>code-intelligence plugins</strong> for typed languages so &quot;go to definition&quot; replaces a grep-then-read-five-files dance. And <strong>trim MCP overhead</strong> — disable unused servers (/mcp) and prefer CLI tools (gh, aws) that don&#39;t add per-tool listing.</p><h3>Make spend visible and bounded (leadership)</h3><p>You can’t manage what you can’t see. <strong>/usage</strong> shows session tokens and attributes them to skills, subagents, plugins, and MCP servers; <strong>ccusage</strong> (third-party) tracks trends. Then bound it: set <strong>workspace spend limits</strong> and a <strong>rate limit</strong> on the API; cap monthly credits with <strong>/usage-credits</strong> on Pro/Max; and give teams <strong>per-user TPM/RPM</strong> guardrails (the per-user share scales down as the team grows). The winning pattern is <strong>track → baseline → bound</strong>: pilot small, set a per-dev baseline, then cap before rollout. Budgets stop being a surprise and become a dial.</p><h3>Part 2 — Go faster and sharper</h3><p>Speed and quality come from the same place as savings — a focused agent — plus a few habits that keep Claude from sprinting in the wrong direction.</p><h3>Plan before you build</h3><p><strong>Plan mode</strong> (Shift+Tab) makes Claude explore and propose an approach <em>before</em> writing code. You approve the plan, which prevents the single most expensive failure mode: twenty confident minutes spent going the wrong way. It’s also the difference between a messy PR and a clean one.</p><h3>Course-correct early</h3><p>Hit <strong>Escape</strong> the instant it drifts. Use <strong>/rewind</strong> to roll back conversation and code to a checkpoint instead of arguing your way out of a bad path. Cheap, fast, and it keeps context clean.</p><h3>Give it a way to check its own work</h3><p>Paste a failing test, the expected output, or a screenshot. When Claude can verify itself, it fixes issues <em>before</em> you ask — one round trip instead of three. This is the highest-leverage thing you can add to a prompt.</p><h3>Isolate verbose work in subagents</h3><p>Running tests, fetching docs, or chewing through logs generates noise. Delegate it to a <strong>subagent</strong> so the verbose output stays in <em>its</em> context and only a clean summary returns to your main conversation — your primary context stays sharp and cheap. Subagents and MCP connections now initialize in parallel, so spinning one up is fast. You can even fire several at once in a single message and they run concurrently:</p><blockquote><em>“Review the auth module for security issues, run the API test suite, and check the migration for breaking changes.”</em></blockquote><p>Three subagents, three clean contexts, running at the same time.</p><h3>Review in a fresh context</h3><p>The strongest quality move: have a reviewer run in a <strong>fresh subagent</strong> that sees only the diff and your criteria — not the reasoning that produced the change. It judges the result on its own terms, the way a real reviewer would.</p><pre># .claude/agents/code-reviewer.md<br>---<br>name: code-reviewer<br>description: Reviews changes for quality and security. Use proactively after edits.<br>tools: Read, Glob, Grep<br>model: sonnet<br>---<br>You are a senior reviewer. Focus on correctness, security, and clarity.<br>Give specific, actionable feedback on the diff only.</pre><h3>Part 3 — Ship more</h3><p>This is where Claude Code stops being a fancy autocomplete and becomes leverage.</p><h3>Treat CLAUDE.md as a lean operating manual</h3><p>CLAUDE.md loads at the start of every session, so it must stay tight — aim for under ~200 lines. The test for each line: <em>if I delete this, will Claude make mistakes?</em> If not, cut it or move it to a skill.</p><pre># Project: payments-api<br>## Stack<br>Node 20, TypeScript, Postgres, Jest.<br>## Conventions<br>- Validate inputs at the route boundary.<br>- Never log secrets or PII.<br>- Tests live next to source as *.test.ts.<br>## Commands<br>- Test: `npm test`   - Lint: `npm run lint`</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*twljlBvpm8vDEK8IJ8m0eQ.png" /></figure><h3>Turn repeated work into skills and commands</h3><p>If you do something more than once a day, stop re-typing the prompt. <strong>Skills</strong> load on demand and bundle real domain logic plus helper files (.claude/skills/&lt;name&gt;/SKILL.md); <strong>slash commands</strong> are lightweight prompt templates for inner-loop workflows like /pr-summary or /release-notes. (The old .claude/commands/ system has merged into skills, and skills take priority over same-named commands.) For risky operations, set disable-model-invocation: true so Claude can&#39;t trigger them by accident. The payoff is double: less repeated prompting <em>and</em> a smaller base context, since specialized instructions live in skills instead of bloating CLAUDE.md.</p><h3>Go parallel when scale demands it</h3><p>One engineer can now drive several Claude sessions at once — the closest thing yet to “multiple engineers on one feature”:</p><ul><li><strong>Git worktrees:</strong> isolated checkouts so parallel sessions don’t overwrite each other. Teams reliably run 4–8 concurrent worktrees per developer; above that you’re usually bottlenecked on <em>review</em>, not Claude.</li><li><strong>Desktop app / Claude Code on the web:</strong> manage multiple local sessions, or run them on Anthropic-managed cloud VMs.</li><li><strong>Agent teams:</strong> automated coordination where one session acts as team lead and teammates each work in their own 200K-token context. Powerful, but note teams can use ~7× the tokens of a single session — keep tasks small and clean them up when done.</li></ul><h3>Share the setup with plugins</h3><p>A <strong>plugin</strong> bundles skills, subagents, commands, hooks, output styles, and MCP definitions into one installable unit. It’s the canonical way to give a whole team the same tuned environment — your security-review skill, your secret-scanning hook, your internal-API MCP — with a single /plugin install.</p><blockquote><strong><em>Note:</em></strong><em> skills, commands, and plugins are how good individual habits become </em>organizational<em> defaults. The best engineer’s workflow becomes everyone’s baseline, which is where team-level productivity actually compounds.</em></blockquote><h3>The whole thing, in one picture</h3><p>One task can run down two very different paths. On the costly path you shovel the entire codebase and every log into the most expensive model on a session that never ends — and the bill, the latency, and the error rate all climb together. On the lean path you filter noise locally, send only what’s needed, right-size the model, and keep context tight — and you arrive cheaper, faster, <em>and</em> with better output. Same destination; the fuel you send decides the trip.</p><p><em>None of this makes Claude Code less capable. It makes it sustainable — and in a world where cost and infrastructure, not raw capability, are becoming the real ceiling on AI, sustainable is the difference between a tool you pilot and one you actually scale.</em></p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5122fb9ce1de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI is Easy to Build, Expensive to Run: An 8-Layer Agentic AI Optimization Playbook]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/agentic-ai-is-easy-to-build-expensive-to-run-an-8-layer-agentic-ai-optimization-playbook-36da6fe42990?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/36da6fe42990</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[anthropics]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Tue, 26 May 2026 03:34:02 GMT</pubDate>
            <atom:updated>2026-09-25T17:12:23.688Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Why the next phase of Agentic AI isn’t about building more agents — it’s about defending the Profit &amp; Loss</strong></p><p>Everyone is building Agentic AI right now. Hackathons, internal POCs, vendor demos, “AI strategy” decks at every Monday standup. It works — until the bill arrives.</p><p>A feature that used to be a 40-millisecond REST call is now a 6-second multi-agent dance that costs four cents per invocation. Multiply that by a million calls a month and the CFO suddenly has a strong opinion about your architecture diagram.</p><p>The first wave of Agentic AI was about <strong><em>building</em></strong> agents. The wave most teams are entering right now is different. It is about <strong>optimizing what we already built</strong> — making it faster, cheaper, more predictable, and operationally boring in the best possible way.</p><h3>The Pre-Question Nobody Asks: Do You Even Need Agentic AI?</h3><p>Before optimizing an agentic system, ask the uncomfortable question: <em>should this have been agentic in the first place?</em></p><p>Take a real pattern I see constantly, A team has 100+ REST APIs and wants a “smart” middle layer to pick which to call based on user input. They build three agents, an LLM orchestrator, a vector store, and demo it proudly.</p><p>That is not an Agentic AI problem. That is a routing problem. A Spring Boot service with a switch statement is <strong>50–100x cheaper</strong>, <strong>20–50x faster</strong>, deterministic, debuggable, and not subject to “the LLM hallucinated the endpoint name today.”</p><p><em>Agents aren’t a free upgrade. They’re the price you pay when things are ambiguous.</em></p><p>Run through this five-question rubric. If you fail even one, skip the agentic approach:</p><ul><li><strong>Are the possible inputs limited?</strong> If users pick from a list or choose from set options, a dropdown beats sending text to an LLM.</li><li><strong>Do the steps always run in the same order?</strong> If it’s always A → B → C, you need a workflow tool, not an AI that “thinks.”</li><li><strong>Do you actually need messy, free-form input?</strong> If not, you’re wasting tokens (and money) on something a regular script could handle.</li><li><strong>Is a wrong answer expensive?</strong> AI agents can snowball small mistakes into big ones. Regular code doesn’t.</li><li><strong>Can users wait?</strong> LLM calls take 0.5–2 seconds. Normal API calls take 20–200 milliseconds — up to 100x faster.</li></ul><p>If you’ve cleared the rubric, the rest of this post is meant for you.</p><h3><strong>The Agentic AI Optimization Stack</strong></h3><p>Most teams treat optimization like a single knob: <em>”let’s switch to a cheaper model.”</em> That’s not optimization. That’s panic.</p><p>Real optimization is a <strong>stack</strong>— eight layers, each with its own techniques, costs, and payoffs. The discipline is to know which layer your problem actually lives in.</p><ul><li><strong>Layer 1 — UI / Interaction:</strong> Constrain input. Cut tokens.</li><li><strong>Layer 2 — Orchestration / Topology:</strong> Avoid over-cognition.</li><li><strong>Layer 3 — Context &amp; Memory:</strong> Budget tokens. Compress.</li><li><strong>Layer 4 — Data / Tool / MCP:</strong> Reuse data. Truncate outputs.</li><li><strong>Layer 5 — Model / Inference:</strong> Right model. Right job.</li><li><strong>Layer 6 — Caching &amp; Execution:</strong> Don’t pay twice.</li><li><strong>Layer 7 — Eval / Validation:</strong> Code before judge model.</li><li><strong>Layer 8 — Observability / FinOps:</strong> Tokens are dollars.</li></ul><h3>Layer 1 — UI / Interaction: “Every text box is a price on your inference bill”</h3><p>The earliest place to save money is the most ignored: the <strong>UI</strong>.</p><p>&quot;Get customer information for customer id 12345&quot; parsed by an LLM means 800 tokens of system prompt, a tool call, a synthesis step. 2.5 seconds. A few cents. A form with one Customer ID field and a Submit button does the same job in 40ms. Free.</p><p>Three patterns:</p><ul><li><strong>Constrained inputs over free text</strong> — dropdowns, date pickers, typed entity selectors. Every UI constraint is tokens you never pay for.</li><li><strong>Progressive disclosure</strong> — start with a form; fall through to chat only when the structured path can’t capture the intent.</li><li><strong>Hybrid UX</strong> — accept natural language but show extracted slots as editable fields: <em>“I understood: customer=12345, action=refund — correct?”</em> Cheaper, more accurate, more auditable.</li></ul><blockquote><strong><em>Real example.</em></strong><em> Payment refund flow has no chat box. It has </em>Order ID<em>, </em>Amount<em>, </em>Reason dropdown<em>. Three fields, one click, zero LLM tokens. Imagine the version where customers type “I want to refund the $50 order from last week because the shoes didn’t fit” — same outcome, fifty times the cost, occasional parsing errors.</em></blockquote><h3>Layer 2 — Orchestration / Agent Topology: “Code is cheaper than cognition”</h3><p>This is where most teams hemorrhage money without realizing it.</p><ul><li><strong>Default to deterministic routing.</strong> Use an LLM to decide the path only when rules genuinely cannot.</li><li><strong>Cap agent count.</strong> Seven agents is not 7x smarter than one. It’s 7x more expensive, 7x more places for hallucinated tool calls, and at least 21 inter-agent message edges with no eval coverage. <em>Most production agentic systems should have one to three agents. Maybe.</em></li><li><strong>Prefer “tools-on-one-agent” over “agent-per-tool.”</strong> A single agent with 8–12 well-scoped tools beats a constellation of specialists passing JSON around.</li><li><strong>Parallelize independent steps.</strong> Pure latency win at zero cost.</li><li><strong>Don’t send everything to the LLM.</strong> Math, validation, comparisons, lookups, conditional branching — use code.</li><li><strong>Short-circuit early.</strong> A cheap classifier at the front door routes 60% of traffic away from the expensive agent.</li></ul><blockquote><strong><em>Real example.</em></strong><em> Sending </em><em>2+2 to an LLM gets you </em><em>4 ninety-nine times out of a hundred. The hundredth time you&#39;ll see </em><em>22, or </em><em>4.0, or </em><em>four, and your downstream parser will choke. Anywhere correctness is binary, </em>use code<em>. The LLM orchestrates; the deterministic layer executes.</em></blockquote><h3>Layer 3 — Context &amp; Memory: Where teams quietly leak the most money</h3><p>The failure mode is always the same: someone keeps the whole conversation in context, plus all retrieved documents, plus a 4,000-token system prompt. By turn five, every call ships 30,000+ tokens of input. Multiply by traffic.</p><p>What good looks like:</p><ul><li><strong>Hard token budgets per turn.</strong> Truncate or summarize what exceeds them.</li><li><strong>Sliding window + rolling summary.</strong> Last N turns verbatim; older turns into a single paragraph.</li><li><strong>Selective memory.</strong> Tag what’s <em>durable</em> (preferences, decisions, IDs) vs. <em>ephemeral</em> (small talk, clarifications).</li><li><strong>Don’t dump full documents.</strong> Retrieve → re-rank → <em>compress</em>. Send the 500 tokens that matter, not the 50,000 the model could tolerate.</li><li><strong>Audit your system prompt quarterly.</strong> A 4,000-token system prompt on every call is the single biggest hidden cost.</li></ul><p><em>If your input token count grows monotonically with conversation length, you have an architecture problem, not a model problem.</em></p><h3>Layer 4 — Data / Tool / MCP: “Use what you already have”</h3><p>Three traps to avoid.</p><ul><li><strong>Don’t build a separate “AI data layer.”</strong> Use existing data platforms with zero-copy access. Your warehouse, APIs, and operational stores already exist. (More on this in <em>The Unified Data Layer for AI/GenAI Products</em>.<a href="https://medium.com/@ramakrishna.sanikommu/the-unified-data-layer-for-ai-genai-products-81f73e06c074">https://medium.com/@ramakrishna.sanikommu/the-unified-data-layer-for-ai-genai-products-81f73e06c074</a>)</li><li><strong>Tool granularity matters more than most people realize</strong>. Make tools too broad (do_everything()) and the LLM loses the ability to compose them. Make them too narrow (50 micro-tools) and selection accuracy falls apart. The sweet spot sits at 5–15 well-scoped tools per agent.</li><li><strong>Truncate tool outputs.</strong> A tool returning 20KB of JSON becomes 5,000+ tokens of input on the <em>next</em> call — and you’ll pay for that JSON over and over as it sits in history. Project and filter at the tool layer.</li><li><strong>MCP vs. plain tools.</strong> MCP is for cross-application standardization and reusability. For one internal API used by one agent, a plain tool is faster to build and cheaper to maintain. Don’t adopt MCP just because it’s trendy.</li></ul><blockquote><strong><em>Real example.</em></strong><em> A Salesforce </em><em>Account object has 100+ fields. Your agent probably needs three: </em><em>name, </em><em>email, </em><em>last_purchase_date. Return three. Every time you return the full object you&#39;re paying for 97 fields the model will never use — once on the way in, and then forever in conversation history.</em></blockquote><h3>Layer 5 — Model / Inference: Right model for the right job</h3><p>The single highest-leverage layer for cost, and the one teams over-spend on by reflex.</p><p>Every major provider ships three clear tiers:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lENKSgIhk3wfVjnL2ImeZg.png" /></figure><p>The cost gap from Frontier to Featherweight is typically <strong>10–30x</strong> per token. The capability gap for the tasks most agents actually do — classification, extraction, summarization, tool selection — is much smaller than that.</p><ul><li><strong>Route by task complexity, not team comfort.</strong> Resist “we just use the best model for everything.”</li><li><strong>Cascade.</strong> Try the cheap model first. Escalate only when confidence or eval signals demand it. <em>This pattern alone cuts costs 60–80%.</em></li><li><strong>max_tokens discipline.</strong> Output costs ~3–5x input. Cap output length per call. Highest-leverage knob in the system.</li><li><strong>Structured outputs</strong> (JSON / tool-call mode) cost less than free prose. No narrative filler.</li><li><strong>Prompt caching</strong> — all three providers expose it now, ~10x cheaper for cache hits. Stable system prompt? <em>Free money.</em></li><li><strong>Reasoning tokens</strong> are billed. Don’t enable extended thinking by default.</li><li><strong>Self-host only above ~$15–30k/month</strong> in API spend. Below that, GPUs + ops + on-call lose on TCO.</li></ul><blockquote><strong><em>Real example.</em></strong><em> Classifying emails as spam/not-spam on Haiku, Flash Lite, or GPT-5 nano costs roughly </em><strong><em>$0.0001 per email</em></strong><em> at production volumes. The same task on Opus or GPT-5 costs ~25x more for the same accuracy. At 10 million emails a month, that’s a $25,000 line item to delete by changing one model string.</em></blockquote><h3>Layer 6 — Caching &amp; Execution: The layer most blog posts skip</h3><p>You can’t out-engineer paying for the same answer twice. Most agentic systems do, every minute of every day.</p><ul><li><strong>Prompt caching (provider-native).</strong> Cache the static prefix of long prompts.</li><li><strong>Semantic caching.</strong> Embed queries. If a near-duplicate was answered recently, return the cached answer. Brilliant for FAQ-style traffic.</li><li><strong>Result caching for deterministic tool calls.</strong> Product catalog, account lookups, currency rates — cache aggressively.</li><li><strong>Idempotency keys.</strong> Prevent paid duplicate calls during retries.</li><li><strong>Streaming.</strong> Doesn’t reduce cost but cuts perceived latency. Free UX win.</li><li><strong>Batch inference.</strong> Async workloads (evals, backfills) get ~50% discounts via batch APIs at every major provider.</li></ul><blockquote><strong><em>Real example.</em></strong><em> An e-commerce bot answering </em>“what are your business hours?”<em> 1,000 times a day pays for one LLM call. The next 999 hit the semantic cache at ~50ms each. That single pattern, applied across a dozen FAQ-style intents, often takes a customer-service AI from a $20K/month bill to a $4K/month bill — without changing the model.</em></blockquote><h3>Layer 7 — Eval / Validation: “Don’t use LLM-as-Judge by default”</h3><p>LLM-as-Judge is the lazy answer. It’s also the most expensive. And it evaluates itself with the same kind of system it’s supposed to validate.</p><p>Build the ladder in this order. Escalate only when the rung above can’t do the job:</p><ol><li><strong>Code-based / programmatic checks.</strong> Schema validation, regex, type checks, range checks. Free, deterministic, instant.</li><li><strong>Rule-based validation.</strong> Business logic, policy compliance.</li><li><strong>Ground-truth comparison.</strong> Labeled data + exact-match, fuzzy-match, or embedding similarity.</li><li><strong>Human-in-the-loop.</strong> High-stakes outputs or building the gold set itself.</li><li><strong>LLM-as-Judge.</strong> Last resort. Use a <em>cheaper</em> model with rubric-driven prompts, calibrated against human labels.</li></ol><blockquote><strong><em>Real example.</em></strong><em> Generating SQL? Execute the query against a read-only replica. If it errors, you know it’s wrong. If it returns zero rows on a query that should return data, you know it’s wrong. No judge model required, no rubric to write, no eval cost. The database is the judge — and the database is free.</em></blockquote><h3>Layer 8 — Observability / FinOps: Treat tokens like cloud spend</h3><p>You wouldn’t run a fleet of EC2 instances without cost dashboards. Don’t run a fleet of agents without them.</p><p>Instrument:</p><ul><li><strong>Per-request token accounting</strong> — input, output, cached, reasoning — broken down by user, feature, application.</li><li><strong>Cost attribution.</strong> Without it, no serious conversation about ROI.</li><li><strong>Latency budgets per layer.</strong> Is the slow path retrieval, model, or tool? Optimization without observability is guessing.</li><li><strong>Drift detection.</strong> Providers update endpoints silently. Pin model versions. Watch eval scores over time.</li><li><strong>Feedback loop.</strong> Thumbs up/down piped back into the eval set. The system gets better without retraining.</li></ul><blockquote><strong><em>Real example.</em></strong><em> Tag every LLM call with </em><em>feature, </em><em>user_id, </em><em>application. A month from now, when finance asks &quot;why is our AI bill $40K,&quot; your answer is </em>&quot;$28K is the document-generation feature, mostly from three enterprise applications whose docs are 4x average length&quot;<em> — not </em>&quot;I don&#39;t know.&quot;<em> That conversation either gets a tier-pricing change shipped, or a token cap on those tenants, or both. Either way: ROI from the dashboard.</em></blockquote><h3>A Practical Order of Operations</h3><p>If this feels overwhelming, work through it in order. Each step is mostly independent, and each one pays for itself before you move on.</p><ol><li><strong>Audit agentic decisions.</strong> Review each feature and confirm agentic AI was the right choice. Kill what shouldn’t have been agentic.</li><li><strong>Set token budgets everywhere.</strong> Add max_tokens caps across the board. Quick win, often 20–30% cost reduction with zero engineering effort.</li><li><strong>Enable prompt caching on stable prefixes.</strong> Provider-native and near-zero risk.</li><li><strong>Introduce model tiering and cascading.</strong> The biggest single cost lever — expect 60–80% reduction on suitable traffic.</li><li><strong>Trim system prompts.</strong> Audit them carefully. Cap output sizes on tools.</li><li><strong>Add semantic caching</strong> for high-traffic, low-variance queries.</li><li><strong>Build the eval ladder</strong> before reaching for LLM-as-Judge anywhere.</li><li><strong>Wire up per-feature cost dashboards.</strong> Now ROI conversations can get serious.</li></ol><h3>Closing Thought</h3><blockquote><em>There is a comfortable lie in our industry right now: that “agentic” is automatically better. It isn’t. It’s a tool. Sometimes the right one, often overkill — and overkill in this domain isn’t free. It’s a recurring cost on a monthly invoice.</em></blockquote><blockquote><em>The next phase of Agentic AI isn’t about who can build the most impressive multi-agent demo. It’s about who can ship agents that are </em><strong><em>cheap per correct answer</em></strong><em>, fast under load, predictable in production, and boring in the best way.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/189/1*0T0M8A04awW2bxHWHoIeNA.png" /></figure><h3><em>Agentic AI without optimization is a demo. Agentic AI with optimization is a product.</em></h3><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36da6fe42990" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic Mesh: The New Kid on the AI Block — Innovation or Just More Buzzwords?]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/agentic-mesh-the-new-kid-on-the-ai-block-innovation-or-just-more-buzzwords-fd4efeb4f76b?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd4efeb4f76b</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-orchestration]]></category>
            <category><![CDATA[agentic-mesh]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Tue, 20 Jan 2026 02:27:30 GMT</pubDate>
            <atom:updated>2026-09-25T17:12:52.586Z</atom:updated>
            <content:encoded><![CDATA[<h3>Agentic Mesh: The New Kid on the AI Block — Innovation or Just More Buzzwords?</h3><p>We already have AI Automation and AI Orchestration. Do we really need another complicated diagram? Yes, and here is why.</p><p><strong>“Agentic Mesh.”</strong></p><p>It sounds cool. It sounds futuristic. It also sounds suspiciously like something made up by a marketing department at 3 AM to sell more cloud credits.</p><p>You might be asking: <em>“Wait, I already have AI automation scripts. I’m starting to build AI orchestration workflows. Why do I need a ‘Mesh’? Is this just the same old thing with a fancier hat?”</em></p><p>It’s a fair question. The AI space is drowning in terminology right now. But “Agentic Mesh” is actually one of the few new terms describing a necessary architectural shift for enterprises trying to scale AI beyond simple chatbots.</p><p>Let’s break down what it is, how it’s different from what you’re already doing, and why — as an engineering leader — you should care.</p><h3>The Evolution: From Interns to Ecosystems</h3><p>To understand Agentic Mesh, we need to look at the stepping stones that got us here. Let’s use a simple corporate analogy.</p><h4>Level 1: AI Automation (The Tired Intern)</h4><p>This is where most companies started in 2023.</p><ul><li><strong>What it is:</strong> A linear, single-task execution. You give an LLM an input, it performs one action, and gives you an output. It doesn’t “think” about what happens next.</li><li><strong>The Analogy:</strong> Think of a very literal intern. You say, “Read this 50-page PDF and list the three main risks.” They do exactly that and go back to sleep until you give the next command.</li><li><strong>The Data Reality:</strong> Highly efficient for high-volume, repeatable tasks.</li><li><em>Example:</em> A Python script that automatically categorizes incoming support tickets based on sentiment. It works 99% of the time but breaks if the ticket is complex.</li></ul><h4>Level 2: AI Orchestration (The Micromanager)</h4><p>This is where most sophisticated teams are right now (using tools like LangChain or LangGraph).</p><ul><li><strong>What it is:</strong> A managed workflow with a central “brain.” The orchestrator handles state, branching logic, and error handling. It says: “First do A, check the result of A, if it’s good, do B; otherwise, do C.”</li><li><strong>The Analogy:</strong> A Project Manager supervising a team. The PM knows the entire plan and tells everyone what to do and when to do it. If the “Research Agent” fails, the PM decides whether to retry or give up.</li><li><strong>The Data Reality:</strong> Great for complex, multi-step processes.</li><li><em>Example:</em> An advanced RAG pipeline. The orchestrator receives a query, decides <em>which</em> vector index to search, re-ranks the results, and then sends it to GPT-4 for an answer. It’s powerful, but it’s <strong>centralized</strong>.</li></ul><h3>Level 3: What is Agentic Mesh? (The Decentralized Ecosystem)</h3><p>Now we arrive at the new kid on the block.</p><p>If Orchestration is a centralized hierarchy, <strong>Agentic Mesh is a decentralized network of autonomous experts.</strong></p><p>It is an architectural pattern where multiple, specialized AI agents collaborate to achieve a goal <em>without</em> a single, monolithic controller telling them every move to make. It’s moving from a “command and control” structure to a “discovery and collaboration” structure.</p><p>Think of it like a microservices architecture, but for intelligent agents.</p><p>In an <strong>Agentic Mesh</strong>, specialized agents — like Salesforce, Legal, Finance, and Inventory — operate as independent experts that “discover” and collaborate with each other via a semantic control plane instead of following a rigid, central script. For example, when a Salesforce agent detects a deal, it broadcasts a request that the Legal agent autonomously fulfills by drafting a contract, while the Finance agent simultaneously triggers a credit check. This architecture is <strong>self-healing</strong>; if the Legal agent fails, the mesh automatically detects the outage and reroutes the task to a standby Compliance agent. By decentralizing intelligence, you eliminate the monolithic “orchestrator” bottleneck, ensuring the business process remains resilient even if individual components fail.</p><h4>Why do we need this shift?</h4><p>Because centralized orchestration doesn’t scale infinitely. When you try to build one giant graph to manage HR, Finance, Legal, and IT workflows across an entire enterprise, you end up with a brittle, unmaintainable monstrosity of code.</p><p>A mesh allows teams to build their own “Agentic Data Products” independently, and simply plug them into the corporate network.</p><h3>The Architecture Diagram</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LlI6j2RROX6OUEs40VPKmg.png" /><figcaption>Agentic Mesh</figcaption></figure><ul><li><strong>Autonomous Agents:</strong> Each domain (Salesforce, Finance, Legal) has its own agent with specialized tools and data access.</li><li><strong>Agentic Mesh Core (The Intelligent Fabric):</strong> This is the connective tissue. It handles <strong>discovery</strong> (how does the Finance agent know the Legal agent exists?) and <strong>semantic routing</strong> , Policy engine etc.</li></ul><h3>What Are the Benefits?</h3><p>Why should you invest engineering hours into this instead of just sticking with orchestration?</p><h4>1. Massive Scalability via Decoupling</h4><p>If the Legal team wants to upgrade their agent from GPT-4 to Claude 3 Opus, they just do it. They don’t need to ask the central orchestration team to rewrite the main application code. You can scale departments independently.</p><h4>2. Resilience and Self-Healing</h4><p>In a centralized system, if the orchestrator crashes, everything stops. In a mesh, if the Finance Agent goes offline, the other agents can continue working, or the mesh can queue requests until it comes back up. It’s far more robust for mission-critical enterprise apps.</p><h4>3. Cost and Latency Reduction (The Data Angle)</h4><p>This is huge. A smart mesh uses <strong>semantic routing</strong>.</p><ul><li><em>Without a mesh:</em> You might send a simple “reset my password” query to your most expensive, complex RAG pipeline.</li><li><em>With a mesh:</em> The router identifies it as a low-level IT task and routes it to a cheap, fast, specialized agent.</li><li><em>The Result:</em> We are seeing potential for <strong>30–40% reduction in wasted API token costs</strong> by ensuring the right model handles the right task.</li></ul><h3>The Toolkit: Frameworks Supporting Agentic Mesh</h3><p>This is the bleeding edge, so “push-button” mesh infrastructure is still emerging. However, these are the tools building the foundation:</p><ul><li><strong>LangGraph (by LangChain):</strong> Currently the best tool for building the <em>nodes</em> in the mesh. It allows you to build stateful, cyclic agents that can handle complex tasks autonomously.</li><li><strong>Microsoft AutoGen &amp; CrewAI:</strong> Excellent frameworks for defining how multiple agents converse and collaborate on tasks. They are great for simulating the mesh behavior.</li><li><strong>The Future:</strong> Expect to see infrastructure players (like cloud providers or specialized startups) launching “Control Planes” for AI agents — essentially Kubernetes or Istio for LLMs — handling the security, discovery, and traffic management between agents.</li></ul><h3>Where to Implement This?</h3><p>Don’t try to “mesh” your entire company tomorrow. Start where silos are currently causing the most pain.</p><p><strong>The best starting point: Cross-Functional Onboarding.</strong></p><p>Onboarding a new employee requires coordination between HR (paperwork), IT (laptop provisioning), and Finance (payroll setup).</p><p>Instead of one human chasing three departments, you build an “HR Agent,” an “IT Agent,” and a “Finance Agent,” connect them via a mesh, and let them coordinate the process autonomously. The human HR manager just monitors the final output.</p><h3>Final Verdict</h3><p>Agentic Mesh isn’t just a buzzword; it’s the inevitable architectural maturity of Enterprise AI. It moves us from building neat little AI toys to building scalable, resilient corporate nervous systems.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd4efeb4f76b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[One LLM Was Never the Endgame: The Inevitable Rise of AI Gateway Architecture]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/one-llm-was-never-the-endgame-the-inevitable-rise-of-ai-gateway-architecture-c6393ce4478f?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6393ce4478f</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-gateway]]></category>
            <category><![CDATA[google-gemini]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Mon, 15 Dec 2025 02:59:53 GMT</pubDate>
            <atom:updated>2026-09-25T17:13:18.662Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why Companies Use Multiple LLM Vendors</h3><p>Today, no single LLM is the best at everything. Different models shine in different areas — whether it’s deep reasoning, response validation, multimodal understanding, cost efficiency, or data privacy.</p><p>By using multiple LLM vendors, teams can:</p><ul><li><strong>Improve quality</strong> through cross-validation of responses.</li><li><strong>Optimize costs</strong> by routing tasks to the most efficient model.</li><li><strong>Increase reliability</strong> by failing over when one provider is unavailable.</li><li><strong>Avoid vendor lock-in</strong> and stay adaptable as models rapidly evolve.</li></ul><p>This multi-LLM approach keeps AI systems resilient, flexible, and future-proof.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JlkeFvd-Qhj3PEZIgkajlA.png" /></figure><h3>The Challenge of Multiple LLM Providers</h3><p>The AI landscape now offers a plethora of large language models (LLMs) from providers like OpenAI (GPT series), Anthropic (Claude), Google (Gemini), Meta (LLaMA), Hugging Face, and many others. Each provider offers different models and versions, each with its own API interface and quirks. For a company trying to mix and match models (for example, using OpenAI’s GPT-4 for one task and validating the result with Anthropic’s Claude), the integration complexity grows quickly.</p><h3>key challenges include:</h3><ul><li><strong>Divergent APIs and SDKs:</strong> Every vendor has a unique API format and authentication method. If you integrate multiple LLMs directly, your code becomes tightly coupled to each provider’s API.</li><li><strong>Multiple Credentials and Configurations:</strong> Each service requires managing separate API keys, endpoints, and monitoring dashboards.</li><li><strong>No Unified Monitoring or Cost Control:</strong> With siloed integrations, it’s hard to get a unified view of usage, latency, or costs.</li><li><strong>Lack of Redundancy and Fallbacks:</strong> If you rely on one provider per task, an outage or rate limit hit on that provider can bring your application down.</li><li><strong>Integration Overhead and Maintenance:</strong> Orchestrating multiple vendors for one product feature (such as transcription + LLM analysis) is fragile and time-consuming.</li></ul><p>In short, using multiple AI models in production without a unifying layer can lead to brittle code, higher maintenance costs, and a slower ability to adopt the latest models. This complexity is driving the need for a better solution: an <strong>AI Gateway</strong>.</p><h3>AI Gateway — A Unified Solution</h3><p>An AI Gateway (also called an LLM Gateway or LLM router) is a middleware layer that abstracts away the complexity of dealing with many different model APIs. Instead of your application integrating separately with each vendor’s API, you integrate once with the gateway. The gateway then handles routing your requests to the appropriate model/provider and returning the results in a consistent format. This unified interface decouples your code from any single vendor.</p><p><strong>What problems does this solve?</strong></p><ul><li><strong>Seamless Provider Flexibility:</strong> You can switch out models or use multiple providers without rewriting your application code. For example, if you want to test a new model like Google’s Gemini instead of OpenAI’s GPT, you just change a parameter or configuration — you don’t have to rebuild your integration from scratch.</li><li><strong>Unified API &amp; SDK:</strong> Many gateways expose a single API endpoint (often compatible with OpenAI’s API schema or a standard REST/GraphQL interface). Your app makes requests to this one endpoint, and the gateway translates them to the proper calls for OpenAI, Anthropic, Google, etc.</li><li><strong>Centralized Monitoring and Logging:</strong> The gateway can log all requests and responses across providers in one place. You gain real-time insights into usage, latency, errors, and costs for every model call.</li><li><strong>Intelligent Routing &amp; Load Balancing:</strong> Because the gateway sits in the middle, it can implement smart routing policies. It might automatically direct requests to the fastest or cheapest provider available. You can also configure rules (e.g., send 80% of traffic to a primary model and 20% to a cheaper backup, or route EU users to an EU-based model for compliance).</li><li><strong>Automatic Failovers:</strong> If a model call fails due to an API outage or rate limit, the gateway can catch the error and retry the request on an alternate model. For instance, if GPT-4 times out, the gateway can automatically invoke Claude.</li><li><strong>Unified Security and Governance:</strong> You can enforce organizational policies (allowed models, data filtering, rate limits per user) centrally in the gateway rather than implementing checks across many integrations.</li><li><strong>Future-Proofing:</strong> When a new model or provider launches, you can adopt it quickly by adding it to the gateway configuration without touching your application code.</li></ul><h3>How AI Gateways Work</h3><p>At a high level, using an AI gateway looks like this: your application sends a request to the gateway (usually an HTTPS call to a gateway URL) with an indication of which model you need. The gateway forwards that request to the provider, handles the response, and sends the answer back to you.</p><p><strong>The typical workflow:</strong></p><ol><li><strong>Unified Request Format:</strong> The gateway defines a single API schema. Many gateways mimic the OpenAI API format (JSON body with fields like model, messages, max_tokens), allowing you to use one format regardless of the underlying model.</li><li><strong>Routing Logic:</strong> Inside the gateway, the request is parsed and routed to a specific backend.</li><li><strong>Provider Connectors:</strong> The gateway holds the API keys and endpoint URLs for OpenAI, Anthropic, etc. It formats the call according to the specific provider’s API specs.</li><li><strong>Response Handling:</strong> The response from the model provider is received by the gateway, standardized, and relayed back to your application.</li><li><strong>Logging and Observability:</strong> As each request flows through, the gateway logs metadata — model used, latency, token count, and errors — aggregating this data into dashboards.</li></ol><p><strong>A Note on Orchestration vs. Gateways</strong> It is important to distinguish between <em>access</em> and <em>orchestration</em>. An AI Gateway simplifies the connection to models, but it does not replace the logic needed for complex workflows.</p><p>If your application requires sequential chains (e.g., “Step 1: Summarize text with Claude; Step 2: Extract entities with GPT-4”), you still manage that sequence in your application code or via an orchestration framework. However, the Gateway makes this easier: instead of managing two different SDKs for that chain, your application simply makes two standardized calls to the Gateway.</p><p>While some advanced enterprise platforms are beginning to merge these concepts — offering features for multi-step agent workflows and tool usage directly in the gateway — the core value remains the same: a single, unified interface to interact with any model, anywhere.</p><h3>Key Features and Capabilities</h3><p>To summarize, the core technical capabilities that an AI Gateway provides include:</p><ul><li><strong>Unified API Interface:</strong> A single, standardized REST or GraphQL endpoint for all models. This allows you to interact with any LLM using one consistent request schema, regardless of the underlying provider.</li><li><strong>Multi-Provider Support:</strong> Pre-built connectors for a wide range of providers (e.g., OpenAI, Anthropic, AWS Bedrock) and model types, allowing for “plug-and-play” integration.</li><li><strong>Intelligent Routing &amp; Load Balancing:</strong> The ability to dynamically route requests based on real-time criteria such as latency, cost, geographic region, model health, or custom business weights.</li><li><strong>Automatic Failover:</strong> Configurable fallback logic to ensure high availability. If a primary model is unavailable or returns an error, the gateway automatically retries the request on a backup model without the user noticing.</li><li><strong>Caching and Rate Limiting:</strong> Built-in caching layers to store recent responses (reducing costs and latency) and rate limiting to control throughput and prevent budget overruns.</li><li><strong>Observability and Logging:</strong> Comprehensive visibility into system performance. This typically includes detailed logs for every request, tracking metrics like token usage, response time, success/error rates, and full trace data for debugging.</li><li><strong>Security &amp; Governance Controls:</strong> Essential features for enterprise use, including data encryption in transit, SOC2/GDPR compliance, Role-Based Access Control (RBAC), centralized API key management, and PII redaction/content filtering to enforce AI safety policies.</li></ul><h3>Notable AI Gateway Solutions</h3><p>As interest in multi-LLM orchestration has grown, several tools and platforms have emerged:</p><ul><li><strong>Helicone AI Gateway:</strong> An open-source gateway built with Rust, focused on high performance and robust observability. It offers health-aware load balancing and seamless monitoring.</li><li><strong>OpenRouter:</strong> A managed service providing easy access to many popular models through a single API key, emphasizing simplicity for developers.</li><li><strong>Portkey AI Gateway:</strong> An open-source gateway targeting enterprise needs. It supports an extensive list of models, including vision and audio models.</li><li><strong>LiteLLM:</strong> A popular open-source gateway known for flexibility, supporting 100+ models across many providers.</li><li><strong>Unify AI:</strong> A gateway that provides a unified API for various models with features like pass-through billing to manage costs in one place.</li><li><strong>Databricks Mosaic AI Gateway:</strong> Part of the Databricks Lakehouse platform, this is a governance-focused solution ideal for enterprises needing to enforce security and compliance across multiple teams.</li><li><strong>AWS Bedrock:</strong> Acts as a managed gateway layer to foundation models (Anthropic, Meta, Amazon Titan), providing single API access and IAM-based security.</li><li><strong>Azure AI Gateway / Azure OpenAI Service:</strong> Microsoft’s enterprise layer for OpenAI and other models, offering VNet isolation and RBAC.</li><li><strong>Google Vertex AI Model Garden:</strong> A gateway-style interface for Gemini and open-source models with unified APIs and tight GCP integration.</li><li><strong>Others:</strong> TrueFoundry, IBM watsonx Orchestrate, Kong AI Gateway, etc.</li></ul><h3>Conclusion</h3><p>By implementing a gateway, organizations can stay agile in the fast-moving LLM landscape — adopting new models or switching providers without rebuilding everything. They can achieve higher reliability with failovers, gain visibility through unified logs, and enforce safety with centralized policies.</p><p>The AI field is evolving rapidly, and no single model will stay the “best” forever. With an AI gateway in place, you gain the freedom to leverage the best of multiple AI models without drowning in integration complexity. It becomes the linchpin of a scalable, maintainable, and adaptable AI stack.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6393ce4478f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI Orchestration]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/agentic-ai-orchestration-fb4b8e8be8a7?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/fb4b8e8be8a7</guid>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Sun, 12 Oct 2025 17:59:18 GMT</pubDate>
            <atom:updated>2026-09-25T17:13:57.854Z</atom:updated>
            <content:encoded><![CDATA[<p>Agentic AI orchestration is an emerging approach to automation where <strong>AI agents</strong> coordinate tasks across various tools and systems to achieve higher-level goals. Instead of just running isolated scripts or static workflows, these AI agents can make decisions, call APIs, and adapt their actions based on context — all with minimal human intervention. This enables a shift from simply automating tasks to <strong>orchestrating outcomes</strong>. In this guide, we’ll explain what agentic AI orchestration means, how it works (with an easy-to-follow architecture), and compare popular tools for implementing it. The aim is to be beginner-friendly while still diving into technical details for those who want them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RctHSjDhK5awXSZw" /><figcaption>Agentic AI Orchestration</figcaption></figure><h3>What is Agentic AI Orchestration?</h3><p><strong>Agentic AI orchestration</strong> refers to the coordination of multiple AI agents working together toward shared goals, often across different software tools, workflows, or departments. In simpler terms, you have several AI-driven components (the “agents”), each capable of certain tasks, and you manage them in a unified way so they collaborate and hand off tasks intelligently. These agents operate with a degree of autonomy — they don’t just follow a fixed script, they can respond to real-time inputs, make context-aware decisions, and adjust their actions as they go.</p><p>In essence, agentic AI orchestration is the <strong>coordination layer</strong> that manages how different AI tools, agents, and automations work in concert. It determines which tool should act, when it should act, and how information flows between systems to achieve a desired outcome. This concept is sometimes also just called <em>AI orchestration</em>. The key idea is that AI systems (like chatbots, AI analytic tools, etc.) no longer operate in isolation — they are <strong>linked together, share data, and trigger actions</strong> for each other in a seamless workflow</p><h3>How Agentic AI Orchestration Helps (Benefits and Impact)</h3><p>Why go through the trouble of orchestrating AI agents? The approach brings several compelling benefits for businesses and teams:</p><ul><li><strong>Higher efficiency and speed:</strong> Orchestrated AI agents can handle multi-step processes in seconds that might take humans hours. By automating not just individual tasks but entire processes</li><li><strong>Improved decision-making:</strong> When AI agents collaborate, they can share information and provide a more comprehensive analysis. One agent’s output can become another’s input, yielding insights that a single tool might miss.</li><li><strong>End-to-end automation without silos:</strong> Traditional automation often leaves you with disjointed “islands” of automation (your marketing AI doesn’t know what your support AI is doing, etc.). Orchestration <em>connects these islands</em>, breaking down information silos.</li><li><strong>Scalability and adaptability:</strong> An orchestrated setup is easier to scale. As your needs grow, you can add new AI capabilities into the existing orchestration framework without reinventing the wheel.</li><li><strong>integration already exists</strong> — you’re not building every automation from scratch each time. Moreover, agents can learn from each interaction and adapt.</li><li><strong>Reduced complexity (despite involving many tools):</strong> It sounds counterintuitive, but bringing multiple AI and software tools into one orchestrated system can actually <strong>simplify</strong> management.</li><li><strong>Better governance and control:</strong> When AI processes are centrally orchestrated, it’s easier to enforce rules, security, and compliance.</li></ul><h3>Key Components</h3><ol><li><strong>Trigger/Input</strong> — User action, event, or schedule that initiates a workflow.</li><li><strong>AI Orchestrator (Agent Controller)</strong> — The decision brain that plans and routes tasks.</li><li><strong>LLMs/AI Models</strong> — Handle reasoning, text understanding, and content generation.</li><li><strong>Connectors/Tools</strong> — Integrations with apps (Salesforce, Snowflake, Azure, etc.).</li><li><strong>Memory/Context Store</strong> — Retains state or conversation context for continuity.</li><li><strong>Output/Action Layer</strong> — Executes results (e.g., updates records, sends alerts).</li></ol><h3>Popular Tools</h3><ul><li><strong>OpenAI Agent Builder:</strong> Visual, LLM-native to OpenAI; very simple to start; best for straightforward flows.</li><li><strong>Zapier:</strong> No-code; huge app ecosystem; great for quick AI+app protos.</li><li><strong>Workato:</strong> Enterprise iPaaS; strong governance, security, and multi-LLM readiness.</li><li><strong>n8n:</strong> Open-source, self-hostable; flexible and highly customizable.</li><li><strong>Make (Integromat):</strong> Visual automation with powerful branching and data handling.</li><li><strong>CrewAI, Lang Graph, LangChain:</strong> Developer frameworks; maximum code-level control and customization.</li><li>Many more…..</li></ul><h4>When to Use OpenAI Agent Builder vs. Other Tools</h4><p>Use <strong>OpenAI Agent Builder</strong> when your workflow is built entirely around OpenAI models and you need a quick, visual way to create AI agents without coding. It’s perfect for small projects, prototypes, or internal assistants that use GPT-4 and simple integrations. It’s fast, intuitive, and ideal for <strong>OpenAI-only, low-complexity use cases</strong>.</p><p>Choose <strong>Workato, Zapier, n8n, Make, CrewAI, Lang Graph or LangChain</strong> when your orchestration involves <strong>multiple LLMs, complex logic, or enterprise systems</strong> like Salesforce, Snowflake, or Azure. These tools provide richer control — error handling, branching, governance, on-prem deployment, and multi-model flexibility — that Agent Builder doesn’t yet offer.</p><p>In short, <strong>Agent Builder is great for simplicity and speed</strong>, but <strong>other platforms excel when you need scale, customization, and multi-vendor AI orchestration.</strong></p><h3>Conclusion</h3><p>Agentic AI orchestration bridges AI reasoning with enterprise automation — turning standalone models into coordinated, goal-driven systems.<br> <strong>OpenAI Agent Builder</strong> simplifies agent creation within the OpenAI ecosystem, but <strong>it won’t replace other orchestration tools.</strong><br> When your AI solution spans <strong>multiple LLM vendors, complex system integrations, or heavy customization</strong>, tools like <strong>CrewAI, Lang Graph</strong>, <strong>Workato, n8n, Zapier, Make, or LangChain</strong> are the better choice.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fb4b8e8be8a7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Engineering ( RAG 2.0 ) : The Next Chapter in GenAI]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/context-engineering-rag-2-0-the-next-chapter-in-genai-4e53c0382bf4?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e53c0382bf4</guid>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Mon, 25 Aug 2025 02:43:22 GMT</pubDate>
            <atom:updated>2026-09-25T17:17:04.654Z</atom:updated>
            <content:encoded><![CDATA[<h3>Context Engineering ( RAG 2.0 ) : The Next Chapter in GenAI</h3><p><strong>Q: Is Retrieval-Augmented Generation (RAG) dead?</strong><br><strong>A:</strong> <strong>No </strong>— RAG isn’t dead at all. It’s evolving, and a new discipline called <strong>context engineering</strong> is taking the spotlight. In fact, context engineering builds on RAG’s foundations to create smarter AI systems. This blog will explore what context engineering means in GenAI, how it differs from a basic RAG pipeline, and why some people claim “RAG is dead” — and why that’s an oversimplification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FMO-vaNGMK0KS8R4LRtFxw.png" /></figure><h3>What Is Context Engineering in GenAI?</h3><p><strong>Context engineering</strong> is the practice of designing systems that decide <em>what information an AI model sees</em> before it generates a response. Instead of relying on a single one-shot prompt, context engineering involves assembling all relevant background — instructions, data, history, tools — into the model’s input so it can produce an informed answer. In other words, providing the right <strong>information ecosystem</strong> to the model is often more important than tweaking the model itself</p><p>In a context-engineered system, when a user query comes in, the system pulls together multiple sources of information within the model’s context window</p><ul><li><strong>System Instructions:</strong> e.g. a predefined role or persona for the model, and rules it should follow (the “system prompt”)</li><li><strong>User Query:</strong> the immediate question or task from the user (just like a normal prompt).</li><li><strong>Conversation History:</strong> the dialogue so far, or short-term memory of previous interactions (so the AI remembers context from earlier in the conversation).</li><li><strong>Long-Term Memory:</strong> stored facts or user preferences from past sessions (could be fetched from a database or vector memory)</li><li><strong>Retrieved Knowledge:</strong> documents or data fetched via search — this is essentially the R in RAG, pulling in external info relevant to the query</li><li><strong>Tools and API Outputs:</strong> results from external tool calls (e.g. a calculator, a web search, or database query) and definitions of available tools</li><li><strong>Output Constraints or Formats:</strong> any required output schema or formatting instructions (for example, “answer in JSON” or a template).</li></ul><p>All these pieces are <strong>carefully curated and assembled</strong> into the prompt context given to the LLM. The goal is to fill the model’s input with <em>just the right information</em> it needs to complete the task, and nothing extraneous.</p><p>Crucially, context engineering is a <strong>dynamic, system-level discipline</strong> rather than a one-off prompt hack. It treats the AI application as a <em>stateful, multi-turn system</em> rather than a single-turn prompt. Whereas classic prompt engineering might involve cleverly wording a single query for a chatbot, context engineering architects an entire contextual <em>world</em> for the AI to operate in. It requires managing conversation state over time, deciding what old information to keep or forget, and continuously fetching or updating context as needed. The end result is an AI that feels more <em>aware</em>: it can remember past interactions, leverage outside knowledge, and use tools, all within one coherent response.</p><h3>Context Engineering vs. RAG Pipelines</h3><p>Context engineering and RAG are closely related — in fact, you can think of RAG as one early form of context engineering — but there are important differences in scope and approach. RAG (Retrieval-Augmented Generation) traditionally refers to a pipeline where a user query is augmented with retrieved documents before feeding into an LLM. A <em>naïve RAG pipeline</em> typically works like this: you embed the user’s question as a vector, use it to <strong>retrieve</strong> relevant text chunks from a vector database, and then prepend those chunks to the prompt so the LLM can use them as context. This significantly improves the model’s ability to answer questions about external knowledge, compared to using the base LLM alone.</p><p>However, RAG in practice often focuses mainly on <strong>documents</strong>. A basic RAG system might ignore other context signals like the conversation history or tool usage. In contrast, context engineering is <strong>broader</strong>: it’s about assembling <em>all forms of context</em>, not just retrieved text. As one guide puts it, context engineering “gathers relevant details from multiple sources and organize them within the model’s context window,” including dialogue history, user data, retrieved docs, tools, etc. In other words, RAG’s document retrieval is just one component of context engineering.</p><p>Another difference is that context engineering emphasizes <strong>optimization and structure</strong>. It’s not ad-hoc prompting; it’s a systematic process to maximize the task-relevant information under the constraints of the model’s context length. Context engineering treats the context assembly like an engineering problem: which pieces of knowledge to include, how to compress or summarize them if needed, and how to maintain state over time. RAG pipelines already do some of this (e.g. chunking and ranking documents by relevance to fit into token limits), but context engineering extends these ideas to <em>all</em> information types. It may involve maintaining <strong>memory buffers</strong>, using algorithms to decide which facts to carry forward, or partitioning the context into sections (instructions vs. knowledge vs. history) for clarity.</p><p>Importantly, context engineering doesn’t replace prompt engineering so much as <strong>encompass</strong> it. You still need well-crafted prompts (for example, good instructions and formatting) inside your context. The difference is those prompts operate within a rich, managed context rather than starting from scratch each time. Think of prompt engineering as writing a single scene, whereas context engineering is directing an entire movie — setting the scene, recalling previous scenes, bringing in props (data) as needed, and so on.</p><h3>RAG Is Dead? — The Long Context Debate</h3><p>Why are some voices in the AI community saying <em>“RAG is dead”</em> lately? This hype stems from the rapid growth of LLM <strong>context window sizes</strong>. New LLM models can accept inputs of tens or even hundreds of thousands of tokens. There’s talk of models handling <em>million</em>-token contexts in the near future. If a model can theoretically read an entire novel or a whole enterprise knowledge base in one go, the thinking goes, why bother with the complexity of retrieval pipelines? Why not just stuff everything into the prompt?</p><p>It’s true that larger context windows are a game-changer — In demos, they handle <strong>rare-item searches</strong> surprisingly well. But in practice, unlimited context is a <strong>mirage</strong>. Feeding a huge amount of data into an LLM has serious <em>trade-offs</em>. <em>“context is not memory — it’s compute”</em>. Even if your model supports a million tokens, loading it up fully is extremely slow and expensive. RAG was invented for a reason: to avoid shoving the entire data warehouse into every prompt, and instead fetch only the pieces that matter.</p><p>Beyond cost and latency, there’s the issue of model <em>effectiveness</em>. Large context windows do not equal perfect long-term memory. In fact, piling in too much information can confuse the model or degrade its performance. In other words, after a certain point, extra context becomes noise. <em>Quality</em> of context matters more than quantity.</p><p>All this is to say: <strong>RAG is not dead — it’s evolving.</strong> we need smart strategies to decide <em>which</em> snippets of knowledge, from potentially millions of tokens worth of data, should be inserted into the prompt. Instead of “RAG is dead,” a more accurate statement is: <strong><em>RAG is turning into context engineering</em></strong><em>.</em></p><h3>RAG vs. Context Engineering — A Side-by-Side View</h3><p>To make the differences more concrete, let’s compare a <strong>naïve RAG pipeline</strong> to a <strong>context-engineered pipeline</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aE3FcyKKGpCrMPNeCwebWw.png" /></figure><p><em>Comparison of a basic RAG pipeline vs. a context engineering pipeline . A RAG pipeline augments the user’s query with retrieved documents. A context-engineered system goes further — incorporating not just retrieved text, but also instructions, memory (history and long-term), and tool outputs into the LLM’s context.</em></p><h3>Conclusion</h3><p>Far from being “dead,” RAG has given rise to a richer paradigm. Context engineering is about <strong>holistic context management</strong> — figuring out the right mix of background knowledge, memory, and guidance to feed an LLM so it performs at its best. As models get more powerful and context windows grow, blindly throwing data at them is not a sustainable strategy. We need thoughtful curation of context, and that’s exactly what context engineering offers.</p><p>RAG is not dead; it’s evolving into <strong>context engineering</strong>. By orchestrating instructions, memory, retrieval, and tools, we can give our AI models a complete and curated view of the world they operate in. That means more accurate answers, fewer hallucinations, and AI assistants that remember and adapt. In this next phase of GenAI, success will belong to those who engineer not just prompts, but entire <strong>contexts</strong> for their AI.</p><p>— -</p><p>Disclaimer: Views expressed here are my own and don’t reflect the position of my employer or any organization I’m affiliated with. Architectures, diagrams, and code shown are for illustrative purposes — test thoroughly before using in production. Vendor and product mentions are for technical context only, not an endorsement.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e53c0382bf4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Unified Data Layer for AI/GenAI Products]]></title>
            <link>https://medium.com/@ramakrishna.sanikommu/the-unified-data-layer-for-ai-genai-products-81f73e06c074?source=rss-c317b65ef00a------2</link>
            <guid isPermaLink="false">https://medium.com/p/81f73e06c074</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-lake]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Ramakrishna Sanikommu]]></dc:creator>
            <pubDate>Mon, 11 Aug 2025 16:50:45 GMT</pubDate>
            <atom:updated>2025-08-11T16:50:45.397Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A practical, production‑ready guide for LLM chatbots and recommendation systems</em></p><h3>Summary</h3><ul><li><strong><em>Do LLMs need a database?</em></strong><em> No — foundation models are pre‑trained on massive corpora; they don’t persist your proprietary data, real‑time transactions, or per‑user state.</em></li><li><strong><em>Do AI/GenAI products need databases?</em></strong><em> Yes — production apps require multiple stores (lake/lakehouse, vector, document, graph, cache, RDBMS, search, feature store) for correctness, speed, memory, governance, and cost.</em></li><li>LLMs don’t replace databases; real AI apps need a <strong>unified data layer</strong> that blends multiple stores for context, memory, speed, and governance.</li><li>Core building blocks: <strong>Data Lake/Lakehouse</strong> (source of truth), <strong>Vector DB</strong> (semantic retrieval), <strong>In‑Memory Cache</strong> (session memory, low latency), <strong>Document DB</strong> (chat history &amp; semi‑structured content), <strong>Graph DB</strong> (relationships &amp; reasoning), plus <strong>RDBMS, Search Indexes, and Feature Store</strong>.</li><li>Wire them together via <strong>pipelines (batch + streaming)</strong>, an <strong>API/microservices layer</strong>, and <strong>observability, governance, and security</strong>.</li><li>Start small with clear SLAs, iterate with metrics (latency, recall@k, cost), and scale components as usage grows.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ZbognP3ZI3kAX8-kV9Kmg.png" /></figure><h3>Component Guide (What, Why, When)</h3><h3>1) Data Lake / Lakehouse — Source of Truth</h3><p><strong>Role</strong>: Land all raw data (structured, semi, unstructured). Curate and serve refined datasets with ACID tables and governance.</p><p><strong>Use when</strong>: You need inexpensive, durable storage; reproducible pipelines; training/analytics at scale; a single place to build features/embeddings.</p><p><strong>Common picks</strong>: S3/ADLS/GCS + Delta/Iceberg/Hudi; processing via Spark/Flink/DBT; Lakehouse platforms (e.g., Databricks,BigQuery/BigLake, Synapse).</p><h3>2) Vector Database — Semantic Retrieval (RAG)</h3><p><strong>Role</strong>: Store high‑dimensional embeddings and serve k‑NN similarity queries; supports filters/metadata and hybrid keyword+vector search.</p><p><strong>Use when</strong>: You need grounding for LLMs (RAG), semantically similar items, multimodal search.</p><p><strong>Common picks</strong>: Pinecone, Weaviate, Milvus/Qdrant, OpenSearch k‑NN, pgvector (Postgres/AlloyDB/Aurora), Azure AI Search vectors.</p><p><strong>Key settings</strong>: index type (HNSW/IVF), dimension, metric (cosine/IP), filters, reranking, chunking strategy.</p><h3>3) In‑Memory DB (Cache) — Session</h3><p><strong>Role</strong>: Millisecond reads/writes for chat/session state, throttling, and response caching.</p><p><strong>Use when</strong>: You must rebuild prompts from recent history fast; cache expensive calls; store online features.</p><p><strong>Common picks</strong>: Redis (ElastiCache/MemoryDB, Azure Cache, Memorystore), Memcached.</p><h3>4) Document DB — Chat History,Episodic &amp; Semi‑Structured Content</h3><p><strong>Role</strong>: Persist long‑term chat transcripts, tickets, FAQs, and flexible JSON records with evolving schemas.</p><p><strong>Use when</strong>: You need to query per user/session, attach metadata, or support variable content structures.</p><p><strong>Common picks</strong>: MongoDB/Cosmos DB API, DynamoDB, Firestore, Couchbase.</p><h3>5) Graph DB / Knowledge Graph — Semantic, Relationships &amp; Reasoning</h3><p><strong>Role</strong>: Model entities/edges for curated facts and multi‑hop reasoning (e.g., product hierarchies, ontologies, user–item graphs).</p><p><strong>Use when</strong>: You need explainable retrieval, factual grounding, or graph algorithms in recommendations.</p><p><strong>Common picks</strong>: Neo4j (Aura), Amazon Neptune (Gremlin/RDF), Cosmos DB (Gremlin), TigerGraph, JanusGraph.</p><h3>Memory Types: Session vs Episodic vs Semantic</h3><p>A key aspect of GenAI applications is how they manage <strong>memory</strong> — i.e., how the AI remembers information over different timescales. This architecture distinguishes three types of memory, each handled by different components :</p><ul><li><strong>Session Memory (Short-term, ephemeral):</strong> This is the fleeting memory of the current interaction or session — for example, the last few user messages in a live chat session. It’s stored in the <strong>in-memory cache (Redis)</strong> for quick access. Session memory is fast but volatile (not permanently stored). The system checks this first for recent context (since reading from memory is fastest). If the AI needs to recall what was just said a moment ago or a recently looked-up result, it’s pulled from session memory.</li><li><strong>Episodic Memory (Longer-term per user/episode):</strong> This refers to persistent memory of past events or interactions, usually scoped to one user or one conversation across sessions. It is stored in the <strong>document database</strong> (and related indexes). For instance, a user’s past conversations or a history of their actions can be kept in a document store as an “episode” record. When needed, the AI (via the profile or retrieval services) can fetch these records and even vectorize them to find relevant parts . Episodic memory allows the AI to maintain continuity and personalization over time, beyond just the current session.</li><li><strong>Semantic Memory (Knowledge, generalized context):</strong> This is the broad knowledge the AI can draw upon, not tied to one user. It’s captured in the <strong>vector database and knowledge stores (graph and search index)</strong>. The vector DB gives the AI a way to recall information by meaning — for example, find documents related to a new query even if the exact words differ. The knowledge graph and search index provide structured and unstructured knowledge, respectively, that the AI can use to ground its responses. Together, these represent the AI’s understanding of the world or the domain (facts, concepts, documents) — essentially the AI’s <strong>semantic long-term memory</strong>.</li></ul><h3>6) Other Critical Stores</h3><p><strong>RDBMS</strong>: Authoritative transactional data (users, orders, inventory). Runtime APIs query this for real‑time truth.</p><p><strong>Search Index</strong>: BM25/keyword search, facets, filters; great for exact IDs, error codes, logs; often combined with vector search.</p><p><strong>Feature Store</strong>: Offline/online feature parity for ML; prevents training-serving skew; serves low‑latency features to models/agents.</p><p><strong>Time‑Series (optional)</strong>: Metrics/telemetry for IoT/ops signals feeding anomaly detection and feedback loops.</p><h3>Integration Patterns (How It Fits Together)</h3><h3>Dataflow: Batch + Streaming (Lambda/Kappa)</h3><ul><li><strong>Ingest</strong> → Lake (raw) → <strong>Curate</strong> (tables) → <strong>Produce</strong>: embeddings to Vector DB; features to Feature Store; entities/edges to Graph DB; docs to Doc DB.</li><li><strong>Streams</strong> (Kafka/Event Hubs/PubSub) update caches, online features, and near‑real‑time indices.</li></ul><h3>Application &amp; API Layer</h3><ul><li><strong>Retrieval Service</strong> (vector + hybrid search), <strong>Knowledge Service</strong> (graph queries), <strong>Profile Service</strong> (RDBMS/Doc DB), <strong>Recs Engine</strong> (features + similarity + business rules).</li><li><strong>LLM Orchestration</strong>: prompt templating, tools/agents, function calling, policy checks, reranking.</li></ul><h3>Observability, Governance, and Security</h3><ul><li><strong>Metrics</strong>: latency (p95), recall@k, hallucination rate, CTR, cost/QPS.</li><li><strong>Data governance</strong>: lineage/catalog, PII tags, retention/TTL, deletion propagation (lake → indices → logs).</li><li><strong>Security</strong>: authN/Z (OIDC), network and key mgmt, per‑store IAM, row/column‑level controls.</li></ul><h3>Cloud Reference Mappings (Examples)</h3><ul><li><strong>AWS</strong>: S3 + Glue/Athena/Redshift; OpenSearch k‑NN or Aurora pgvector; ElastiCache/MemoryDB; DynamoDB/DocumentDB; Neptune; SageMaker/Feature Store.</li><li><strong>Azure</strong>: ADLS + Synapse/Delta; Azure AI Search (hybrid+vector); Azure Cache for Redis; Cosmos DB; Cosmos Gremlin; Azure ML (Feature Store emerging).</li><li><strong>GCP</strong>: GCS + BigQuery/BigLake; Vertex Matching Engine or AlloyDB pgvector; Memorystore; Firestore; Neo4j Aura; Vertex AI Feature Store.</li></ul><h3>Performance &amp; Cost Tips</h3><ul><li>Right‑size chunking (token‑aware) and choose <strong>cosine/IP</strong> consistently across embedding and search.</li><li>Use <strong>metadata filters</strong> and <strong>pre‑/post‑filters</strong> to narrow search space; add rerankers for quality.</li><li>Cache aggressively (prompt caches, response caches); adopt <strong>TTL</strong> for chat history in Redis.</li><li>Batch writes, upserts, and use <strong>asynchronous pipelines</strong> for embeddings/features.</li></ul><h3>FAQ (Quick Hits)</h3><p><strong>Do I need all components?</strong> No — start with Lake + Vector + Redis + Doc DB and add Graph/Feature Store when justified.</p><p><strong>Can I use one multi‑model DB?</strong> Sometimes (e.g., pgvector + JSONB), but specialized stores scale/perform better for heavy workloads.</p><p><strong>How do I prevent stale data?</strong> Stream updates to online stores; run daily recompute jobs; add freshness checks to prompts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=81f73e06c074" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>