<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aryan Mehrotra on Medium]]></title>
        <description><![CDATA[Stories by Aryan Mehrotra on Medium]]></description>
        <link>https://medium.com/@aryan_mehrotra?source=rss-39ff37811db9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*t5mellrBVgp_h4xy.jpg</url>
            <title>Stories by Aryan Mehrotra on Medium</title>
            <link>https://medium.com/@aryan_mehrotra?source=rss-39ff37811db9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:46:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aryan_mehrotra/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Stop Prompting for Plumbing: How I Use AI Coding Assistants with GoFr]]></title>
            <link>https://blog.stackademic.com/stop-prompting-for-plumbing-how-i-use-ai-coding-assistants-with-gofr-b3c577e94972?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/b3c577e94972</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[agents]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gofr]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Tue, 06 Oct 2026 09:32:51 GMT</pubDate>
            <atom:updated>2026-10-06T09:32:51.826Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ftwjb_CecsQXirdVDaqk9g.png" /></figure><p>AI coding assistants changed what a first draft costs. You describe an endpoint, and thirty seconds later it compiles.</p><p>The catch shows up a week later. The draft has no tracing, no health check and no retry, and somebody has to notice what is missing.</p><p>I maintain GoFr, an opinionated Go framework for accelerated microservice development. Here is how I split the work between the assistant, the framework and myself, in five steps you can copy.</p><h3>The Breakdown: Three Workers, Three Jobs</h3><p>Just like a kitchen has stations, an AI-assisted service has three workers:</p><ol><li><strong>The Assistant:</strong> Writes business logic. Fast, cheap, forgetful.</li><li><strong>The Framework:</strong> Owns the production parts. Boring, consistent, always on.</li><li><strong>You:</strong> Judge what only a person can. Slow, expensive, worth it.</li></ol><p>Most teams let the assistant do all three jobs. That is where the pain comes from.</p><ul><li><strong>Real numbers:</strong> AI-written code compiles almost every time and passes security tests only 56% of the time, flat for a year (<a href="https://www.veracode.com/news/llms-are-getting-smarter-but-not-safer-veracode-2026-genai-code-security-report-finds-ai-generated-code-security-has-stalled-at-56%25-pass-rate/">Veracode, July 2026</a>).</li><li><strong>More real numbers:</strong> Teams with high AI adoption ship 34% more tasks. Their median review time is 5x longer and incidents per pull request are 3x higher (<a href="https://faros.ai/research/ai-acceleration-whiplash">Faros AI, 2026</a>, 22,000 developers).</li></ul><p>Both are vendor studies, so read the exact figures with a pinch of salt.</p><p>Writing got cheap. Reviewing and running did not.</p><p>Let’s fix that, one step at a time.</p><h3>Step 1: Hand Over the Rulebook</h3><p>An assistant with no context writes generic Go. Generic Go means hand-rolled logging, a custom /healthz and a fresh opinion about project layout in every service.</p><p>GoFr ships a guide written for assistants: <a href="https://gofr.dev/AGENTS.md">gofr.dev/AGENTS.md</a>. It is about 100 lines. It works with Claude Code, Cursor, Aider and Codex, and it links to one doc page per topic so the assistant fetches only what the task needs.</p><p>The five rules I care about most:</p><ul><li><strong>One handler signature:</strong> func(c *gofr.Context) (any, error) serves HTTP, gRPC, GraphQL, WebSocket, cron and CLI.</li><li><strong>No hand-wired observability:</strong> gofr.New() already sets up OpenTelemetry, Prometheus and structured logs.</li><li><strong>Pass the context down:</strong> Use c, not context.Background(), or traces and cancellation stop at your handler.</li><li><strong>Config lives in </strong><strong>configs/.env:</strong> No hardcoded ports, hosts or secrets.</li><li><strong>No custom health route:</strong> /.well-known/health and /.well-known/alive already exist.</li></ul><p>Put the link in the file your assistant reads on startup (AGENTS.md, CLAUDE.md or your editor&#39;s rules file). Do it once per repo and forget about it.</p><p>Need more depth? <a href="https://gofr.dev/llms.txt">gofr.dev/llms.txt</a> is a curated index, and <a href="https://gofr.dev/llms-full.txt">gofr.dev/llms-full.txt</a> is the whole documentation as one text file.</p><h3>Step 2: Prompt for Logic, Not Plumbing</h3><p>This is the step that saves the most time. Every line of plumbing in your prompt is a line the assistant can get subtly wrong, and a line you have to check afterwards.</p><p>So ask for what the service does. Leave out how it is wired.</p><p>Here is a prompt that works:</p><pre>Read https://gofr.dev/AGENTS.md first.<br>Build a GoFr service with GET /orders/{id}.<br>It returns the order&#39;s id and status from the MySQL table `orders`.<br>Do not add logging, tracing, metrics or a health endpoint.</pre><p>And here is what should come back:</p><pre>func main() {<br>    app := gofr.New()<br>    app.GET(&quot;/orders/{id}&quot;, getOrder)<br>    app.Run()<br>}<br><br>func getOrder(c *gofr.Context) (any, error) {<br>    o := Order{ID: c.PathParam(&quot;id&quot;)}<br>    err := c.SQL.QueryRowContext(c,<br>        &quot;SELECT status FROM orders WHERE id=?&quot;, o.ID).Scan(&amp;o.Status)<br>    if err != nil {<br>        return nil, err<br>    }<br>    return o, nil<br>}</pre><p>That is the whole service. You can review it in one look.</p><p>Now run it and see what nobody wrote:</p><ul><li><strong>Metrics:</strong> A Prometheus endpoint at /metrics on port 2121.</li><li><strong>Logs:</strong> JSON lines that carry the trace ID.</li><li><strong>Health:</strong> /.well-known/health and /.well-known/alive.</li><li><strong>Profiling:</strong> pprof on the metrics port.</li><li><strong>Shutdown:</strong> Graceful on SIGTERM, 30 seconds by default.</li><li><strong>Traces:</strong> Set TRACE_EXPORTER and TRACER_URL, and spans ship to your backend.</li></ul><p>Three habits keep my prompts small:</p><ol><li><strong>Name the datasource, not the driver:</strong> Say “from MySQL” and let the assistant reach for c.SQL.</li><li><strong>One handler at a time:</strong> The signature is identical everywhere, so handlers compose without glue.</li><li><strong>Ask for a GoFr migration:</strong> For schema changes, the guide carries five rules. The two that surprise people: each migration runs exactly once, and there is no down migration.</li></ol><h3>Step 3: Review What Only You Can Judge</h3><p>Once the framework owns the plumbing, your review time goes to the parts that need a brain.</p><p>This is the split I use on an AI-written GoFr pull request:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WyqLKSQhYxbB_702dPemDA.png" /></figure><p>One shortcut: if a pull request imports a Prometheus client directly or adds a /healthz route, the assistant skipped the rulebook. Send it back with the link. It takes ten seconds.</p><h3>Step 4: The Model Is Just Another Data source</h3><p>Sooner or later the service itself needs an LLM. Summaries, classification, a small agent.</p><p>The usual move is to drop a vendor SDK into a handler. Now you have a network call with no trace, no metric and no health check, sitting in your hot path. And it bills per token.</p><p>In GoFr you register the model once and reach it through the context, exactly like SQL or Redis:</p><pre>app.AddLLM(&amp;llm.Client{Provider: llm.Groq, Model: &quot;llama-3.3-70b-versatile&quot;})<br><br>func summarize(c *gofr.Context) (any, error) {<br>    return c.LLM().Generate(c, &quot;Summarise order &quot;+c.PathParam(&quot;id&quot;))<br>}</pre><p>The API key comes from LLM_API_KEY in the environment, never from code. You can also skip the code entirely and set LLM_PROVIDER, LLM_MODEL, LLM_API_KEY and LLM_BASE_URL.</p><p>What you get without asking:</p><ul><li><strong>Cost:</strong> app_llm_request_count and app_llm_tokens_per_request are Prometheus metrics from the first call. Alert on tokens before the invoice does it for you.</li><li><strong>Traces:</strong> Each model call is a span inside the request’s trace, right next to the SQL query that fed it.</li><li><strong>Health:</strong> The model is checked along with your other datasources.</li></ul><p>The built-in client speaks the OpenAI-compatible API, with presets for OpenAI, Groq, DeepSeek, Together and Ollama. Anything else, including a model running inside your own process, plugs in by implementing the ai.Model interface: chat, streaming, embeddings and tool calls.</p><p>Two things GoFr deliberately does not ship: a vector store and a built-in agent loop. You write the loop yourself in ordinary Go. The examples/using-ai service does it in about 40 lines.</p><h3>Step 5: Your API Is Now an Agent’s Toolbox</h3><p>So far, the assistant has been writing your service. The next shift is agents calling it.</p><p>The Model Context Protocol (MCP) is the open standard for that. It has lived under the Linux Foundation’s Agentic AI Foundation since December 2025 (<a href="https://aaif.io/news/linux-foundation-announces-formation-of-aaif">announcement</a>).</p><p>In GoFr it is one line:</p><pre>app.EnableMCP()</pre><p>Your existing routes become tools an agent can list and call. The defaults are the cautious ones:</p><ul><li><strong>Read-only:</strong> Only GET, HEAD, OPTIONS and QUERY routes are exposed. POST, PUT, PATCH and DELETE never are, so an agent cannot change state through this door.</li><li><strong>Local:</strong> The MCP server listens on 127.0.0.1, on its own port (MCP_PORT, default 8200). MCP_PORT=0 switches it off.</li><li><strong>Same rules:</strong> Tool calls run through the real router with the caller’s headers forwarded. Your auth middleware still applies.</li></ul><p>This changes how I think about read endpoints. A clear route name and a small, well-shaped JSON response now have two readers: the front end and the agent.</p><h3>The Boring Future</h3><p>The code an assistant writes is turning into the cheap part of a service. The expensive parts are knowing it is correct, seeing it in production and paying for it.</p><p>That is why I think the interesting work moves into the framework. The more a framework decides, the less a prompt has to say and the less a reviewer has to read.</p><p>If you try one thing this week, try this:</p><ol><li>Add the <a href="https://gofr.dev/AGENTS.md">gofr.dev/AGENTS.md</a> link to your assistant’s rules file.</li><li>Ask for one endpoint, with the line “do not add logging, tracing, metrics or a health endpoint”.</li><li>Run it, open localhost:2121/metrics and /.well-known/health, and count what you did not have to review.</li></ol><p>All code here was compiled against GoFr v1.62.0, which needs Go 1.26.</p><h3>References</h3><ul><li><a href="https://gofr.dev/AGENTS.md">GoFr guide for AI coding assistants</a></li><li><a href="https://github.com/gofr-dev/gofr">GoFr source and examples</a>, release v1.62.0</li><li><a href="https://www.veracode.com/news/llms-are-getting-smarter-but-not-safer-veracode-2026-genai-code-security-report-finds-ai-generated-code-security-has-stalled-at-56%25-pass-rate/">Veracode 2026 GenAI Code Security Report, press release</a>, 28 July 2026</li><li><a href="https://faros.ai/research/ai-acceleration-whiplash">Faros AI, AI Engineering Report 2026: The Acceleration Whiplash</a>, Q2 2026</li><li><a href="https://aaif.io/news/linux-foundation-announces-formation-of-aaif">Linux Foundation announces the Agentic AI Foundation</a>, 9 December 2025</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b3c577e94972" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/stop-prompting-for-plumbing-how-i-use-ai-coding-assistants-with-gofr-b3c577e94972">Stop Prompting for Plumbing: How I Use AI Coding Assistants with GoFr</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi-Agent Systems Are Just Services. We Forgot the Boring Parts.]]></title>
            <link>https://blog.stackademic.com/multi-agent-systems-are-just-services-we-forgot-the-boring-parts-15de7f8036bb?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/15de7f8036bb</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Tue, 21 Jul 2026 20:00:09 GMT</pubDate>
            <atom:updated>2026-07-23T11:31:02.206Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Du9IKEFdNLGGZNQ9QKzA8A.png" /></figure><p>Everyone is building AI agents. Almost no one is building them like production software — and the gap isn’t a model problem, it’s an engineering one we already know how to solve.</p><p>Gartner expects 40% of enterprise applications to ship task-specific AI agents by the end of 2026, up from roughly zero two years ago. The tooling has exploded to match. A new agent framework lands what feels like every week, each with its own orchestrator, its own memory abstraction, its own protocol with a three-letter name. MCP. A2A. Pick your acronym.</p><p>Here’s something I noticed while actually building one: <strong>almost none of it is new.</strong></p><p>Strip the vocabulary away and an “agent” is a small service that does four things. It calls a model. It calls some tools. Sometimes it calls <strong>another </strong>agent. And around all of that, it needs the same unglamorous machinery every backend service has needed for a decade — authentication, rate limiting, retries, timeouts, health checks, traces, and metrics.</p><p>We have built that thing a million times. It’s the pattern we spent a decade calling <strong>microservices</strong> — but strip the buzzword and it’s just a <strong>service</strong>: a request comes in, work happens, an answer goes out. The interesting part — the LLM — is maybe fifteen lines. The rest is the boring part. And the boring part is exactly where “impressive demo” and “thing you can actually run in production” diverge.</p><p>This is an argument that multi-agent systems got <strong>accidentally</strong> complicated, and that they don’t have to be. Not because the problem is easy, but because it’s a problem we’ve solved before. You just have to be thorough about the parts nobody screenshots.</p><h3><strong>What an agent actually is</strong></h3><p>Forget frameworks for a second. Here is the entire job description of a production agent:</p><p>1. <strong>Talk to a model</strong>— send a prompt, maybe with tools, get back text or a tool call.<br>2. <strong>Use tools</strong> — which, nine times out of ten, means calling an API you already own.<br>3. <strong>Talk to other agents</strong> — hand a sub-task to a specialist and wait for the answer.<br>4. <strong>Survive the real world</strong> — auth so not everyone can call you, rate limits so one caller can’t sink you, circuit breakers and retries so a flaky dependency doesn’t cascade, and enough observability that when it misbehaves at 2 a.m. you can <strong>see why</strong>.</p><p>Notice that only item 1 is new. Items 2, 3, and 4 are just… a networked service. The reason building agents <strong>feels</strong> hard is that the ecosystem sold you a novel abstraction for each of these — a bespoke tool registry, a bespoke agent-to-agent protocol, a bespoke tracing shim — and now you’re gluing seven libraries together and calling the seams “architecture.”</p><p>You don’t need any of that to be new. You need it to be *boring and correct.*</p><h3>The simple version, built</h3><p>To make the point concrete, I built a small multi-agent system the boring way — as ordinary services, leaning on an opinionated framework ([GoFr](<a href="https://gofr.dev/">https://gofr.dev</a>), whose 1.58 release wires LLM calls, tool exposure, and observability into the same batteries-included model it already gives databases). The language doesn’t matter to the argument; the shape does. You could draw the same diagram on any stack that takes infrastructure seriously.</p><p>Four services:</p><p>- an <strong>orchestrator</strong> — the front door, which uses an LLM to route each request to the right specialist;<br>- a <strong>data agent</strong> — answers questions about live business data by calling its <strong>own</strong> endpoints as tools;<br>- a <strong>support agent</strong> — triages a ticket and streams back a drafted reply;<br>- a <strong>knowledge agent</strong> — answers from internal docs (RAG).</p><p>The orchestrator is the whole “multi-agent” story, and it’s underwhelming in the best way. It registers each specialist as a resilient HTTP service and protects its own door — and that’s the entire resilience story, in configuration, not handler code:</p><pre>app.AddHTTPService(&quot;data-agent&quot;, &quot;http://localhost:8000&quot;,<br> &amp;service.CircuitBreakerConfig{Threshold: 4, Interval: 2 * time.Second}, // stop hammering a dead agent<br> &amp;service.RateLimiterConfig{Requests: 20, Window: time.Second, Burst: 25}, // cap the blast radius<br> &amp;service.HealthConfig{HealthEndpoint: &quot;.well-known/health-check&quot;},<br>)<br>app.EnableAPIKeyAuth(&quot;agents-demo-key&quot;) // the front door now requires a key</pre><p>Routing is an LLM call and a `switch`. Calling the chosen specialist is one line:</p><pre>resp, _ := c.GetHTTPService(spec.service).Post(c, spec.path, nil, body)</pre><p>That’s it. No agent-to-agent protocol to adopt, no service mesh to stand up. One agent calls another the same way any service calls any service — except the circuit breaker, the retries, the rate limit, and the trace propagation come for free because the infrastructure is opinionated about them.</p><p>The tool-using agent is the same story from the other side. Instead of hand-registering tools, it exposes the endpoints it already has:</p><pre>app.EnableMCP() // GET handlers become agent-callable tools<br>resp, _ := c.LLM().Chat(c, msgs, ai.WithTools(c.LLM().Tools().List()))</pre><p>The agent’s “tools” <strong>are</strong> its API. There was never a second thing to build.</p><h3>The part people skip</h3><p>Here is where most agent demos quietly end and most production incidents quietly begin: you cannot operate what you cannot see. A multi-agent request fans out — orchestrator to model, orchestrator to specialist, specialist to model, specialist to its own tools. When it’s slow, or wrong, or expensive, “which hop?” is the only question that matters, and a pile of `print` statements will not answer it.</p><p>So the real test isn’t “did the agent answer.” It’s “can I watch one request travel through the whole system.” Because every hop here is traced with no extra code, one `/assistant` call is **one distributed trace across two services** — 32 spans, seven levels deep:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*21RCmkD2GwaafWizICm0fA.png" /><figcaption>One request, one distributed trace across the orchestrator and the data agent</figcaption></figure><p>You can read the entire story off that timeline. The orchestrator’s routing call to the model (teal). The jump to the data agent (yellow — a <strong>different service</strong>, same trace). The data agent’s own agent loop, calling the model and then its own `/orders` endpoint as a tool. If any of it were slow, the slow bar would be right there. That’s not a nice-to-have. That’s the difference between a system you run and a system that runs you.</p><p>The same is true for cost and load. Token usage, request rates, inter-agent call volume, per-route latency — all of it falls out of the same instrumentation, onto one dashboard:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H2kA6GA35N10f0O_fQsnrw.png" /><figcaption>Live metrics across the multi-agent system</figcaption></figure><p>None of this was written by hand. It’s the dividend of treating agents as services on infrastructure that already knew how to trace, count, and health-check a request. The “boring parts” turned out to be the whole product.</p><h3>Be thorough, not clever</h3><p>The lesson generalizes past any one framework or language. If you’re building agents right now, the highest-leverage thing you can do is *resist the urge to treat them as a new category of software.* They aren’t. They’re services with a language model in the request path.</p><p>That reframing hands you twenty years of hard-won operational instinct for free:</p><p>- An agent that calls another agent is a network call. Give it a <strong>timeout, a retry, and a circuit breaker</strong>— the same way you’d never let one service block on another forever.<br>- An agent with a public endpoint is an attack surface. Put <strong>auth and rate limiting</strong> in front of it, because “it’s just an AI thing” is not a security model.<br>- An agent you can’t trace is an agent you can’t debug. Wire <strong>distributed tracing and metrics on day one</strong>, not after the first outage.<br>- A tool is usually an API you already have. <strong>Expose it; don’t reinvent it.</strong></p><p>Every one of those is unremarkable. That’s the point. The teams who ship reliable agents in 2026 won’t be the ones with the cleverest orchestration abstraction. They’ll be the ones who were thorough about the parts that don’t demo well.</p><h3>Try it</h3><p>The whole system runs locally with no API key and no model download — a small shim routes the model calls through your local CLI, so you can watch the traces light up for yourself:</p><pre>git clone https://github.com/aryanmehrotra/agents<br>cd agents<br># start the shim + agents, then:<br>curl -s localhost:8080/assistant -H &#39;X-Api-Key: agents-demo-key&#39; \<br> -d &#39;{&quot;query&quot;:&quot;which products are out of stock and what is our shipped revenue?&quot;}&#39;</pre><p>Open Jaeger and watch one question become one trace across the system.</p><p>Agents are not a new kind of software. They’re the oldest kind — a service that takes a request, does some work, and answers — with a genuinely new and powerful component dropped into the middle. Build the new part with care. Build everything around it like the boring, reliable backend engineering it has always been.</p><p>That’s not a step backward. It’s the shortcut everyone’s looking for.</p><p>— -</p><p><strong>The full multi-agent system — orchestrator, three specialists, the resilient inter-agent wiring, and the observability stack that produced the screenshots above — is on </strong><a href="https://github.com/aryanmehrotra/agents"><strong>GitHub</strong></a></p><p>Built using <a href="https://gofr.dev/">GoFr</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15de7f8036bb" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/multi-agent-systems-are-just-services-we-forgot-the-boring-parts-15de7f8036bb">Multi-Agent Systems Are Just Services. We Forgot the Boring Parts.</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ask Your LLM What Your Azure Bill Looks Like]]></title>
            <link>https://medium.com/@aryan_mehrotra/ask-your-llm-what-your-azure-bill-looks-like-a37ae09d1169?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/a37ae09d1169</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[azure]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 06:13:55 GMT</pubDate>
            <atom:updated>2026-04-30T06:13:55.758Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>The </em><a href="https://zopnight.com/"><em>zopnight</em></a><em> </em><a href="https://zopnight.com/docs/mcp-server/overview"><em>MCP server</em></a><em> lets your AI assistant answer real cost questions about your cloud — and behind every answer is a careful pipeline that picks the right Azure cost type, attributes reservations correctly, and never hides which number it’s showing you.</em></blockquote><h3>The pitch in one paragraph</h3><p>You should be able to open Claude or Cursor and ask, <em>“What did we spend on Azure last month, by resource?”</em> — and get an answer that matches the invoice. Not a number that pretends your reserved VMs cost zero. Not a dashboard you have to build first. Just an answer.</p><p><a href="https://zopnight.com/">Zopnight</a> ships an MCP (Model Context Protocol) server that exposes your cost data as read-only tools any MCP-compatible LLM can call. That’s the headline. The rest of this article is about why those answers can be trusted — because the moment you involve Azure Reservations or Savings Plans, getting cost questions right requires real work upstream.</p><h3>What the MCP server actually does</h3><p>The <a href="https://zopnight.com/docs/mcp-server/overview">zopnight MCP server</a> is a read-only HTTP endpoint that speaks the Model Context Protocol. Any client that supports MCP — Claude Desktop, Claude Code, Cursor, Windsurf, Codex — can connect to it with a Personal Access Token and start asking questions.</p><p>For cost questions specifically, the relevant tools are:</p><ul><li>get_costs — top-line cost and savings summary for an org.</li><li>get_cost_by_resources — per-resource cost breakdown, paginated and sortable. The &quot;what did each thing cost&quot; tool.</li><li>get_cost_by_provider — split across AWS, Azure, GCP for multi-cloud orgs.</li><li>get_cost_trends — daily / weekly / monthly time series, with date-range filters.</li><li>get_resource_cost_history — daily cost history for one specific resource.</li></ul><p>So in practice, you ask your assistant questions like:</p><ul><li><em>“What did we spend on Azure last month, broken down by resource?”</em> → get_cost_by_resources.</li><li><em>“Show me the daily cost trend for our Azure subscriptions over the last 90 days.”</em> → get_cost_trends.</li><li><em>“What’s the daily history for this VM?”</em> → get_resource_cost_history. For a reserved VM, the answer is the real prorated daily cost, not $0.</li><li><em>“Compare our cost across AWS, Azure, and GCP for this month.”</em> → get_cost_by_provider.</li></ul><p>A few things worth naming up front:</p><ul><li><strong>Read-only by construction.</strong> Write operations (create, update, delete, start, stop) return an explicit permission-denied error at the dispatcher level. There are no destructive tools exposed at all.</li><li><strong>Org-scoped tokens.</strong> A PAT belongs to one organisation. The tools that take an org_id argument can only see data for the org the token belongs to.</li><li><strong>Audited.</strong> Every tool call is logged to the same audit trail you can see in the Audit Logs page.</li><li><strong>Rate-limited.</strong> Validation attempts are throttled to keep token-stuffing from being viable.</li></ul><h3>Connecting it</h3><p>Generate a Personal Access Token in Developer Settings. The token is hashed on the server and shown to you exactly once. Then drop this into Claude Desktop’s claude_desktop_config.json (or the equivalent file for Cursor / Windsurf):</p><pre>{<br>  &quot;mcpServers&quot;: {<br>    &quot;zopnight&quot;: {<br>      &quot;url&quot;: &quot;https://&lt;your-gateway&gt;/mcp-server&quot;,<br>      &quot;headers&quot;: {<br>        &quot;Authorization&quot;: &quot;Bearer zn_pat_&lt;your-token&gt;&quot;<br>      }<br>    }<br>  }<br>}</pre><p>Claude Code users:</p><pre>claude mcp add zopnight \<br>  --transport http \<br>  --url &quot;https://&lt;your-gateway&gt;/mcp-server&quot; \<br>  --header &quot;Authorization: Bearer zn_pat_&lt;your-token&gt;&quot;</pre><p>That’s the whole setup. Restart your client and start asking questions.</p><h3>Why the answer matters — Azure has two cost numbers</h3><p>Now the part that turns a tool call from a parlour trick into something you can run a business on. When the MCP server answers <em>“how much did this Azure resource cost yesterday?”</em>, where does that number come from? And why is it more trustworthy than asking Azure directly?</p><p>Azure Cost Management can return two completely different numbers for the exact same subscription, the exact same day, the exact same query. One of them will tell you a fleet of reserved VMs cost you nothing. The other will tell you what the invoice will tell you. Most dashboards (and most ad-hoc API calls) don’t tell you which one they’re showing.</p><p>The Cost Management API takes a parameter called Type. It accepts two values that matter:</p><p><strong>ActualCost</strong> is cash-basis. It reports the money that left your account on the day it left. When you buy a one-year Reservation or a Savings Plan, that purchase shows up as a single lump charge against the billing account on the day you bought it. From then on, the reserved VMs themselves report <strong>$0/day</strong> for compute at the subscription scope, because the money was already spent up front. Pay-as-you-go resources continue to look normal.</p><p><strong>AmortizedCost</strong> is accrual-basis. It takes that lump-sum reservation purchase and *spreads it across the term*, attributing each day’s slice to the specific resources that consumed the reservation that day. A reserved VM that would have cost $50/day on-demand might show $30/day amortized — the real, effective rate you’re paying once the commitment is factored in. Over the term of the reservation, the amortized daily numbers sum to the same total as the Actual Cost lumpsum did. They just land where the work actually happened.</p><p>There’s also a Pricing Model dimension in the response: OnDemand, Reservation, Savings Plan, Spot. Only AmortizedCost gives you a useful per-resource breakdown across these — under ActualCost the reservation rows live at the billing-account scope, not at the resource.</p><p>The TL;DR:</p><ul><li>A fully-reserved VM fleet under <strong>ActualCost</strong> looks free for a year, then jumps when the reservation expires.</li><li>The same fleet under <strong>AmortizedCost</strong> shows steady, accurate daily spend that ties out to the invoice the entire time.</li></ul><p>Same API, same permissions, one parameter difference, wildly different stories. If you wired your LLM directly to Azure Cost Management with default settings, the model would happily report $0/day for your reserved fleet and you wouldn’t know it was wrong.</p><h3>Why this matters for actual decisions</h3><p>A useful way to think about any cost number you see in any dashboard or get from any tool: every shown cost lives in one of three buckets.</p><ol><li><strong>Free</strong> — the resource genuinely costs $0. Security groups, IAM roles, tags, NACLs. $0 is the truth.</li><li><strong>Rack rate</strong> — the resource has a cost, but you don’t have a working billing export. The cost is calculated from list price × hours × metadata (instance family, OS, tenancy, lifecycle). It’s an estimate based on what the resource <em>should</em> cost.</li><li><strong>Billing-backed</strong> — the resource has a cost and a working billing export. The cloud’s billing data is the source of truth.</li></ol><p>For Azure orgs sitting in bucket 3, the choice between <strong>ActualCost</strong> and <strong>AmortizedCost</strong> determines whether the bucket-3 numbers are usable at all. Pick wrong and:</p><ul><li>Reserved and Savings-Plan workloads look free, so nobody feels urgency to optimize them.</li><li>Right-sizing recommendations can’t compute real savings — savings against $0 is always 0%.</li><li>Showback and chargeback under-attribute spend to the teams running committed workloads. The team that drives the bill looks cheaper than the team running on-demand.</li><li>Reconciliation against the Azure invoice fails. Dashboard numbers don’t add up to the bill.</li></ul><p>If you have any Reservations or Savings Plans at all, ActualCost is for accountants reconciling the credit-card statement. It is not for cost-attribution dashboards, and it is definitely not for the data your AI assistant is pulling answers from.</p><h3>How zopnight gets the answer right</h3><p>Zopnight made the call early: every Azure account it manages is queried with AmortizedCost, full stop. There’s no per-org toggle, because there’s no scenario where ActualCost gives a more useful per-resource view. Here’s the path the data takes before the MCP server hands it back to your LLM.</p><h3>1. Pull the right cost type from Azure</h3><p>The Azure billing client requests ExportTypeAmortizedCost when calling Cost Management — both for the daily per-resource fetch (grouped by ResourceId and PricingModel) and for the monthly invoice-matching total (with Granularity=None, returning a single row per billing currency that ties out to the invoice).</p><p>The comment in the source captures the <em>why</em> directly:</p><blockquote><em>Amortized cost distributes reservation/savings-plan purchases across the term, so each resource shows its real effective daily cost and correct PricingModel (Reservation, SavingsPlan, Spot, OnDemand). ActualCost would show $0 for reserved VMs at subscription scope because the reservation purchase lives at the billing-account level.</em></blockquote><p>Same Azure permissions as ActualCost. Same endpoint. One parameter.</p><h3>2. Aggregate across pricing models</h3><p>A single VM’s daily cost can come back as multiple rows from Azure — for instance, a Savings Plan covers most of the hours and an OnDemand row covers the top-up. Zopnight sums those rows per (resource, day) and picks the highest-priority purchase type for the row, with priority <strong>Reservation &gt; SavingsPlan &gt; Spot &gt; OnDemand</strong>. The chosen purchase_type then drives downstream recommendation logic.</p><h3>3. Overlay onto the canonical cost-records table</h3><p>Zopnight stores a single row per (resource, day) in a cost_records table, with two cost columns side by side:</p><ul><li>cost_usd — the <strong>rack rate</strong>: zopnight&#39;s own calculation from list price × hours × metadata.</li><li>actual_cost_usd — the <strong>unblended cost</strong>: the number Azure billing reported, when available.</li></ul><p>When billing data lands, an overlay step writes actual_cost_usd, sets a cost_source = &#39;actual&#39; marker, and re-derives hourly_rate as actual_cost_usd / 24 so subsequent reads stay consistent. The rack-rate number in cost_usd is <em>never</em> overwritten — it stays as a fallback and as a comparison value. If billing data is missing for a day, reads fall back to rack rate cleanly. If billing data is wrong, the rack rate is right there to compare against.</p><p>For month-to-date views, partial months are projected to a full month via mtdSum / daysElapsed × 30, so users get a useful &quot;where will this month land&quot; estimate without waiting for finalization. Azure&#39;s late-arriving billing rows (typically 3–15 days after month end) re-overlay correctly because the schema permits it.</p><h3>4. Make recommendations honest about commitments</h3><p>This is where the pricing-model field earns its keep. Rules like “convert this VM to spot” or “buy a reservation for this VM” check the purchase_type from billing first. If a VM is already on a Reservation or Savings Plan, those rules <strong>skip it</strong> — recommending savings against an already-committed resource is noise, and noisy recommendations destroy trust faster than missing recommendations do.</p><p>Lifecycle is also captured at discovery time, not just at billing time. The Azure Resource Graph exposes a priority field on VMs (Spot, Low, Regular), which zopnight maps to a lifecycle of spot / low_priority / on_demand. Spot fleets are recognized for what they are even before billing data has caught up.</p><h3>5. Surface the source on every cost view</h3><p>In the UI, every cost view carries a single label that names the source of the number:</p><ul><li><strong>“Unblended Cost”</strong> when billing is wired up and populated for the org.</li><li><strong>“Rack Rate”</strong> when it isn’t.</li></ul><p>No silent fallback. No mixing. Cost trend widgets and reports use actualCostUsd with costUsd as the fallback, and the label tracks exactly which one is being shown. The MCP server inherits the same behaviour — when an LLM asks for cost data, the underlying response carries the same distinction, so an assistant can answer not just <em>&quot;what did this cost&quot;</em> but <em>&quot;and is that the unblended billing number or the rack rate?&quot;</em></p><h3>Putting it together</h3><p>When your LLM calls get_cost_by_resources against the zopnight MCP server, the chain looks like this:</p><ol><li><strong>Azure Cost Management</strong> is queried with ExportTypeAmortizedCost — so reservations and savings plans are already attributed to the resources that consumed them.</li><li><strong>Pricing-model rows are aggregated</strong> per resource and day, with the strongest commitment type winning the purchase_type slot.</li><li>cost_records<strong> is the single source of truth</strong> — actual_cost_usd for the unblended cost, cost_usd as the rack-rate fallback, never overwritten, with cost_source naming whichever is in play.</li><li><strong>The MCP tool reads from </strong>cost_records and returns the unblended numbers, paginated, sorted, with the source visible.</li></ol><p>The same dataset powers the UI’s “Unblended Cost” label, the recommendation engine’s commitment-aware skip logic, and the answer your LLM gives you in chat. One pipeline, one source of truth, surfaced wherever you happen to be asking the question.</p><h3>The takeaway</h3><ul><li><strong>Wire your LLM to your cost data via MCP</strong>, not to Azure Cost Management directly. The default Type parameter would make your reserved fleet look free.</li><li><strong>If you query Azure Cost Management yourself, query AmortizedCost.</strong> ActualCost is for reconciling the credit-card statement, not for attribution, right-sizing, or showback.</li><li><strong>Whatever tool you use, demand to know which cost type is on screen and which fallback applies.</strong> A dashboard — or an AI answer — that won’t tell you is one that’s hiding something.</li></ul><p><a href="https://zopnight.com/">Zopnight</a> does this by default for every Azure account it manages. That’s what “Unblended Cost” means when you see it in the UI. And because the same numbers are exposed via the MCP server, you can stop building one-off dashboards for every cost question and just ask.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a37ae09d1169" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We Built an MCP Server for Multi-Cloud. Here’s What AI Can Do With Your Infrastructure Data.]]></title>
            <link>https://medium.com/@aryan_mehrotra/we-built-an-mcp-server-for-multi-cloud-heres-what-ai-can-do-with-your-infrastructure-data-9d22c5c76ab1?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d22c5c76ab1</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Thu, 09 Apr 2026 21:42:51 GMT</pubDate>
            <atom:updated>2026-04-09T21:42:51.133Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ou94uxYigo2sWDSVbkh1w.png" /></figure><p>I work on ZopNight — a platform that connects to your AWS, GCP, and Azure accounts with read-only access and discovers every resource across all of them. We pull metrics, normalize costs, and run optimization rules.</p><p>We had a UI for all of this. Dashboards, tables, filters — it works. But dashboards show you what we pre-built a view for. If you want to know something we didn’t anticipate — say, “which of my databases across all three clouds have no encryption?” — you’re writing scripts or digging through three consoles.</p><p>So we built an MCP server. You connect it to Claude Code, Cursor, Windsurf, or Codex, and the AI gets 43 read-only tools to query your infrastructure. Not a chatbot we trained on cloud docs. An AI with live access to YOUR data — resources, costs, metrics, recommendations — across every connected cloud account.</p><h3>How It Actually Works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MYCREY140xP9Q97Z5_WZ8g.png" /></figure><p>MCP (Model Context Protocol) is a standard for AI tools to talk to external data sources over JSON-RPC 2.0. Our server exposes 43 tools over a single HTTP POST endpoint at /mcp.</p><p>Here’s a real tool call. The AI wants your resource summary:</p><pre>{<br>  &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>  &quot;id&quot;: &quot;1&quot;,<br>  &quot;method&quot;: &quot;tools/call&quot;,<br>  &quot;params&quot;: {<br>    &quot;name&quot;: &quot;get_resource_summary&quot;,<br>    &quot;arguments&quot;: { &quot;org_id&quot;: &quot;your-org-uuid&quot; }<br>  }<br>}</pre><p>The server validates the user’s token, checks org membership, checks that MCP is enabled for the org, and proxies to the right backend service. Response comes back as raw JSON in the MCP content format.</p><p>Authentication is a personal access token tied to your email. The gateway validates it, injects X-User-Email, and the MCP server verifies you belong to the org you&#39;re querying. Every request, every time.</p><p>The AI doesn’t get a special admin role. It sees exactly what you’d see. If you don’t have access to an org, neither does the AI using your token.</p><h3>What the 43 Tools Cover</h3><p>The tools span 8 domains: <strong>resources</strong> (full inventory, metrics, discovery status), <strong>costs</strong> (per-resource, per-provider, trends, history), <strong>recommendations</strong> (314 rules with savings estimates), <strong>teams</strong> (members, resource assignments), <strong>budgets</strong> (health tracking per team), <strong>schedules</strong> (start/stop automation), <strong>audit</strong> (logs, state history, events), and <strong>cloud accounts</strong> (connections, billing sync).</p><p>Each resource comes with rich metadata — name, type, instance type, status, region, provider, tags, VPC/subnet, parent-child relationships, and a provider-specific metadata blob. Costs are normalized across providers into one model. Recommendations carry severity, estimated savings, and the specific rule that fired.</p><p>The full tool reference with parameters is in the <a href="https://zopnight.com/">docs</a>.</p><h3>What This Makes Possible</h3><p>The tools are primitives. The AI composes them. Here’s what that actually looks like in practice.</p><h3>One Place for Your Entire Cloud</h3><p>Ask <em>“How many resources do I have across all providers?”</em> and the AI gives you the full inventory grouped by provider and account. AWS EC2 instances, GCP Cloud SQL databases, Azure AKS clusters — one response. No console switching.</p><p>This is table stakes but it’s genuinely something no single cloud provider gives you. AWS shows you AWS. GCP shows you GCP. Nobody shows you everything.</p><h3>The AI Connects Domains You’d Check Separately</h3><p>A dashboard shows you one view at a time. Resources on one page, costs on another, recommendations on a third.</p><p>The AI composes across domains. <em>“Show me my open recommendations sorted by savings impact with the cost of each affected resource.”</em> That’s two tools composed — recommendations and costs — into an answer no single dashboard view provides.</p><p><em>“What does the platform team own and what does it cost? Are they on budget?”</em> — the AI pulls team resources, matches costs, checks budget health. Three domains, one answer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Im9u0G07Zo5zTyd_W6M3rw.png" /></figure><h3>Cross-Cloud Cost Visibility</h3><p><em>“What am I spending on AWS vs GCP vs Azure? Which is growing fastest?”</em> — the AI pulls cost splits by provider and trends over time.</p><p><em>“Which individual resources are my most expensive across all clouds?”</em> — per-resource costs, normalized across providers, sorted by the AI.</p><p>No cloud provider builds a view that compares your spend against a competitor. That’s the view none of them will ever ship. An outside tool can.</p><h3>Compliance and Governance</h3><p>Every resource carries metadata — tags, encryption settings, network configuration, backup status. The AI can check whatever condition you describe.</p><p><em>“Which resources have no team tag?”</em> — reads tags on each resource, filters the gaps.</p><p><em>“Do I have resources in unexpected regions?”</em> — reads regions, flags outliers.</p><p><em>“Any databases without encryption at rest?”</em> — checks metadata across all providers in one shot.</p><p>These aren’t pre-built reports. The resource metadata is rich enough that the AI answers governance questions we never anticipated.</p><h3>Recommendation Analysis</h3><p>314 rules generate recommendations automatically. The dashboard shows them in a list. The AI reasons over them.</p><ul><li><em>“What are my top 10 highest-savings recommendations?”</em></li><li><em>“How much would I save if I addressed all idle resource recommendations?”</em></li><li><em>“Which team has the most open critical recommendations?”</em></li></ul><p>The AI cross-references recommendations with costs, teams, and resources — connecting data that lives on separate dashboard pages.</p><h3>Budget Accountability</h3><p><em>“Which teams are over budget and why?”</em> — the AI checks budget health, traces to that team’s resources, matches costs, and tells you where the overrun is coming from. That’s a multi-page dashboard investigation compressed into one question.</p><h3>Investigating Cost Changes</h3><p><em>“My costs jumped this month. What changed?”</em> — the AI pulls cost trends with daily granularity to pinpoint the date, checks per-resource costs to find what shifted, and looks at discovery status to see if new resources appeared. An hour in consoles becomes a 2-minute conversation.</p><h3>The Cloud Provider Conflict of Interest</h3><p>This is something I think about a lot.</p><p>AWS Trusted Advisor, GCP Recommender, Azure Advisor — they all give you recommendations. They’re built by the same company that bills you. Their revenue is your spend.</p><p>They’ll recommend downsizing an instance. They won’t tell you to move a workload to a competitor where it’s 40% cheaper. They’ll flag an idle resource — but not aggressively enough to surface 20 small orphaned resources that add up. They’ll recommend reserved instances, which locks you into their platform for 1–3 years.</p><p>ZopNight sits outside all three. Read-only IAM roles. Normalized data model. When the AI reasons over your infrastructure through our MCP server, it’s not filtered through the lens of someone who profits from your spend.</p><p>What providers expose through APIs is richer than what their consoles show. Instance metadata, network topology, security group chains, storage class details — it’s all in the API response, but the console shows a curated subset. We pull the full picture and make it queryable.</p><h3>The 314 Rules</h3><p>We wrote 314 recommendation rules:</p><ul><li><strong>Idle detection</strong> — stopped instances still billing for attached storage, databases with zero connections, unused NAT gateways, idle load balancers</li><li><strong>Orphaned resources</strong> — unattached volumes, dangling snapshots, unused network interfaces, reserved IPs pointing at nothing</li><li><strong>Right-sizing</strong> — CPU utilization below 5%, memory below 20%, storage type upgrades (gp2→gp3 on AWS)</li><li><strong>Compliance</strong> — missing encryption at rest, public access exposure, missing required tags, no backup configuration</li></ul><p>Each produces a recommendation with severity (critical/high/medium/low), estimated savings, and the affected resource. When a rule stops firing because you fixed the issue, the recommendation moves to optimised — the history stays.</p><p>These rules catch known patterns. But the 43 tools give the AI access to the same raw data the rules use. You can describe checks we never wrote rules for, and the AI applies them on the spot.</p><h3>The Tradeoffs</h3><p><strong>Discovery takes ~10 minutes.</strong> First-time account connection runs a full scan across all resource types. After that, you’re ready to query.</p><p><strong>The AI needs good questions.</strong> The 314 rules run automatically. The cross-cloud analysis that makes this more than a dashboard requires you to think of the right question. The tools are primitives. The value is in what you compose.</p><h3>Get Started</h3><p>Try it at <a href="https://zopnight.com/">zopnight.com</a>.</p><p>43 tools. 314 rules. Three cloud providers. The metadata your cloud providers surface through APIs but bury in their UIs — all of it queryable by an AI that’s on your side, not theirs.</p><p>The providers show you their cloud. This shows you yours.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d22c5c76ab1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Pub/Sub in Go: One Interface for Kafka, NATS, and SQS]]></title>
            <link>https://medium.com/gofr/pub-sub-in-go-one-interface-for-kafka-nats-and-sqs-efa2e8ebf361?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/efa2e8ebf361</guid>
            <category><![CDATA[pub-sub]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Wed, 01 Apr 2026 13:40:22 GMT</pubDate>
            <atom:updated>2026-05-01T10:36:20.525Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4r-5pIp3CrMW5zs0weHVyQ.png" /></figure><p>The first time we had to switch our message broker — from Kafka to SQS because of an infrastructure decision three levels above me — I spent three days rewriting consumer code. Not business logic. Infrastructure code. Every handler had kafka.Message structs, every commit was msg.Ack(), every offset was manually tracked. None of it was reusable.</p><p>The second time it happened (NATS JetStream, same story), I’d already switched to GoFr. That migration took about 20 minutes. I changed two environment variables and deleted a single line where I’d registered the broker. The handlers didn’t change at all.</p><p>That’s the point of GoFr’s pub/sub system: one interface, any broker. The same Subscribe() call works for Kafka, NATS JetStream, MQTT, Google Pub/Sub, Redis Streams, and AWS SQS. Your handler receives a *gofr.Context, calls ctx.Bind(), and returns an error. The broker details live entirely in environment variables.</p><h3>Writing Your First Subscriber</h3><p>A GoFr subscriber looks exactly like an HTTP handler — same signature, same context, same Bind() method:</p><pre>package main  <br>  <br>import &quot;gofr.dev/pkg/gofr&quot;  <br>  <br>func main() {  <br> app := gofr.New()  <br> app.Subscribe(&quot;order-events&quot;, handleOrder) app.Subscribe(&quot;inventory-updates&quot;, handleInventory)  <br> app.Run()}  <br>  <br>func handleOrder(ctx *gofr.Context) error {  <br> var order struct { OrderID    string  `json:&quot;orderId&quot;` CustomerID string  `json:&quot;customerId&quot;` Total      float64 `json:&quot;total&quot;` }  <br> if err := ctx.Bind(&amp;order); err != nil { ctx.Errorf(&quot;failed to parse order: %v&quot;, err) return err }  <br> ctx.Infof(&quot;processing order=%s customer=%s total=%.2f&quot;, order.OrderID, order.CustomerID, order.Total)  <br> // ... process the order return nil}  <br>  <br>func handleInventory(ctx *gofr.Context) error {  <br> var update struct { ProductID string `json:&quot;productId&quot;` Delta     int    `json:&quot;delta&quot;` }  <br> if err := ctx.Bind(&amp;update); err != nil { return err }  <br> _, err := ctx.SQL.ExecContext(ctx, &quot;UPDATE products SET inventory = inventory + ? WHERE id = ?&quot;, update.Delta, update.ProductID) return err}</pre><p>That’s the complete subscriber application. No Kafka client import, no consumer group management, no offset commits, no partition assignment. The broker is configured entirely through environment variables.</p><h3>Switching Brokers Without Touching Handler Code</h3><p>The handler code above runs unchanged on any of these configurations:</p><p><strong>Kafka:</strong></p><pre>PUBSUB_BACKEND=KAFKA  <br>PUBSUB_BROKER=kafka:9092  <br>CONSUMER_ID=order-processor</pre><p><strong>NATS JetStream:</strong></p><pre>PUBSUB_BACKEND=NATS  <br>PUBSUB_BROKER=nats://nats:4222  <br>NATS_STREAM=orders  <br>NATS_CONSUMER=order-processor</pre><p><strong>AWS SQS:</strong></p><pre>PUBSUB_BACKEND=SQS  <br>AWS_REGION=us-east-1  <br>AWS_ACCESS_KEY_ID=...  <br>AWS_SECRET_ACCESS_KEY=...</pre><p><strong>MQTT:</strong></p><pre>PUBSUB_BACKEND=MQTT  <br>MQTT_HOST=mqtt-broker  <br>MQTT_PORT=1883  <br>MQTT_CLIENT_ID_SUFFIX=order-processor</pre><p>The topic names in app.Subscribe(&quot;order-events&quot;, ...) map directly to Kafka topics, NATS subjects, SQS queue names, or MQTT topics depending on the backend. Your handlers never know which broker they’re talking to.</p><p>When I was working with an AI coding agent on a new event processor, I described the business logic and the agent wrote the handler. I asked it to make the handler testable with both Kafka and SQS. Its response: “you just change the env vars — the handler code doesn’t need to change.” It had picked up on GoFr’s pattern from the single ctx.Bind() call.</p><h3>Publishing Messages</h3><p>Publishing uses ctx.GetPublisher() from inside any HTTP handler:</p><pre>func placeOrder(ctx *gofr.Context) (any, error) {  <br> var req OrderRequest if err := ctx.Bind(&amp;req); err != nil { return nil, err }  <br> // Write to DB _, err := ctx.SQL.ExecContext(ctx, &quot;INSERT INTO orders (id, customer_id, total) VALUES (?, ?, ?)&quot;, req.ID, req.CustomerID, req.Total) if err != nil { return nil, err }  <br> // Publish event — same code regardless of Kafka/NATS/SQS msg, _ := json.Marshal(map[string]any{ &quot;orderId&quot;:    req.ID, &quot;customerId&quot;: req.CustomerID, &quot;total&quot;:      req.Total, &quot;status&quot;:     &quot;placed&quot;, })  <br> if err = ctx.GetPublisher().Publish(ctx, &quot;order-events&quot;, msg); err != nil { ctx.Errorf(&quot;failed to publish order event: %v&quot;, err) // Decide: return error or continue — the DB write succeeded }  <br> return map[string]string{&quot;orderId&quot;: req.ID, &quot;status&quot;: &quot;placed&quot;}, nil}</pre><p>The Publisher interface has one method: Publish(ctx, topic, message). Every broker implements it the same way. There’s no Kafka producer configuration, no SQS message attributes struct to fill, no NATS JetStream publish options to set for basic use.</p><h3>Observability Comes Free</h3><p>Every message GoFr processes — published or subscribed — is automatically traced and metered. You don’t add instrumentation.</p><ul><li>app_pubsub_publish_total_count — messages published per topic</li><li>app_pubsub_subscribe_total_count — messages consumed per topic</li><li>Each message processing creates a span in your distributed trace</li></ul><p>When a subscriber returns an error, it’s logged with the correlationID from the span, so you can find the exact failed message in Jaeger or your log aggregator. When processing is slow, the histogram tells you which topic’s consumers are lagging.</p><p>For event-driven systems where a message failure at 2am might not surface until a user complains the next morning, this observability is the difference between 5 minutes of debugging and 5 hours.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VfqJRycabJlAByhC-2_asg.png" /></figure><h3>Getting Started: Complete Pub/Sub Application</h3><p>A publisher service that accepts HTTP and publishes events:</p><pre>// publisher/main.go  <br>package main  <br>  <br>import (  <br> &quot;encoding/json&quot; &quot;gofr.dev/pkg/gofr&quot;)  <br>  <br>func main() {  <br> app := gofr.New() app.POST(&quot;/orders&quot;, createOrder) app.Run()}  <br>  <br>func createOrder(ctx *gofr.Context) (any, error) {  <br> var order struct { ID    string  `json:&quot;id&quot;` Total float64 `json:&quot;total&quot;` } if err := ctx.Bind(&amp;order); err != nil { return nil, err }  <br> msg, _ := json.Marshal(order) return &quot;published&quot;, ctx.GetPublisher().Publish(ctx, &quot;orders&quot;, msg)}</pre><p>A subscriber service that processes the events:</p><pre>// subscriber/main.go  <br>package main  <br>  <br>import &quot;gofr.dev/pkg/gofr&quot;  <br>  <br>func main() {  <br> app := gofr.New() app.Subscribe(&quot;orders&quot;, processOrder) app.Run()}  <br>  <br>func processOrder(ctx *gofr.Context) error {  <br> var order struct { ID    string  `json:&quot;id&quot;` Total float64 `json:&quot;total&quot;` } if err := ctx.Bind(&amp;order); err != nil { return err }  <br> ctx.Infof(&quot;processed order=%s total=%.2f&quot;, order.ID, order.Total) return nil}</pre><p>Both services share the same broker config in their environment. Change PUBSUB_BACKEND and PUBSUB_BROKER and both services switch brokers without a code change.</p><h3>Docs and Further Reading</h3><ul><li><a href="https://gofr.dev/docs/advanced-guide/using-publisher-subscriber">Using Publisher-Subscriber</a> — broker setup, all env vars, handler patterns</li><li><a href="https://gofr.dev/docs/quick-start/observability">Observability Guide</a> — pub/sub metrics and tracing</li><li><a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">Monitoring Service Health</a> — pub/sub health checks</li><li>Checkout GoFr at https://gofr.dev</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efa2e8ebf361" width="1" height="1" alt=""><hr><p><a href="https://medium.com/gofr/pub-sub-in-go-one-interface-for-kafka-nats-and-sqs-efa2e8ebf361">Pub/Sub in Go: One Interface for Kafka, NATS, and SQS</a> was originally published in <a href="https://medium.com/gofr">gofr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying GoFr Services to Kubernetes in 15 Minutes]]></title>
            <link>https://medium.com/@aryan_mehrotra/deploying-gofr-services-to-kubernetes-in-15-minutes-895b16dd97f4?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/895b16dd97f4</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[gofr]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Thu, 12 Mar 2026 03:44:59 GMT</pubDate>
            <atom:updated>2026-03-12T03:44:59.738Z</atom:updated>
            <content:encoded><![CDATA[<p>There’s a specific kind of frustration I associate with deploying Go services to Kubernetes for the first time. Your service works locally. You Dockerize it — fine. You write the Deployment manifest. Then you spend an afternoon figuring out: which health endpoint should the liveness probe hit? How do I get Prometheus to scrape my metrics? How do I configure the database connection without hardcoding it? Why do my pods keep restarting?</p><p>GoFr makes this almost anticlimactic. Health endpoints are automatic. Metrics are served on a fixed port. Config comes from environment variables. Every piece of information Kubernetes needs to manage your service — is it alive? is it ready? is it healthy? — is already provided by the framework.</p><p>This is one of those areas where I’ve noticed GoFr’s conventions produce AI-correct manifests on the first pass. When I describe a GoFr service to an AI agent and ask for a Kubernetes Deployment, the agent knows the liveness endpoint is /.well-known/alive, the readiness endpoint is /.well-known/health, and metrics are on port 2121. No guessing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c-zDtx8XGPZaK3ZNCv_Nuw.png" /><figcaption>GoFr Kubernetes deployment architecture</figcaption></figure><h3>The Dockerfile</h3><p>GoFr services are standard Go binaries. A multi-stage Dockerfile:</p><pre># Build stage  <br>FROM golang:1.23-alpine AS builder  <br>  <br>WORKDIR /app  <br>COPY go.mod go.sum ./  <br>RUN go mod download  <br>  <br>COPY . .  <br>RUN CGO_ENABLED=0 GOOS=linux go build -o service ./cmd/main.go  <br>  <br># Production stage  <br>FROM gcr.io/distroless/static-debian12  <br>  <br>WORKDIR /app  <br>COPY --from=builder /app/service .  <br>  <br># GoFr reads config from ./configs/ relative to the binary  <br>COPY --from=builder /app/configs ./configs  <br>  <br>EXPOSE 8000  # HTTP server  <br>EXPOSE 2121  # Metrics  <br>  <br>USER nonroot:nonroot  <br>ENTRYPOINT [&quot;/app/service&quot;]</pre><p>The distroless base image is safe and minimal. No shell, no package manager, no attack surface. GoFr doesn’t need anything in the OS.</p><p>One thing to note: GoFr can read config from a ./configs/.env file at the binary location, but in Kubernetes you’ll use actual environment variables (injected via envFrom pointing to ConfigMaps and Secrets), not mounted config files. The two approaches work the same way — GoFr reads both.</p><h3>The Kubernetes Deployment Manifest</h3><pre>apiVersion: apps/v1  <br>kind: Deployment  <br>metadata:  <br> name: product-service labels: app: product-servicespec:  <br> replicas: 3 selector: matchLabels: app: product-service template: metadata: labels: app: product-service annotations: # Tell Prometheus to scrape this pod prometheus.io/scrape: &quot;true&quot; prometheus.io/port: &quot;2121&quot; prometheus.io/path: &quot;/metrics&quot; spec: containers: - name: product-service image: your-registry/product-service:latest ports: - containerPort: 8000 name: http - containerPort: 2121 name: metrics env: - name: APP_NAME value: &quot;product-service&quot; - name: APP_VERSION valueFrom: fieldRef: fieldPath: metadata.labels[&#39;app.kubernetes.io/version&#39;] envFrom: - configMapRef: name: product-service-config - secretRef: name: product-service-secrets  <br> # Liveness: is the service process alive? livenessProbe: httpGet: path: /.well-known/alive port: 8000 initialDelaySeconds: 10 periodSeconds: 15 failureThreshold: 3  <br> # Readiness: are all dependencies (DB, Redis) healthy? readinessProbe: httpGet: path: /.well-known/health port: 8000 initialDelaySeconds: 15 periodSeconds: 10 failureThreshold: 2  <br> resources: requests: cpu: &quot;100m&quot; memory: &quot;128Mi&quot; limits: cpu: &quot;500m&quot; memory: &quot;256Mi&quot;</pre><p>The key insight here: liveness uses /.well-known/alive and readiness uses /.well-known/health. This is the correct pattern for database-backed services.</p><p>/.well-known/alive always returns 200 if the process is running. Kubernetes uses this to know if it should restart the pod (it shouldn’t restart a pod just because MySQL is temporarily unavailable).</p><p>/.well-known/health returns 200 only when all connected datasources are healthy. Kubernetes uses this to know if the pod should receive traffic. If MySQL is down, the pod stops receiving requests — but it doesn’t get killed and restarted. When MySQL comes back, the health check passes and traffic resumes automatically.</p><p>GoFr automatically checks every initialized datasource in the health response:</p><pre>{  <br> &quot;status&quot;: &quot;UP&quot;, &quot;name&quot;: &quot;product-service&quot;, &quot;sql&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;mysql:3306&quot;}, &quot;redis&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;redis:6379&quot;}, &quot;inventory&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;inventory-svc:8080&quot;}, &quot;payments&quot;: {&quot;status&quot;: &quot;DOWN&quot;, &quot;error&quot;: &quot;connection refused&quot;}}</pre><p>If any datasource is DOWN, the overall status becomes DEGRADED and the HTTP status code becomes non-200, which fails the readiness check.</p><h3>ConfigMap and Secrets</h3><pre>apiVersion: v1  <br>kind: ConfigMap  <br>metadata:  <br> name: product-service-configdata:  <br> APP_NAME: &quot;product-service&quot; LOG_LEVEL: &quot;INFO&quot; TRACE_EXPORTER: &quot;jaeger&quot; TRACER_URL: &quot;jaeger-collector:14317&quot; TRACER_RATIO: &quot;0.1&quot; METRICS_PORT: &quot;2121&quot; DB_HOST: &quot;mysql.database.svc.cluster.local&quot; DB_PORT: &quot;3306&quot; DB_NAME: &quot;products&quot; REDIS_HOST: &quot;redis.cache.svc.cluster.local:6379&quot;---  <br>apiVersion: v1  <br>kind: Secret  <br>metadata:  <br> name: product-service-secretstype: Opaque  <br>stringData:  <br> DB_USER: &quot;app&quot; DB_PASSWORD: &quot;your-password&quot; OPENAI_API_KEY: &quot;sk-...&quot;</pre><p>Every GoFr configuration option is an environment variable. You never modify code to change tracing backends, log levels, or database connections across environments. The ConfigMap for staging has LOG_LEVEL=DEBUG and TRACER_RATIO=1.0. Production has LOG_LEVEL=INFO and TRACER_RATIO=0.1. Same binary, different config.</p><h3>Prometheus Scraping and Grafana</h3><p>With the prometheus.io/scrape: &quot;true&quot; annotation, Prometheus’s pod discovery automatically picks up your service and starts scraping :2121/metrics. No ServiceMonitor object needed if you’re using annotation-based discovery.</p><p>The metrics GoFr exposes are immediately useful in Grafana:</p><pre># Request rate  <br>rate(app_http_response_count[5m])  <br>  <br># p95 latency  <br>histogram_quantile(0.95, rate(app_http_response_bucket[5m]))  <br>  <br># Error rate  <br>rate(app_http_response_count{status=~&quot;5..&quot;}[5m]) /  <br>rate(app_http_response_count[5m])  <br>  <br># Circuit breaker status (alert when open for &gt; 2 minutes)  <br>app_http_circuit_breaker_state{service=&quot;payments&quot;} == 1  <br>  <br># DB connection pool utilization  <br>app_sql_stats{stats=&quot;open_connections&quot;} /  <br>app_sql_stats{stats=&quot;max_open_connections&quot;}</pre><p>These dashboard queries work for every GoFr service with the same metric names. Your Grafana dashboard template works across all your services.</p><h3>Graceful Shutdown</h3><p>GoFr handles SIGTERM gracefully — it stops accepting new connections, waits for in-flight requests to complete, closes database connections, and exits. Your Kubernetes Deployment’s terminationGracePeriodSeconds gives GoFr time to do this cleanly.</p><pre>spec:  <br> terminationGracePeriodSeconds: 30</pre><p>No code needed. GoFr registers signal handlers and implements a coordinated shutdown across HTTP server, gRPC server, metrics server, and all datasource connections. When Kubernetes sends SIGTERM during a rolling update, your old pods drain cleanly before new pods take over.</p><h3>Docs and Further Reading</h3><ul><li><a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">Monitoring Service Health</a> — alive vs health endpoints, datasource status</li><li><a href="https://gofr.dev/docs/quick-start/observability">Observability Guide</a> — metrics port, Prometheus format, tracing config</li><li><a href="https://gofr.dev/docs/references/configs">GoFr Configuration Reference</a> — all env vars</li><li><a href="https://gofr.dev/docs/advanced-guide/http-communication">HTTP Communication</a> — downstream service health in the health check</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=895b16dd97f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why AI Coding Agents Use Fewer Tokens with GoFr]]></title>
            <link>https://medium.com/@aryan_mehrotra/why-ai-coding-agents-write-better-gofr-code-64359e104341?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/64359e104341</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Sat, 07 Mar 2026 15:01:57 GMT</pubDate>
            <atom:updated>2026-03-07T19:48:40.172Z</atom:updated>
            <content:encoded><![CDATA[<p>I run a lot of AI-assisted code generation. Claude, Cursor, GitHub Copilot — at this point, most of the scaffolding in our Go microservices started as AI output that I reviewed and shipped. After a few months of this, I noticed something: the code generated for GoFr services needed far less editing than code generated for services using raw net/http or even Gin.</p><p>It wasn’t because the AI knew GoFr better. It was because GoFr left fewer decisions open.</p><p>When I ask an AI agent to “add an HTTP service call with retry and circuit breaking,” a raw net/http service needs the agent to make a dozen micro-decisions: which HTTP client pattern, how to configure the transport, which retry library, where to put the circuit breaker logic, how to wire in metrics. Every decision is a chance to get something slightly wrong, and every wrong decision costs tokens to fix.</p><p>With GoFr, there’s exactly one answer to each of those questions. The agent writes the right code the first time. Less back-and-forth, fewer tokens spent on corrections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dC1g-k6FhG-sV5WEVa2Blg.png" /><figcaption>GoFr microservice connectivity and AI coding workflow</figcaption></figure><h3>The Real Cost of Framework Flexibility</h3><p>Flexible frameworks are marketed as a virtue. But when you’re working with an AI coding agent, framework flexibility becomes framework ambiguity — and ambiguity has a token cost.</p><p>Let me make this concrete. I asked three AI agents to add a database-backed HTTP endpoint to a Go service. Here’s roughly what each approach cost in terms of lines of code the agent had to generate before the business logic:</p><p><strong>Raw net/http + manual wiring:</strong></p><ul><li>HTTP handler registration: 15 lines</li><li>Database connection and pool setup: 25 lines</li><li>Structured logging setup: 20 lines</li><li>Prometheus metrics registration: 30 lines</li><li>OpenTelemetry tracing setup: 40 lines</li><li>Request/response middleware: 35 lines</li></ul><p>That’s ~165 lines of infrastructure scaffolding before a single line of business logic. The agent also had to make choices at each step — which logger package, which router, which metric names and label conventions.</p><p><strong>GoFr:</strong></p><pre>package main<br><br>import &quot;gofr.dev/pkg/gofr&quot;<br><br>type Product struct {<br>    ID   int    `json:&quot;id&quot;`<br>    Name string `json:&quot;name&quot;`<br>}<br><br>func main() {<br>    app := gofr.New()<br>    app.GET(&quot;/products/{id}&quot;, GetProduct)<br>    app.Run()<br>}<br><br>func GetProduct(ctx *gofr.Context) (any, error) {<br>    id := ctx.PathParam(&quot;id&quot;)<br><br>    var product Product<br><br>    err := ctx.SQL.QueryRowContext(ctx, &quot;SELECT id, name FROM products WHERE id = ?&quot;, id).<br>        Scan(&amp;product.ID, &amp;product.Name)<br>    if err != nil {<br>        return nil, err<br>    }<br><br>    return product, nil<br>}</pre><p>That’s 22 lines. Logging, metrics, and tracing are already active. The SQL connection is configured via DB_HOST, DB_USER, DB_PASSWORD, DB_NAME env vars. The agent writes the business logic and nothing else.</p><p>The AI doesn’t need to decide anything about infrastructure. It just needs to know the handler signature and the ctx API.</p><h3>Connected Services Without the Wiring</h3><p>The part that surprised me most was inter-service communication. In a standard Go setup, adding a call to a downstream service means the agent has to wire an HTTP client, configure connection pooling, add timeout logic, handle retries, and think about how to propagate trace context across the call boundary. That’s 80–100 lines of plumbing per service dependency.</p><p>GoFr reduces this to a one-line registration at startup:</p><pre>app.AddHTTPService(&quot;inventory&quot;, &quot;http://inventory-svc:8080&quot;,<br>    &amp;service.RetryConfig{MaxRetries: 3},<br>    &amp;service.CircuitBreakerConfig{Threshold: 5, Interval: 10 * time.Second},<br>)<br>app.AddHTTPService(&quot;pricing&quot;, &quot;http://pricing-svc:8080&quot;)<br>app.AddHTTPService(&quot;recommendations&quot;, &quot;http://rec-svc:8080&quot;)</pre><p>And then in any handler, the agent just calls:</p><pre>func GetProductDetails(ctx *gofr.Context) (any, error) {<br>    id := ctx.PathParam(&quot;id&quot;)<br><br>    // Call inventory service — trace context propagated automatically<br>    inv := ctx.GetHTTPService(&quot;inventory&quot;)<br>    resp, err := inv.Get(ctx, &quot;stock/&quot;+id, nil)<br>    // ...<br><br>    // Call pricing service<br>    pricing := ctx.GetHTTPService(&quot;pricing&quot;)<br>    priceResp, err := pricing.Get(ctx, &quot;price/&quot;+id, nil)<br>    // ...<br>}</pre><p>Every call through GetHTTPService automatically:</p><ul><li>Propagates the W3C traceparent header so Jaeger shows the full distributed trace</li><li>Records latency in app_http_service_response{service=&quot;inventory&quot;}</li><li>Retries on network errors and 5xx responses</li><li>Opens the circuit breaker after 5 consecutive failures</li></ul><p>The agent doesn’t need to know any of this. It calls the service by name and the framework handles the rest. When I paste this pattern into my prompt (“use GoFr’s AddHTTPService”), I get working, observable, resilient code in one pass.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eyshomJAKYHflZ6OvwdD2g.png" /><figcaption>GoFr service mesh connectivity diagram</figcaption></figure><h3>Observability as AI Context</h3><p>Here’s something I hadn’t anticipated. GoFr’s built-in observability doesn’t just help in production — it helps when I’m working with AI coding agents to debug or extend the service.</p><p>When something goes wrong, I can give the agent specific evidence instead of vague descriptions:</p><p><strong>Before GoFr</strong> (debugging with an AI agent):</p><blockquote><em>“The product details endpoint is slow sometimes. Help me figure out why.”</em></blockquote><p>The agent has to ask clarifying questions. What’s slow? How slow? Which dependencies? It needs the code context for every layer — HTTP handling, database queries, service calls. That’s hundreds of lines of context just to understand the system.</p><p><strong>With GoFr</strong> (debugging with an AI agent):</p><blockquote><em>“Here’s the trace for a 4.2s request: [paste Jaeger trace]. The </em><em>inventory service span took 3.8s. Here&#39;s the log line: </em><em>{&quot;trace_id&quot;:&quot;abc123&quot;,&quot;response_code&quot;:200,&quot;response_time&quot;:3821}. Help me figure out what&#39;s wrong.&quot;</em></blockquote><p>Now the agent has precise, structured information. It doesn’t need to read the infrastructure code at all. It sees a named span with a duration, a structured log entry with the correlationID, and the metrics showing this is consistently slow (not a spike). The debugging conversation is about the <em>problem</em>, not about understanding the codebase structure.</p><p>GoFr adds the correlationID (X-Correlation-ID) to every response header. Your logs include it on every line. Your Jaeger traces are linked to it. When you&#39;re debugging with an AI agent, you paste the trace and the log lines and you&#39;re immediately in problem-solving mode.</p><h3>The Health Check as System Documentation</h3><p>One more thing I’ve found genuinely useful when working with AI agents: GoFr’s /.well-known/health endpoint.</p><p>Every GoFr service exposes this automatically. It returns the status of every connected datasource and downstream service:</p><pre>{<br>  &quot;status&quot;: &quot;UP&quot;,<br>  &quot;name&quot;: &quot;product-service&quot;,<br>  &quot;version&quot;: &quot;1.2.3&quot;,<br>  &quot;sql&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;mysql:3306&quot;, &quot;stats&quot;: {&quot;open_connections&quot;: 4, &quot;idle&quot;: 3}},<br>  &quot;redis&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;redis:6379&quot;, &quot;stats&quot;: {&quot;connected_clients&quot;: 1}},<br>  &quot;inventory&quot;: {&quot;status&quot;: &quot;UP&quot;, &quot;host&quot;: &quot;inventory-svc:8080&quot;},<br>  &quot;pricing&quot;: {&quot;status&quot;: &quot;DOWN&quot;, &quot;error&quot;: &quot;connection refused&quot;}<br>}</pre><p>I started using this as context when prompting AI agents about the service. Instead of describing the architecture, I just paste the health response:</p><blockquote><em>“This is the health output for our product service. The pricing service is down. Add a fallback that returns a cached price when the circuit breaker is open.”</em></blockquote><p>The agent immediately knows the topology — what services exist, what datasources are connected, which ones are healthy. It doesn’t need to read the main.go to understand the service&#39;s dependencies.</p><p>MIT named generative coding a 2026 Breakthrough Technology. I think the reason frameworks like GoFr fit this era so well is exactly this: not that they were designed for AI, but that their conventions make the system’s intent legible — to both humans and AI agents. The same properties that help a new team member get productive quickly (few conventions to learn, no infrastructure decisions to make) also help an AI agent generate correct code quickly.</p><h3>What This Looks Like in Practice</h3><p>I want to show a complete before/after of what an AI coding session looks like with and without an opinionated framework. The prompt is identical in both cases: “Add an endpoint that fetches a user’s recent orders from the orders service and caches the result in Redis for 5 minutes.”</p><p><strong>Without GoFr</strong> — what the agent generates:</p><ol><li>Looks up the existing HTTP client setup (or creates one)</li><li>Figures out the connection pool configuration</li><li>Writes retry logic (or picks a library)</li><li>Looks up the Redis client setup</li><li>Writes serialization for the cache key</li><li>Writes the TTL cache logic manually</li><li>Adds trace propagation headers to the downstream call</li><li>Registers new Prometheus metrics for cache hit/miss</li></ol><p>Result: ~150 lines, probably needs review for consistency with existing patterns.</p><p><strong>With GoFr</strong> — what the agent generates:</p><pre>func GetUserOrders(ctx *gofr.Context) (any, error) {<br>    userID := ctx.PathParam(&quot;userID&quot;)<br>    cacheKey := &quot;orders:&quot; + userID<br><br>    // Check Redis cache<br>    cached, err := ctx.Redis.Get(ctx, cacheKey).Bytes()<br>    if err == nil {<br>        var orders []Order<br>        _ = json.Unmarshal(cached, &amp;orders)<br>        return orders, nil<br>    }<br><br>    // Fetch from orders service<br>    svc := ctx.GetHTTPService(&quot;orders&quot;)<br>    resp, err := svc.Get(ctx, &quot;users/&quot;+userID+&quot;/orders&quot;, nil)<br>    if err != nil {<br>        return nil, err<br>    }<br>    defer resp.Body.Close()<br><br>    var orders []Order<br>    _ = json.NewDecoder(resp.Body).Decode(&amp;orders)<br><br>    // Cache for 5 minutes<br>    data, _ := json.Marshal(orders)<br>    ctx.Redis.Set(ctx, cacheKey, data, 5*time.Minute)<br><br>    return orders, nil<br>}</pre><p>Result: ~28 lines, deploys correctly on the first try. Tracing and metrics are automatic. Redis and the orders service were already configured by the time this handler was written.</p><p>The difference isn’t AI capability. It’s that GoFr left the agent with nothing to decide except the actual problem.</p><h3>Getting Started</h3><p>If you’re building microservices that call each other and you’re using AI coding tools, the setup is minimal:</p><pre>package main<br><br>import (<br>    &quot;gofr.dev/pkg/gofr&quot;<br>    &quot;gofr.dev/pkg/gofr/service&quot;<br>    &quot;time&quot;<br>)<br><br>func main() {<br>    app := gofr.New()<br><br>    // Register downstream services<br>    app.AddHTTPService(&quot;orders&quot;, &quot;http://orders-svc:8080&quot;,<br>        &amp;service.RetryConfig{MaxRetries: 2},<br>        &amp;service.CircuitBreakerConfig{Threshold: 5, Interval: 10 * time.Second},<br>    )<br>    app.AddHTTPService(&quot;pricing&quot;, &quot;http://pricing-svc:8080&quot;,<br>        &amp;service.RetryConfig{MaxRetries: 1},<br>    )<br><br>    app.GET(&quot;/users/{id}/orders&quot;, GetUserOrders)<br>    app.GET(&quot;/products/{id}&quot;, GetProductDetails)<br><br>    // /.well-known/health and /.well-known/alive registered automatically<br>    // Metrics at :2121/metrics — prometheus format<br>    // Tracing via TRACE_EXPORTER env var<br><br>    app.Run()<br>}</pre><p>Configure in .env:</p><pre>APP_NAME=product-service<br>APP_VERSION=1.0.0<br>DB_HOST=mysql<br>DB_USER=app<br>DB_PASSWORD=secret<br>DB_NAME=products<br>REDIS_HOST=redis:6379<br>LOG_LEVEL=INFO<br>TRACE_EXPORTER=jaeger<br>TRACER_URL=localhost:14317</pre><p>That’s the entire infrastructure. Now every prompt you give an AI agent about this service can focus on the business problem. The agent knows the pattern: ctx.GetHTTPService(&quot;name&quot;) for service calls, ctx.Redis for cache, ctx.SQL for database, ctx.Trace(&quot;span-name&quot;) for custom spans, ctx.Metrics() for custom metrics.</p><p>One pattern. Zero infrastructure decisions. Correct code on the first pass.</p><h3>Docs and Further Reading</h3><ul><li><a href="https://gofr.dev/docs/quick-start/introduction">GoFr Quick Start</a> — get a service running in minutes</li><li><a href="https://gofr.dev/docs/advanced-guide/http-communication">HTTP Communication</a> — inter-service clients with retry and circuit breaking</li><li><a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">Monitoring Service Health</a> — the /.well-known/health endpoint and what it reports</li><li><a href="https://gofr.dev/docs/quick-start/observability">Observability Guide</a> — traces, metrics, structured logs via env vars</li><li><a href="https://gofr.dev/docs/advanced-guide/circuit-breaker">Circuit Breaker Support</a> — failure thresholds, health-based recovery</li><li><a href="https://gofr.dev/docs/advanced-guide/publishing-custom-metrics">Publishing Custom Metrics</a> — counter, histogram, gauge API</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=64359e104341" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Observability Stack Every LLM Service Needs]]></title>
            <link>https://medium.com/@aryan_mehrotra/the-observability-stack-every-llm-service-needs-765778cbf138?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/765778cbf138</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Tue, 03 Mar 2026 04:18:43 GMT</pubDate>
            <atom:updated>2026-03-03T04:18:43.791Z</atom:updated>
            <content:encoded><![CDATA[<p>I added an OpenAI call to our Go service six months ago. One morning I got a message: “the AI feature feels slow.” My logs showed nothing. My traces showed nothing. I had no idea whether the bottleneck was the database lookup before the call, the embedding generation, the chat completion itself, or the response parsing. I had a Go service calling an LLM and absolutely zero visibility into what was happening.</p><p>The problem wasn’t the LLM call. The problem was that I’d been so focused on making the AI feature <em>work</em> that I hadn’t thought about making it <em>observable</em>. And LLM calls are brutal when they go wrong: they’re slow, they have rate limits, they return 429s silently, and when the provider has an outage, your entire feature dies with zero warning.</p><p>I’ve since standardized on GoFr for all our Go services that touch LLMs. Not because it was “designed for AI” — it wasn’t. But because its built-in observability is exactly what you need when your service depends on an external API you don’t control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-VtoAic2u-K9MrYy9i365A.png" /><figcaption>GoFr observability architecture for LLM services</figcaption></figure><h3>The Three Signals You’re Probably Missing</h3><p>When something goes wrong with an LLM-backed feature, you need three things:</p><ol><li><strong>Traces</strong> — which operations ran, in what order, and how long each took</li><li><strong>Metrics</strong> — aggregate latency, error rates, retry counts, circuit breaker state</li><li><strong>Logs</strong> — structured per-request context, correlated to the trace</li></ol><p>Most developers building Go services that call LLMs set up at least one of these manually. They add log.Printf calls. They maybe instrument Prometheus. If they&#39;re disciplined, they add OpenTelemetry spans. But each one is a decision, a dependency, and boilerplate — and usually they end up with inconsistent coverage.</p><p>GoFr gives you all three automatically. Every HTTP request your service receives gets a trace span, a structured log line, and contributes to Prometheus metrics. You write none of that code.</p><pre>package main<br><br>import &quot;gofr.dev/pkg/gofr&quot;<br><br>func main() {<br>    app := gofr.New()<br>    app.POST(&quot;/complete&quot;, CompletionHandler)<br>    app.Run()<br>}<br><br>func CompletionHandler(ctx *gofr.Context) (any, error) {<br>    // Your LLM logic here.<br>    // Logging, tracing, and metrics are already running.<br>    return nil, nil<br>}</pre><p>That’s it. The moment app.Run() starts, GoFr has registered a tracing middleware, a metrics middleware, and a logging middleware on every route. Set two environment variables and your traces land in Jaeger:</p><pre>TRACE_EXPORTER=jaeger<br>TRACER_URL=localhost:14317</pre><p>No OpenTelemetry bootstrap code. No otel.SetTracerProvider(). No prometheus.MustRegister(). It&#39;s already done.</p><h3>Tracing the Actual LLM Call</h3><p>The automatic HTTP tracing covers your service’s <em>inbound</em> requests. But what you really want to see is the span for the OpenAI call itself — how long it took, what the upstream latency was, whether it got retried.</p><p>This is where GoFr’s HTTP service client earns its place. Instead of creating an http.Client manually, you register your LLM provider as a named service:</p><pre>import &quot;gofr.dev/pkg/gofr/service&quot;<br><br>app.AddHTTPService(&quot;openai&quot;, &quot;https://api.openai.com&quot;,<br>    &amp;service.RetryConfig{MaxRetries: 3},<br>    &amp;service.CircuitBreakerConfig{<br>        Threshold: 5,<br>        Interval:  10 * time.Second,<br>    },<br>)</pre><p>Then in your handler:</p><pre>func CompletionHandler(ctx *gofr.Context) (any, error) {<br>    svc := ctx.GetHTTPService(&quot;openai&quot;)<br><br>    body, _ := json.Marshal(map[string]any{<br>        &quot;model&quot;:    &quot;gpt-4o&quot;,<br>        &quot;messages&quot;: []map[string]string{{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}},<br>    })<br><br>    resp, err := svc.PostWithHeaders(ctx, &quot;v1/chat/completions&quot;, nil, body, map[string]string{<br>        &quot;Authorization&quot;: &quot;Bearer &quot; + os.Getenv(&quot;OPENAI_API_KEY&quot;),<br>    })<br>    // ...<br>}</pre><p>Every call through this service client automatically:</p><ul><li>Creates a child span in your current trace</li><li>Injects W3C traceparent headers into the outgoing request</li><li>Records latency in the app_http_service_response histogram (labeled with service, method, status)</li><li>Logs the request/response with the correlationID</li><li>Retries up to 3 times on network errors or 5xx responses</li><li>Opens the circuit breaker after 5 failures, stopping calls and exposing app_http_circuit_breaker_state=1 in Prometheus</li></ul><p>When I pull up a trace now, I see a single tree: the inbound POST span, the database lookup child span, and the OpenAI completion child span — each with its own duration. The slowness jumps out immediately.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9LlAk3mi4vd99vPrRfAS3g.png" /></figure><h3>Custom Spans for Your LLM Pipelines</h3><p>LLM applications often have multi-step pipelines: fetch context, build prompt, call model, parse response, store result. You want each step visible in your traces without creating a span for every function manually.</p><p>GoFr’s c.Trace() gives you named spans that automatically nest under the current request span:</p><pre>func CompletionHandler(ctx *gofr.Context) (any, error) {<br>    // Step 1: Fetch context from vector DB<br>    defer ctx.Trace(&quot;fetch-context&quot;).End()<br>    docs, err := fetchRelevantDocs(ctx)<br>    if err != nil {<br>        return nil, err<br>    }<br><br>    // Step 2: Build prompt<br>    span2 := ctx.Trace(&quot;build-prompt&quot;)<br>    prompt := buildRAGPrompt(docs, userQuery)<br>    span2.End()<br><br>    // Step 3: LLM call (traced automatically by service client)<br>    svc := ctx.GetHTTPService(&quot;openai&quot;)<br>    resp, err := svc.Post(ctx, &quot;v1/chat/completions&quot;, nil, prompt)<br><br>   return parseResponse(resp), err<br>}</pre><p>The logging side is just as clean. Every log you write inside a handler automatically inherits the trace ID, so you can filter all logs for a single request by correlationID:</p><pre>ctx.Infof(&quot;RAG query: docs=%d prompt_tokens=%d&quot;, len(docs), tokenCount)<br>ctx.Errorf(&quot;OpenAI 429: rate limited, backing off model=%s&quot;, model)</pre><p>In Jaeger or Grafana Tempo, every log line for a specific LLM call is one filter away. In production with multiple concurrent requests, this is the difference between debugging in minutes versus hours.</p><h3>Custom Metrics for Token Usage and Model Performance</h3><p>The built-in metrics cover HTTP latency and error rates. But for LLM services, you also want token usage, model-specific performance, and cache hit rates. GoFr’s metrics API is four methods:</p><pre>// Register once, at startup<br>app.Metrics().NewCounter(&quot;llm_requests_total&quot;, &quot;Total LLM API calls by model and status&quot;)<br>app.Metrics().NewHistogram(&quot;llm_latency_seconds&quot;, &quot;LLM response time in seconds&quot;,<br>    0.5, 1, 2, 5, 10, 30)<br>app.Metrics().NewCounter(&quot;llm_tokens_used_total&quot;, &quot;Prompt + completion tokens consumed&quot;)<br>app.Metrics().NewGauge(&quot;llm_context_cache_hit_ratio&quot;, &quot;RAG context cache hit rate&quot;)</pre><p>Then record them in your handlers:</p><pre>start := time.Now()<br>resp, err := svc.Post(ctx, &quot;v1/chat/completions&quot;, nil, body)<br>elapsed := time.Since(start)<br><br>status := &quot;success&quot;<br>if err != nil {<br>    status = &quot;error&quot;<br>}<br><br>ctx.Metrics().IncrementCounter(ctx, &quot;llm_requests_total&quot;,<br>    &quot;model&quot;, &quot;gpt-4o&quot;, &quot;status&quot;, status)<br>ctx.Metrics().RecordHistogram(ctx, &quot;llm_latency_seconds&quot;, elapsed.Seconds(),<br>    &quot;model&quot;, &quot;gpt-4o&quot;)<br><br>if usage := parseTokenUsage(resp); usage != nil {<br>    ctx.Metrics().IncrementCounter(ctx, &quot;llm_tokens_used_total&quot;,<br>        &quot;model&quot;, &quot;gpt-4o&quot;, &quot;type&quot;, &quot;prompt&quot;)<br>}</pre><p>These land on the metrics endpoint at port 2121 in Prometheus format. Your Grafana dashboard can plot p50/p95/p99 latency by model, token burn rate, and circuit breaker state — all without any Prometheus client library code in your service.</p><p>I noticed when building this that the same qualities making GoFr easy for me to instrument also make it easy for AI coding tools to get right. When I asked Claude to add metrics to a handler, it got the four-method API correct on the first try. No lookup, no guessing at configuration objects. The surface is just small enough that there’s only one obvious way to do each thing.</p><h3>Getting Started: A Complete LLM Service With Full Observability</h3><p>Here’s a minimal but production-ready GoFr service that calls an LLM with all three observability pillars wired up:</p><pre>package main<br><br>import (<br>    &quot;encoding/json&quot;<br>    &quot;os&quot;<br>    &quot;time&quot;<br>    &quot;gofr.dev/pkg/gofr&quot;<br>    &quot;gofr.dev/pkg/gofr/service&quot;<br>)<br><br>func main() {<br>    app := gofr.New()<br><br>    // Register OpenAI as a named HTTP service with resilience<br>    app.AddHTTPService(&quot;openai&quot;, &quot;https://api.openai.com&quot;,<br>        &amp;service.RetryConfig{MaxRetries: 2},<br>        &amp;service.CircuitBreakerConfig{<br>            Threshold: 5,<br>            Interval:  15 * time.Second,<br>        },<br>    )<br><br>    // Register custom metrics<br>    app.Metrics().NewCounter(&quot;llm_requests_total&quot;, &quot;LLM calls by model and outcome&quot;)<br>    app.Metrics().NewHistogram(&quot;llm_latency_seconds&quot;, &quot;LLM latency&quot;,<br>        0.5, 1, 2, 5, 10, 30)<br><br>    app.POST(&quot;/chat&quot;, ChatHandler)<br>    app.Run()<br>}<br><br>type ChatRequest struct {<br>    Prompt string `json:&quot;prompt&quot;`<br>    Model  string `json:&quot;model&quot;`<br>}<br><br>func ChatHandler(ctx *gofr.Context) (any, error) {<br>    var req ChatRequest<br>    if err := ctx.Bind(&amp;req); err != nil {<br>        return nil, err<br>    }<br><br>    if req.Model == &quot;&quot; {<br>        req.Model = &quot;gpt-4o-mini&quot;<br>    }<br><br>    ctx.Infof(&quot;chat request model=%s prompt_len=%d&quot;, req.Model, len(req.Prompt))<br><br>    defer ctx.Trace(&quot;openai-chat&quot;).End()<br><br>    body, _ := json.Marshal(map[string]any{<br>        &quot;model&quot;:    req.Model,<br>        &quot;messages&quot;: []map[string]string{{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: req.Prompt}},<br>    })<br><br>    start := time.Now()<br><br>    svc := ctx.GetHTTPService(&quot;openai&quot;)<br>    resp, err := svc.PostWithHeaders(ctx, &quot;v1/chat/completions&quot;, nil, body,<br>        map[string]string{&quot;Authorization&quot;: &quot;Bearer &quot; + os.Getenv(&quot;OPENAI_API_KEY&quot;)},<br>    )<br><br>    elapsed := time.Since(start)<br>    status := &quot;success&quot;<br>    if err != nil {<br>        status = &quot;error&quot;<br>        ctx.Errorf(&quot;openai call failed: %v&quot;, err)<br>    }<br><br>    ctx.Metrics().IncrementCounter(ctx, &quot;llm_requests_total&quot;,<br>        &quot;model&quot;, req.Model, &quot;status&quot;, status)<br>    ctx.Metrics().RecordHistogram(ctx, &quot;llm_latency_seconds&quot;, elapsed.Seconds(),<br>        &quot;model&quot;, req.Model)<br><br>     if err != nil {<br>        return nil, err<br>    }<br><br>    return parseOpenAIResponse(resp), nil<br>}</pre><p>Configure in .env or environment:</p><pre>LOG_LEVEL=INFO<br>TRACE_EXPORTER=jaeger<br>TRACER_URL=localhost:14317<br>TRACER_RATIO=0.1<br>METRICS_PORT=2121<br>OPENAI_API_KEY=sk-...</pre><p>Run it, hit /chat, and open Jaeger. You&#39;ll see the full trace: inbound handler span, OpenAI call child span, latency at each step. Hit /metrics on port 2121 and you&#39;ll see llm_requests_total and llm_latency_seconds ready for Grafana.</p><p>No OpenTelemetry bootstrap. No Prometheus client setup. No middleware registration. The framework does it.</p><h3>What This Looks Like in Production</h3><p>Six months after adding that first unobservable LLM call, I now have dashboards showing:</p><ul><li><strong>p95 latency by model</strong> — tells me when to route to a faster model</li><li><strong>circuit breaker state</strong> — alerts me when OpenAI is degraded before users complain</li><li><strong>retry rate</strong> — shows me when rate limiting is hitting</li><li><strong>error rate by status code</strong> — distinguishes 429s (rate limit) from 500s (provider errors) from 503s (outages)</li></ul><p>None of this required writing instrumentation code. The GoFr service client handles tracing, metrics, retries, and circuit breaking. I just register the service, name my custom metrics, and write business logic.</p><p>When an LLM provider goes down now, I see app_http_circuit_breaker_state{service=&quot;openai&quot;} flip to 1 in Prometheus, my alert fires in under a minute, and the service is already failing fast instead of hanging on timeouts. That&#39;s what observability actually means in production — not charts, but the ability to respond before users notice.</p><h3>Docs and Further Reading</h3><ul><li><a href="https://gofr.dev/docs/quick-start/observability">GoFr Observability Guide</a> — logging, metrics, tracing config</li><li><a href="https://gofr.dev/docs/advanced-guide/http-communication">HTTP Communication</a> — service client, connection pooling, auth</li><li><a href="https://gofr.dev/docs/advanced-guide/circuit-breaker">Circuit Breaker Support</a> — configuration and failure modes</li><li><a href="https://gofr.dev/docs/advanced-guide/publishing-custom-metrics">Publishing Custom Metrics</a> — counter, histogram, gauge API</li><li><a href="https://gofr.dev/docs/advanced-guide/custom-spans-in-tracing">Custom Spans in Tracing</a> — adding named spans to handlers</li><li><a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">Monitoring Service Health</a> — health endpoints and datasource stats</li><li><a href="https://gofr.dev/docs/advanced-guide/remote-log-level-change">Remote Log Level Change</a> — change log verbosity without restart</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=765778cbf138" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Ported LiteLLM to Go. Here’s What GoFr Made Trivial]]></title>
            <link>https://levelup.gitconnected.com/i-ported-litellm-to-go-heres-what-gofr-made-trivial-7ce94bdfcc44?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ce94bdfcc44</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[gofr]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Sun, 01 Mar 2026 16:51:16 GMT</pubDate>
            <atom:updated>2026-03-02T15:15:36.253Z</atom:updated>
            <content:encoded><![CDATA[<p>Go is fast. Obvious statement. But “fast” is not actually the interesting part when you sit down to build something complex.</p><p>The interesting question is: <em>how much infrastructure code do you have to write before you get to the part that’s actually your problem?</em> With Python and a mature framework, the answer is “not much.” Go’s reputation is the opposite — performance, yes, but you pay in boilerplate.</p><p>I wanted to test that assumption. So I picked <a href="https://github.com/BerriAI/litellm">LiteLLM</a> — one of the most complex tools in the Python AI ecosystem — and tried to build its core in Go using <a href="https://gofr.dev/">GoFr</a>. Not as a drop-in replacement, but as a GoFr example worth taking seriously. Not a todo app. Something with actual surface area: multiple HTTP backends, routing strategies, middleware, database migrations, caching, an admin UI.</p><p>The result is <a href="https://github.com/aryanmehrotra/LiteLLM-Go">LiteLLM-Go</a>. Here is exactly what GoFr handled and what I actually had to write.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GpwOu3rHkkFMT0c1QTiNpg.png" /><figcaption>Architecture diagram showing Your Code, GoFr framework layer, and LLM Providers</figcaption></figure><h3>The “Pico” Tradition in Go</h3><p>There is a quiet tradition in the Go community of porting popular Python and Java tools to Go — not to beat them at their own game, but to see what the core idea looks like without the runtime overhead. ripgrep is the famous one. But it happens constantly in quieter corners: lightweight CI runners, build tools, HTTP proxies.</p><p>LiteLLM felt like a natural candidate. It is genuinely impressive software — a unified Python proxy for routing requests across OpenAI, Anthropic, Gemini, and dozens more. The Python runtime gives it flexibility. It also gives it a 200MB+ baseline, a 3–5 second startup, and a Docker image as a deployment artifact.</p><p>The Go version: a single ~12MB binary. Starts in under 100ms. Compiles to a static executable you can drop anywhere.</p><p>The interesting constraint was: <em>could I replicate the meaningful features — provider routing, guardrails, virtual keys, cost tracking, admin dashboard — without drowning in infrastructure code?</em></p><p>The answer was yes, and GoFr is the reason why.</p><h3>What GoFr Gave Me Without Asking</h3><p>The honest test of a framework is what you get before you write a single line of business logic. Here is what gofr.New() brings to the table.</p><h3>Zero-config observability</h3><pre>app := gofr.New()</pre><p>That one call initializes a structured JSON logger, an OpenTelemetry tracer that auto-instruments HTTP calls and database queries, and a Prometheus metrics server. There is no setup ceremony — no init() functions, no global tracer registration, no Prometheus registry wiring.</p><p>Every request to the gateway is automatically traced end-to-end: from the incoming HTTP call, through the provider selection logic, to the outbound HTTP request to OpenAI or Anthropic, down to any Redis or PostgreSQL queries along the way. All correlated by trace ID.</p><p>For a proxy that touches six external services per request, this matters. Without GoFr, this is a week of work and an opentelemetry-go integration you are maintaining forever.</p><h3>Circuit breakers and connection pools — one call per provider</h3><p>The gateway talks to six LLM providers. Each can go down, rate-limit, or timeout. Handling this robustly in plain Go means vendoring a circuit breaker library, managing per-provider HTTP transports, and wiring up connection pools manually.</p><p>GoFr handles it:</p><pre>poolCfg := &amp;service.ConnectionPoolConfig{MaxIdleConnsPerHost: 20}<br>cbCfg := &amp;service.CircuitBreakerConfig{<br>    Threshold: cbThreshold,<br>    Interval:  time.Duration(cbIntervalSec) * time.Second,<br>}<br><br>app.AddHTTPService(&quot;openai&quot;, openaiBaseURL, poolCfg, cbCfg)<br>app.AddHTTPService(&quot;anthropic&quot;, &quot;https://api.anthropic.com&quot;, poolCfg, cbCfg)<br>app.AddHTTPService(&quot;ollama&quot;, ollamaBaseURL, poolCfg, cbCfg)<br>app.AddHTTPService(&quot;groq&quot;, &quot;https://api.groq.com/openai&quot;, poolCfg, cbCfg)<br>app.AddHTTPService(&quot;deepseek&quot;, &quot;https://api.deepseek.com&quot;, poolCfg, cbCfg)<br>app.AddHTTPService(&quot;gemini&quot;, &quot;https://generativelanguage.googleapis.com&quot;, poolCfg, cbCfg)</pre><p>Each call registers a pooled HTTP client with a circuit breaker and automatic tracing. Inside a provider adapter, getting the client is one line:</p><pre>svc := ctx.GetHTTPService(&quot;anthropic&quot;)<br>resp, err := svc.PostWithHeaders(ctx, &quot;v1/messages&quot;, nil, body, headers)</pre><p>The ctx here is GoFr&#39;s request context — it carries the trace span, logger, database handle, and Redis client. Pass it through and everything stays correlated.</p><h3>Database migrations — no external tool</h3><p>The gateway uses PostgreSQL for virtual keys, spend tracking, audit logs, teams, users, and organizations. That is 14 tables. Normally that means a separate migration tool (goose, flyway, migrate), a separate CI step, and a separate failure mode.</p><p>GoFr runs migrations as part of application startup:</p><pre>app.Migrate(migrations.All())</pre><p>migrations.All() returns a versioned map of migration functions. GoFr handles the advisory lock (so concurrent pods don&#39;t race on startup), version tracking, and idempotency:</p><pre>func createVirtualKeysTable() migration.Migrate {<br>    return migration.Migrate{<br>        UP: func(d migration.Datasource) error {<br>            _, err := d.SQL.Exec(`CREATE TABLE IF NOT EXISTS virtual_keys (<br>                id SERIAL PRIMARY KEY,<br>                key_hash VARCHAR(64) NOT NULL UNIQUE,<br>                name VARCHAR(255) NOT NULL,<br>                max_budget DECIMAL(10,4) DEFAULT 0,<br>                rate_limit_rpm INT DEFAULT 0,<br>                expires_at TIMESTAMP,<br>                is_active BOOLEAN NOT NULL DEFAULT TRUE,<br>                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP<br>            )`)<br>            return err<br>        },<br>    }<br>}</pre><p>No tooling to install, no schema files to keep in sync. The migrations live in the binary.</p><h3>Startup hooks and health checks</h3><p>After migrations, the gateway loads virtual keys into memory, seeds a default organization, and discovers locally available Ollama models. GoFr’s app.OnStart() hooks run after migrations but before the server opens for traffic:</p><pre>app.OnStart(func(ctx *gofr.Context) error {<br>    return keyStore.LoadFromDB(ctx)<br>})<br><br>app.OnStart(func(ctx *gofr.Context) error {<br>    ollamaProvider.RefreshModels(ctx)<br>    return nil<br>})</pre><p>The /health endpoint is registered automatically and checks PostgreSQL, Redis, and every HTTP service. I did not write a single health check handler. If you point a Kubernetes readiness probe at /health, it correctly reflects whether the gateway&#39;s dependencies are alive.</p><h3>What I Actually Had to Build</h3><p>GoFr is not magic. It does not know what Anthropic’s message format looks like, how to implement latency-based routing, or what PII detection means. The interesting parts were mine to write.</p><h3>Provider adapters — translating formats</h3><p>Every provider has its own request structure. OpenAI is the reference format. Anthropic uses a different messages structure, x-api-key headers, and stop_sequences instead of stop. Gemini uses a completely different nested format. Ollama is local with no authentication.</p><p>The translation layer is the core of the project. For Anthropic:</p><pre>func (a *Anthropic) ChatCompletion(ctx *gofr.Context, req models.ChatCompletionRequest) (*models.ChatCompletionResponse, error) {<br>    svc := ctx.GetHTTPService(&quot;anthropic&quot;)<br><br>    aReq := translateToAnthropic(req)  // OpenAI → Anthropic format<br>    body, _ := json.Marshal(aReq)<br><br>    headers := map[string]string{<br>        &quot;Content-Type&quot;:      &quot;application/json&quot;,<br>        &quot;x-api-key&quot;:         a.apiKey,<br>        &quot;anthropic-version&quot;: &quot;2023-06-01&quot;,<br>    }<br><br>    resp, err := svc.PostWithHeaders(ctx, &quot;v1/messages&quot;, nil, body, headers)<br>    // ...<br><br>    return translateFromAnthropic(aResp), nil  // Anthropic → OpenAI format<br>}</pre><p>The translateToAnthropic function handles the structural differences: system messages become a top-level field, tool calls become tool_use blocks, and stop_reason: tool_use maps back to OpenAI&#39;s finish_reason: tool_calls. Function calling translation is the fiddliest part — each provider has a slightly different schema for tools, arguments, and results.</p><h3>Routing strategies — six algorithms, one interface</h3><p>A Strategy interface picks a deployment from candidates:</p><pre>type Strategy interface {<br>    Select(deployments []Deployment, model string) *Deployment<br>}</pre><p>Six implementations: simple (first available), round-robin (atomic counter), weighted (proportional random), least-busy (reads in-flight tracker), latency (reads exponential moving average), usage (reads windowed token counter).</p><p>The Router wraps the strategy with retry logic and cooldown tracking:</p><pre>func (r *Router) ChatCompletion(ctx *gofr.Context, p ChatProvider, model string, req models.ChatCompletionRequest) (*models.ChatCompletionResponse, error) {<br>    if !r.Cooldown.IsAvailable(p.Name()) {<br>        return nil, fmt.Errorf(&quot;provider %s is in cooldown&quot;, p.Name())<br>    }<br><br>    r.InFlight.Increment(p.Name())<br><br>    defer r.InFlight.Decrement(p.Name())<br><br>    start := time.Now()<br>    resp, _, err := r.RetryPolicy.Execute(ctx, func() (*models.ChatCompletionResponse, error) {<br>        return p.ChatCompletion(ctx, req)<br>    })<br><br>    if err != nil {<br>        r.Cooldown.RecordFailure(p.Name())<br>        return nil, err<br>    }<br><br>    r.Latency.Record(p.Name(), time.Since(start))<br>    <br>    return resp, nil<br>}</pre><p>Getting the edge cases right — when to retry vs when to fail fast, how cooldown interacts with circuit breakers, streaming vs non-streaming paths — was where I spent the most time.</p><h3>Guardrails — pre and post processing</h3><p>Every request runs through a check before reaching the provider (block keywords, detect PII, enforce token limits) and every response runs through a filter (redact PII, truncate output):</p><pre>if err := guardrails.Check(cfg, req.Messages); err != nil {<br>    return nil, err<br>}<br><br>resp, err := router.ChatCompletion(ctx, provider, model, req)<br><br>resp = guardrails.Filter(ctx, cfg, resp)</pre><p>PII detection uses regex patterns for emails, phone numbers, SSNs, and credit card numbers. When PIIAction is &quot;redact&quot;, the filter replaces matches in the response with [REDACTED_EMAIL], [REDACTED_PHONE], etc. When it is &quot;log&quot;, it writes to the structured logger with the trace ID attached — so you can find the request in your observability stack without modifying the response.</p><h3>The Numbers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IkL7ZbsuJk1oEyMK9dZYUQ.png" /></figure><p>MetricLiteLLM (Python)LiteLLM-Go (GoFr)Memory at idle~200 MB~15 MBStartup time3–5 s&lt; 100 msDeployment artifactDocker image (500MB+)Single ~12MB binaryObservability setupManual integrationgofr.New()Circuit breakersWrite it yourselfAddHTTPService()DB migrationsSeparate toolapp.Migrate()Health checksWrite it yourselfAuto-registered</p><p>LiteLLM has far more providers — 100+ vs 6 here. That gap is a matter of writing more translation adapters, not a framework limitation. The infrastructure story is the point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6m2ga6E6ZanDa2ZOuG4_Jw.png" /></figure><h3>Running It</h3><pre>git clone https://github.com/aryanmehrotra/LiteLLM-Go<br>cd LiteLLM-Go</pre><pre>export OPENAI_API_KEY=sk-...<br>export ANTHROPIC_API_KEY=sk-ant-...</pre><pre>go run .</pre><p>The admin dashboard is at http://localhost:9000/admin. The OpenAI-compatible API is at /v1/chat/completions.</p><p>Point any OpenAI SDK at it:</p><pre>from openai import OpenAI<br><br>client = OpenAI(<br>    base_url=&quot;http://localhost:9000/v1&quot;,<br>    api_key=&quot;sk-gateway-key-1&quot;<br>)<br><br>response = client.chat.completions.create(<br>    model=&quot;anthropic/claude-sonnet-4-20250514&quot;,  # provider/model format<br>    messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Hello&quot;}]<br>)</pre><p>For the full stack with PostgreSQL, Redis, Prometheus, and Grafana:</p><pre>cd docker &amp;&amp; docker-compose up -d</pre><p>GoFr reads the environment, connects to PostgreSQL, runs the migrations, initializes Redis, starts the tracer, and opens for traffic. All without additional setup code.</p><h3>What This Experiment Showed</h3><p>The Go tax is real, but it is mostly concentrated in infrastructure code — the observability, health checks, connection management, and migration running that every production service needs. GoFr absorbs that tax almost entirely.</p><p>What remained was genuinely interesting: format translation between six different API schemas, routing algorithms with different tradeoffs, guardrail logic that works across streaming and non-streaming paths. That is the kind of code you want to write.</p><p>I also noticed something when I started using Claude Code to generate provider adapters: it never got confused about where things go. GoFr’s conventions — ctx carrying all dependencies, AddHTTPService for external clients, handlers as methods on structs — are predictable enough that the generated code was consistently idiomatic. I am not claiming GoFr was designed for AI coding tools. But opinionated frameworks produce predictable patterns, and predictable patterns are what code generation handles well.</p><p>The “pico” version of a Python tool is usually missing 80% of the features. This one has the features that matter: all major providers, routing strategies, guardrails, virtual keys, cost tracking, batch processing, and a built-in admin dashboard. The binary is 12MB. The startup is under 100ms.</p><p>That is what happens when the framework handles the infrastructure and you handle the logic.</p><p><strong>Links:</strong></p><ul><li><a href="https://github.com/aryanmehrotra/LiteLLM-Go">LiteLLM-Go source</a></li><li><a href="https://gofr.dev/docs/quick-start/introduction">GoFr quick start</a></li><li><a href="https://gofr.dev/docs/advanced-guide/http-communication">GoFr HTTP services guide</a></li><li><a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">GoFr observability</a></li><li><a href="https://gofr.dev/docs/advanced-guide/handling-data-migrations">GoFr database migrations</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ce94bdfcc44" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/i-ported-litellm-to-go-heres-what-gofr-made-trivial-7ce94bdfcc44">I Ported LiteLLM to Go. Here’s What GoFr Made Trivial</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tracing AI Pipelines in Go: Why Messages Vanish Between Brokers]]></title>
            <link>https://levelup.gitconnected.com/tracing-ai-pipelines-in-go-why-messages-vanish-between-brokers-e6625c9a3131?source=rss-39ff37811db9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6625c9a3131</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <dc:creator><![CDATA[Aryan Mehrotra]]></dc:creator>
            <pubDate>Fri, 27 Feb 2026 19:23:57 GMT</pubDate>
            <atom:updated>2026-03-02T15:24:52.413Z</atom:updated>
            <content:encoded><![CDATA[<p>You have a clean distributed trace. An HTTP request hits Service A, calls Service B over gRPC, and the whole thing shows up as one beautiful waterfall in Jaeger. Then Service B publishes a message to Kafka. Service C consumes it. And your trace just… stops. Service C starts a brand new trace. Two disconnected trees. No way to follow the message.</p><p>This is one of the most frustrating problems in event-driven Go architectures. Distributed tracing works perfectly for synchronous calls — HTTP, gRPC — because the trace context rides along in headers. But message brokers sit in between. There’s no HTTP header to propagate. The consumer might run minutes later, on a different machine. Traditional parent-child span relationships don’t make sense for async messaging.</p><p>The solution exists — <strong>OpenTelemetry span links</strong> — but most Go developers either don’t know about them or dread implementing the carrier/propagation boilerplate for every broker. In this article, you’ll learn how span links work, how to implement them in Go, and how frameworks like <a href="https://gofr.dev/docs/quick-start/introduction">GoFr</a> handle all of this automatically across Kafka, Google Pub/Sub, and AWS SQS.</p><h3>Why Parent-Child Doesn’t Work for Pub/Sub</h3><p>In synchronous tracing, the relationship is obvious: Service A calls Service B, so B’s span is a <strong>child</strong> of A’s span. They share a trace ID. The child starts after the parent and ends before it.</p><p>But pub/sub breaks every one of those assumptions:</p><p>Synchronous (HTTP/gRPC) Asynchronous (Pub/Sub) Consumer runs immediately Consumer may run hours later One producer → one consumer One producer → many consumers Parent span still active Parent span already ended Same trace context New process, new context</p><p>If you force a parent-child relationship, your traces become misleading. The consumer span appears “inside” the producer span, but the producer finished long ago. Latency calculations are wrong. Fan-out patterns create impossibly deep trace trees.</p><p>The <a href="https://opentelemetry.io/docs/specs/semconv/messaging/messaging-spans/">OpenTelemetry messaging semantic conventions</a> solve this with <strong>span links</strong>: the consumer span is its own root span, but it has a <strong>link</strong> back to the producer span. Same trace visibility, correct semantic relationship.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dz4ncCaa5NVeZoocmFHj8Q.png" /><figcaption>Parent-child spans break for async messaging — span links preserve the causal relationship without forcing a hierarchy</figcaption></figure><h3>Implementing Span Links in Go: The Manual Way</h3><p>To propagate trace context through a message broker, you need three pieces:</p><p>1. A carrier — an adapter that lets the OpenTelemetry propagator inject/extract trace context into your broker’s message format</p><p>2. Inject on publish — put the traceparent into the message</p><p>3. Extract on subscribe — pull it out and create a span link</p><p>Here’s what that looks like for a generic map[string]string attribute system (like Google Pub/Sub):</p><pre>// Step 1: Carrier adapter<br>type attributeCarrier map[string]string</pre><pre>func (c attributeCarrier) Get(key string) string    { return c[key] }<br>func (c attributeCarrier) Set(key, value string)     { c[key] = value }<br>func (c attributeCarrier) Keys() []string {<br>    keys := make([]string, 0, len(c))<br>    for k := range c {<br>        keys = append(keys, k)<br>    }<br>    return keys<br>}</pre><pre>// Step 2: Inject on publish<br>func injectTraceContext(ctx context.Context, attrs map[string]string) map[string]string {<br>    if attrs == nil {<br>        attrs = make(map[string]string)<br>    }<br>    otel.GetTextMapPropagator().Inject(ctx, attributeCarrier(attrs))<br>    return attrs<br>}</pre><pre>// Step 3: Extract on subscribe<br>func extractTraceLinks(attrs map[string]string) []trace.Link {<br>    carrier := attributeCarrier(attrs)<br>    extractedCtx := otel.GetTextMapPropagator().Extract(context.Background(), carrier)<br>    spanCtx := trace.SpanContextFromContext(extractedCtx)</pre><pre>    if spanCtx.IsValid() {<br>        return []trace.Link{{SpanContext: spanCtx}}<br>    }<br>    return nil<br>}</pre><p>Then on the publish side, you create a producer span and attach the trace context to the message:</p><pre>ctx, span := tracer.Start(ctx, &quot;publish&quot;, trace.WithSpanKind(trace.SpanKindProducer))<br>defer span.End()</pre><pre>attrs := injectTraceContext(ctx, nil)<br>// publish message with attrs...</pre><p>On the subscribe side, you receive the message first, then create a consumer span with the link:</p><pre>msg := receiveMessage() // get the message first<br>links := extractTraceLinks(msg.Attributes)</pre><pre>opts := []trace.SpanStartOption{<br>    trace.WithSpanKind(trace.SpanKindConsumer),<br>}<br>if len(links) &gt; 0 {<br>    opts = append(opts, trace.WithLinks(links...))<br>}</pre><pre>ctx, span := tracer.Start(ctx, &quot;subscribe&quot;, opts...)<br>defer span.End()</pre><p>That’s the pattern. But here’s the catch: <strong>every broker has a different message attribute format</strong>. Kafka uses byte headers. SQS uses MessageAttributeValue with a DataType field. Google Pub/Sub uses map[string]string. You need a different carrier for each one. And you need to make sure the subscribe span is created <em>after</em> the message arrives — not before — so you can access the attributes.</p><p>Most teams either skip this entirely or implement it for one broker and forget the others.</p><h3>How GoFr Handles It Automatically</h3><p><a href="https://gofr.dev/docs/advanced-guide/using-publisher-subscriber">GoFr</a> now handles span link propagation for all three major message brokers out of the box. You don’t write carrier code. You don’t change your handlers. You just publish and subscribe, and the traces connect.</p><p>The publish side injects trace context into message attributes automatically:</p><pre>// Your GoFr handler — no tracing code needed<br>func PublishOrder(ctx *gofr.Context) (any, error) {<br>    order := map[string]string{&quot;id&quot;: &quot;123&quot;, &quot;item&quot;: &quot;widget&quot;}<br>    data, _ := json.Marshal(order)</pre><pre>    err := ctx.GetPublisher().Publish(ctx, &quot;orders&quot;, data)<br>    return nil, err<br>}</pre><p>Behind the scenes, GoFr’s publish method creates a producer span with the right messaging.system attribute and injects traceparent into the message. For Google Pub/Sub, that goes into map[string]string attributes (<a href="https://github.com/gofr-dev/gofr/pull/3019">PR #3019</a>). For SQS, it goes into MessageAttributeValue with DataType: &quot;String&quot; (<a href="https://github.com/gofr-dev/gofr/pull/3018">PR #3018</a>). For Kafka, it goes into message headers (<a href="https://github.com/gofr-dev/gofr/pull/2952">PR #2952</a>).</p><p>On the subscribe side, GoFr receives the message first, extracts the trace context from the broker-specific attribute format, and creates a consumer span with a link to the producer:</p><pre>// Your subscriber handler — still no tracing code<br>func ProcessOrder(ctx *gofr.Context) error {<br>    var order map[string]string<br>    err := ctx.Bind(&amp;order)<br>    if err != nil {<br>        return err<br>    }</pre><pre>    ctx.Logger.Infof(&quot;processing order %s&quot;, order[&quot;id&quot;])<br>    return nil<br>}</pre><p>The span link happens inside GoFr’s subscribe implementation. Here’s the key change in the Google Pub/Sub subscriber — the span is now created <strong>after</strong> the message arrives:</p><pre>// Inside GoFr&#39;s Google Pub/Sub Subscribe method:<br>case m := &lt;-g.receiveChan[topic]:<br>    // Create span with links to producer span from message attributes<br>    spanCtx, span := startSubscribeSpan(ctx, topic, extractMessageAttrs(m.MetaData))<br>    defer span.End()</pre><p>This is a subtle but critical design choice. The old code created the span before receiving the message, so it couldn’t access the trace context in the message attributes. The new code flips the order: receive first, then create the span with the link.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WME7ZXSqD8HOlHCMlfxxlw.png" /><figcaption>Before: disconnected traces at every broker. After: span links connect producer and consumer spans across Kafka, Google Pub/Sub, and AWS SQS</figcaption></figure><h3>Why This Matters More in the Age of AI Pipelines</h3><p>The services most likely to hit this tracing gap are AI inference pipelines — and they’re everywhere now.</p><p>A typical AI-powered feature looks like this: an HTTP request arrives, your service publishes a job to SQS (“classify this document”), a worker picks it up, calls an embedding API, publishes the result to Google Pub/Sub, and a downstream service writes the enriched data to a database. Three brokers, four services, one user request. Without span links, debugging “why did this classification take 8 seconds?” means correlating timestamps across four separate traces by hand.</p><p>This is the architecture pattern that’s exploding in 2026 — RAG pipelines, real-time classification, embedding generation, agent workflows — and it’s almost always event-driven. The more your system relies on async message passing between AI components, the more you need traces that don’t break at every queue boundary.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J6uD0ao1umlzDKXgNco0zg.png" /><figcaption>AI inference pipeline with 4 services across SQS and Google Pub/Sub — GoFr span links make the entire pipeline visible in one Jaeger trace</figcaption></figure><h3>AI-generated pub/sub code skips tracing</h3><p>Here’s the other problem. If you ask an AI coding assistant to “build a Go service that publishes to SQS,” it will give you working publish/subscribe code — but it won’t add trace context injection. I’ve tried this with every major AI tool. The generated code creates spans, yes, but disconnected ones. The carrier adapter, the inject/extract calls, the subscribe-before-span reordering — none of that shows up.</p><p>This is the same pattern from the <a href="https://medium.com/2026-02-24-gofr-getting-started-microservice.md">getting started article</a>: AI produces better code when the framework handles more. With GoFr, the AI generates a publish handler like ctx.GetPublisher().Publish(ctx, &quot;orders&quot;, data) and the trace propagation is already there. The AI doesn&#39;t need to know about TextMapCarrier or extractTraceLinks because the framework owns it.</p><p>If you’re building AI pipelines with multiple message hops, this matters. Every hop where trace context isn’t propagated is a hop where your AI assistant also can’t help you debug. Connected traces mean you can paste a trace ID into your observability tool and see the entire pipeline — from the initial HTTP request through every queue, every AI inference call, every database write.</p><h3>Key Takeaways</h3><p>Distributed tracing breaks at message brokers because there’s no HTTP header to carry the trace context. OpenTelemetry span links fix this by letting consumer spans link back to producer spans without forcing a parent-child hierarchy. The pattern is the same for every broker: inject traceparent on publish, extract it on subscribe, create a span link.</p><p>The boilerplate — carrier adapters, inject/extract functions, tests — is mechanical and broker-specific. Let AI generate it. For <a href="https://gofr.dev/">GoFr</a> users, it’s handled automatically across Kafka, Google Pub/Sub, and SQS. Your handlers stay clean. Your traces connect.</p><pre>go get gofr.dev</pre><p>• <a href="https://gofr.dev/docs/advanced-guide/using-publisher-subscriber">GoFr Publisher/Subscriber guide</a></p><p>• <a href="https://gofr.dev/docs/advanced-guide/monitoring-service-health">Monitoring and observability</a></p><p>• <a href="https://gofr.dev/docs">GoFr quickstart</a></p><p>• <a href="https://github.com/gofr-dev/gofr">GoFr on GitHub</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6625c9a3131" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/tracing-ai-pipelines-in-go-why-messages-vanish-between-brokers-e6625c9a3131">Tracing AI Pipelines in Go: Why Messages Vanish Between Brokers</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>