<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sushanth on Medium]]></title>
        <description><![CDATA[Stories by Sushanth on Medium]]></description>
        <link>https://medium.com/@sushanthk07dev?source=rss-dbc26777c21c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*lRR14O7_JbARDY2S</url>
            <title>Stories by Sushanth on Medium</title>
            <link>https://medium.com/@sushanthk07dev?source=rss-dbc26777c21c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 23:40:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sushanthk07dev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[One API Catalog, Many Audiences]]></title>
            <link>https://medium.com/@sushanthk07dev/one-api-catalog-many-audiences-453207c72047?source=rss-dbc26777c21c------2</link>
            <guid isPermaLink="false">https://medium.com/p/453207c72047</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[open-api]]></category>
            <category><![CDATA[api-documentation]]></category>
            <category><![CDATA[zod]]></category>
            <dc:creator><![CDATA[Sushanth]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 14:01:49 GMT</pubDate>
            <atom:updated>2026-11-02T14:01:49.853Z</atom:updated>
            <content:encoded><![CDATA[<h4>How one Zod schema per endpoint is replacing a hand-maintained OpenAPI file in our Express backend, and giving every audience a reference it can trust.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ogt407KQEZydyreoEfJL-g.png" /></figure><p><strong>In short</strong></p><ul><li>Every migrated endpoint is defined once, as a Zod contract that validates requests, checks responses and generates the docs.</li><li>One catalog produces a separate reference for our apps, our customers and individual partners.</li><li>Writing the contracts surfaced drift and bugs that nobody had reported.</li></ul><p>Our API reference told integrators that conference-call search results carry a field called sentiment_analysis_link. The server has never sent it. It sends sentiment_link. No one made a mistake on any particular day. The reference and the code simply stopped agreeing, and nothing noticed. We only found out when we wrote a schema for that endpoint.</p><p>That reference was a 3,597-line OpenAPI file we maintained by hand, and everyone who builds on Needl.ai depended on it: our own applications, enterprise customers with API keys, and partners who embed Needl.ai in their platforms. Here’s how we made sure the next mismatch gets noticed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Aem-IV8yx383-Oqsqjn38Q.png" /></figure><h4>Part one · The idea</h4><h4>Nine terms this post uses</h4><ul><li><strong>Endpoint.</strong> One operation an API offers, such as listing a user’s workspaces.</li><li><strong>API reference.</strong> The document developers read to learn what an API accepts and returns. Ours follow the OpenAPI standard and render in Swagger UI.</li><li><strong>Schema.</strong> A precise description of the shape of some data: which fields exist, their types, which are required.</li><li><strong>Zod.</strong> An open-source JavaScript library for writing schemas as code and checking data against them.</li><li><strong>Contract.</strong> Our name for the schema of one endpoint, covering its inputs and every response it can return.</li><li><strong>Validation.</strong> Checking real data against a schema, and rejecting it when it does not match.</li><li><strong>Drift.</strong> When the reference and the running code disagree.</li><li><strong>Audience.</strong> A group of readers who get their own reference: our applications, external customers, or a specific partner.</li><li><strong>CI.</strong> The automated checks every change must pass before it can merge.</li></ul><h3>Why hand-maintained references drift</h3><p>Every endpoint used to be described twice. Request rules lived in validation code. The published reference lived in the JSON file, and response shapes existed only there, as prose and examples that nothing checked against the server. Every change had to land in both places. In practice, it rarely did.</p><p>No single mismatch was a crisis. Together they broke the one promise a reference makes: that you can build against it without checking. An integrator reading ours would have coded against a field that never arrives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-SBDddK8TzM2jyvKjV7q4A.png" /><figcaption>Three places where the hand-maintained reference disagreed with the code. We found all three while writing contracts, not from bug reports.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xa1-mUbTKBhhjGlo8n3__w.png" /><figcaption>Before, validation and the reference were two unconnected descriptions of each endpoint. Now one contract drives validation, response checks and the reference.</figcaption></figure><h3>One definition, three jobs</h3><p>A <strong>contract</strong> is a plain JavaScript object that declares what an endpoint accepts and what it returns, written in <a href="https://zod.dev/">Zod</a>. Because it runs on every request, it can’t go stale without something noticing. We get three things from it:</p><ul><li><strong>Request validation.</strong> Bad input gets a structured 400 that names the field and the rule it broke.</li><li><strong>Response checks.</strong> In tests, a response that doesn’t match its contract fails the build. In production, we log it after the response has gone out.</li><li><strong>Generated references.</strong> We build an OpenAPI 3.1 document for each audience, with descriptions, examples and error responses taken from the same definitions.</li></ul><blockquote><em>Drift no longer waits for a customer to find it. In tests, it breaks the build. In production, it files a warning.</em></blockquote><h3>One catalog, many audiences</h3><p>The harder problem was that we don’t have one reader. Needl.ai ships one backend to many customers. Some endpoints are for everyone, some exist for a single partner, and some are internal interfaces our apps rely on but no customer should build against. The old approach filtered one JSON file by tag and patched the result with per-customer overrides. Every audience inherited the drift of the source file, plus the drift of its overrides.</p><p>Now we group contracts by product capability and register each group once, in a single catalog. An <strong>audience</strong> is a short file that picks the groups it publishes, the server they live on and how callers authenticate. Exposing a capability to one customer is a one-line change to that customer’s audience file. No one else’s reference moves.</p><p>Visibility also works below the endpoint. A shared endpoint can carry a field that only one integration uses. Instead of forking the endpoint, we tag the field with the audiences allowed to see it. Every other reference drops the field and its required entry. The tags shape what each reference documents. They don&#39;t change what the server returns, and access control still lives in middleware.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7kuxw1mTYmbQ0gFNc3aQPg.png" /><figcaption>Our applications: every group is published, and the partner-only field is pruned, along with its required entry.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0hQHVnopm9W80VYNZPdT8g.png" /><figcaption>A partner: groups B and D are published, and the partner-only field appears, required.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0ObFhAALiAnhAxxdgv3Sqw.png" /><figcaption>Public API: only group A is published, so the record schema isn’t part of this reference at all. Names are illustrative.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A5USfBqM5gDjHa1aC7xCuQ.png" /><figcaption>GET /user/workspaces in a generated reference. The description and example come from the contract shown in Part two. Nobody wrote this page separately.</figcaption></figure><h3>What this means for teams building on Needl.ai</h3><p>For every migrated endpoint, the reference you integrate against comes from the code that serves your requests, and every tested path is checked against it before release. Partners get a reference scoped to their integration, with their authentication and only the capabilities open to them.</p><h4>Part two · The engineering</h4><h3>Why we chose Zod and zod-openapi</h3><p>Our requirements were narrow. We wanted one definition per endpoint for both validation and docs, no second specification file, and a migration we could run route by route without pausing feature work. We weighed six approaches against that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MLH8oXqNHnrptXEk4zkkTA.png" /><figcaption>The six approaches we weighed, and the one we chose.</figcaption></figure><p>We picked zod-openapi because it builds on Zod 4&#39;s own JSON Schema conversion rather than keeping a separate engine. New schema features reach our docs without waiting on anyone. The Express glue is about 100 lines of middleware that we own.</p><h3>Anatomy of a contract</h3><p>Here’s an abridged production contract for the endpoint shown above:</p><pre>const WORKSPACE_SCHEMA = z<br>  .object({<br>    id: z.uuid().meta({ description: &quot;Unique identifier of the workspace&quot; }),<br>    name: z.string().meta({ description: &quot;Display name of the workspace&quot; }),<br>    organization_id: z.int().meta({ description: &quot;Numeric id of the backing organization&quot; }),<br>    status: z.enum([&quot;member&quot;, &quot;invited&quot;]),<br>    active: z.boolean().meta({ description: &quot;True for the workspace the session operates in&quot; }),<br>  })<br>  .meta({ id: &quot;UserWorkspace&quot; });   // becomes components/schemas/UserWorkspace<br><br>const contracts = {<br>  getWorkspaces: {<br>    method: &quot;get&quot;,<br>    path: &quot;/user/workspaces&quot;,<br>    summary: &quot;List workspaces the user belongs to&quot;,<br>    request: { query: AUTH_QUERY_SCHEMA },<br>    responses: {<br>      200: { content: { &quot;application/json&quot;: { schema: GET_WORKSPACES_RESPONSE_SCHEMA, example } } },<br>      403: FORBIDDEN_RESPONSE,<br>      500: INTERNAL_SERVER_ERROR_RESPONSE,<br>    },<br>  },<br>};</pre><p>And its whole footprint in the routing layer:</p><pre>router.get(&quot;/workspaces&quot;, zodValidate(workspaceContracts.getWorkspaces), getWorkspaces);</pre><p>Controllers still read validated input from req.matchedData, exactly as before. Migrating an endpoint doesn&#39;t have to touch its business logic, which is what let us move one route at a time.</p><h3>Checking both directions</h3><p>zodValidate(contract) checks the request on the way in and wraps res.json to check the response on the way out.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4nH3yH_k1sq1lxtSgHxmHg.png" /><figcaption>Bad input never reaches the controller. A drifting response fails the request in tests, and is logged without delaying anyone in production.</figcaption></figure><p>Rejected requests always get the same envelope, with issues grouped by the part of the request that failed:</p><pre>{<br>  &quot;error&quot;: &quot;Validation failed&quot;,<br>  &quot;details&quot;: {<br>    &quot;params&quot;: [<br>      { &quot;code&quot;: &quot;invalid_format&quot;, &quot;path&quot;: [&quot;report_id&quot;], &quot;message&quot;: &quot;report_id must be a valid UUID v4&quot; }<br>    ]<br>  }<br>}</pre><p>The response check changed how we test. In tests, a mismatch throws before the body is sent and the request fails. That makes every happy-path test of an endpoint a test of its contract too, with no extra assertion to write.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oa02NZesk4Bw4PERI2I84A.png" /><figcaption>The response check’s message, formatted for reading, when a controller returns organization_id as a string and drops active. In tests, the request fails and so does the route’s spec.</figcaption></figure><h4>Watching production</h4><p>In production the same check runs after the response has gone out, and only logs a warning. Within weeks it caught something real. Once search moved to a contract, it started logging a mismatch on GET /search every night. Some results carried a null thumbnail where the contract promised a string. Every response was still delivered. When we dug in, the contract turned out to be right and the response wrong, so we fixed search to leave out null thumbnails instead of loosening the schema.</p><p>Since the first contracts reached production traffic in September, the check has flagged four distinct mismatches. As of publication, none has fired since 27 October.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RFyg-I2-WH5F3JTdhMbYVA.jpeg" /><figcaption>Every distinct mismatch the check has raised on production traffic, as of publication, and one of the nightly thumbnail warnings with trace identifiers removed.</figcaption></figure><h3>Generating an OpenAPI document per audience</h3><p>The document builder is about 110 lines. It walks the groups an audience picks, turns each contract into an OpenAPI operation and hands the result to zod-openapi&#39;s createDocument. It then removes the parameters our authentication middleware injects, which matter internally but aren&#39;t part of the public interface, and prunes fields tagged for other audiences, as the audience figures above show.</p><h3>Testing the contracts themselves</h3><p>A contract is code, and it can be wrong in ways an endpoint’s own tests never see. So we treat the generated references as build output and check them in CI with four suites. Each owns one concern, so a failure points to one file. Two of them walk the catalog directly, which means a new contract is covered the moment it’s registered.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3Qt-qBqRO26Ohzypl72z2Q.png" /><figcaption>Four suites, each owning one concern.</figcaption></figure><p>The lint suite failed on its very first run. One published document carried a relative documentation URL, which OpenAPI forbids and our old validation had never caught. Other rules close risks before they happen. Operations are keyed by method and path, so two contracts claiming the same route would silently overwrite each other and drop an endpoint from the reference. CI now rejects that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lIEuQJAnm7Yg3zY_QDEmhQ.png" /><figcaption>Give a request example a numeric name where the schema says string, and CI fails before the reference can ship.</figcaption></figure><h3>What the migration found</h3><p>We write every contract from the controller and the SQL behind it, never from the old reference. That rule turned the migration into an audit:</p><ul><li><strong>A validator guarding the wrong field.</strong> Session updates were validated against state.settings.metadata, which nothing reads. A client that satisfied the validator got sessions listed with a null type and no document attachment on read.</li><li><strong>Responses no one had described.</strong> An update endpoint could already return 404 and 409. Neither appeared anywhere in the reference.</li><li><strong>A crash on a normal input.</strong> A report-status endpoint returned 500 whenever a report had no steps, which is the normal state for some report types.</li><li><strong>A flag that couldn’t be turned off.</strong> A boolean query flag was read with Boolean(value), so the string &quot;false&quot; came out true.</li><li><strong>An endpoint documented at the wrong address.</strong> The reference placed it under a base URL that never served it.</li><li><strong>A disclosure caught before publishing.</strong> A field typed as an enum would have listed other customers’ report types in another customer’s reference. We publish it as a plain string, a reminder that audience scoping applies to values as well as fields.</li></ul><h3>Migrating without stopping the line</h3><p>We never stopped to rewrite. Old and new validation run side by side, and we ordered the work so it paid off early.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ON9xJvNez2Ba-lab9suBcg.png" /><figcaption>The migration, from first contract to cutover.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dvl9pp2ggIZ5lkriUe7IeA.png" /><figcaption>Endpoints defined as contracts, from the foundation on 24 August to today, counted from the repository history. The curve climbs in steady steps, with steeper runs when a whole product area moved across.</figcaption></figure><h3>Trade-offs and limits</h3><ul><li><strong>Two validators during migration.</strong> Until the last route moves, every engineer works with both the old and the new style, sometimes in the same file.</li><li><strong>Extra fields pass.</strong> Our response schemas accept fields they don’t declare, so an undocumented field won’t trip the check. An optional field that never appears passes too, which means a misnamed optional field can slip through. A strict schema closes both gaps.</li><li><strong>Not every response is checked.</strong> Streamed and binary responses are documented but never verified at runtime.</li><li><strong>The production check isn’t free.</strong> It runs off the request’s path, but parsing still uses the same event loop. If it ever shows up in profiles for the largest payloads, we’ll sample.</li></ul><h3>What we learned</h3><ul><li><strong>We were specifying responses for the first time.</strong> Our validators had only ever looked at requests, so most of what the migration found was hiding in responses.</li><li><strong>Treat the reference as build output.</strong> A contract can be inconsistent while every endpoint test passes. Only linting the generated document catches that.</li><li><strong>Observe in production, enforce in tests.</strong> Enforcing response schemas in production on routes with years of history would have turned old drift into customer-facing errors. Observing first let real traffic show us where data and contracts disagreed, while strict tests kept new drift out.</li><li><strong>A convention holds only where a test enforces it.</strong> Rules that lived in review comments drifted, just as the old reference had. Rules written as tests held for every contributor and every coding agent.</li><li><strong>Validation decides well-formedness, nothing more.</strong> Ownership, quotas and permissions stay in middleware, where they map to 401, 403 and 404. That way a 400 from our API means exactly one thing.</li></ul><h3>What’s next</h3><p>Chat is the last group the legacy reference documented. When it moves, customers switch to the generated references and we delete the hand-maintained file, its overrides and the legacy route. From then on, every API reference Needl.ai publishes comes from the code that serves it.</p><p>Our generated reference already says sentiment_link. We wrote it from the code that sends it, not from the file that got it wrong.</p><h3>FAQ</h3><p><strong>Does this approach require TypeScript?</strong><br>No. Our backend is CommonJS JavaScript, and Zod and zod-openapi both run there without a build step.</p><p><strong>Why not write the OpenAPI document first and validate against it?</strong><br>That keeps a hand-written specification at the centre, which is the problem we set out to remove. With contracts, the specification is something the code produces.</p><p><strong>How do you keep examples in the reference accurate?</strong><br>CI checks every example, request and response, at every depth and status code, against the schema it illustrates. An example that contradicts its schema fails the build.</p><p><strong>Can I use this pattern in my own Express app?</strong><br>Yes. You need Zod 4, zod-openapi and a small middleware that validates params, query and body against a contract and wraps res.json. Ours is about 100 lines. Start with one route, and add the OpenAPI lint to CI before the second.</p><p><strong>What happens when an endpoint’s shape needs to change?</strong><br>You change its contract. Validation, the response check and every reference that publishes the endpoint pick up the change in the same commit. If the change alters what an audience publishes, the audience tests fail until you update them, which makes the change visible in review.</p><p><strong>Can readers try requests from the reference?</strong><br>Yes. Each reference renders in Swagger UI with “Try it out”, and the examples it shows are the same ones CI validates against the schema.</p><p><strong>What does a new customer-specific reference cost?</strong><br>One audience file listing its groups, server and security schemes, plus one registry entry. The builder and the test suites cover it from there.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=453207c72047" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ From Chatbot to Context Engine: Building AskNeedl’s Remote MCP Server]]></title>
            <link>https://medium.com/@sushanthk07dev/from-chatbot-to-context-engine-building-askneedls-remote-mcp-server-759eb503e829?source=rss-dbc26777c21c------2</link>
            <guid isPermaLink="false">https://medium.com/p/759eb503e829</guid>
            <category><![CDATA[fastmcp]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <category><![CDATA[sse]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[fastapi]]></category>
            <dc:creator><![CDATA[Sushanth]]></dc:creator>
            <pubDate>Mon, 19 May 2025 19:24:41 GMT</pubDate>
            <atom:updated>2025-05-19T19:24:41.572Z</atom:updated>
            <content:encoded><![CDATA[<h4>How we turned AskNeedl into a composable, async-first context provider for LLMs — powered by Remote Model Context Protocol and FastAPI.</h4><h3><strong>Introduction</strong></h3><p>When we built <strong>AskNeedl </strong>at needl.ai, our flagship Q&amp;A chatbot at Needl, the early results were promising. It could dig deep into user-linked data sources — annual reports, presentations, exchange filings — and produce insightful answers using RAG and structured prompts.</p><blockquote><em>But we quickly hit a wall: </em>prompt quality was everything.</blockquote><p>If the prompt wasn’t perfectly framed, or if we didn’t guide the model through a series of intermediate reasoning steps, the output would drop in quality fast. What we needed was a system that could break down, reframe, and enrich user queries dynamically.</p><p>That’s when we started exploring <a href="https://modelcontextprotocol.io/"><strong>Model Context Protocol (MCP)</strong></a>. By exposing AskNeedl’s core capabilities (tools, prompts, data access) via a fully async, SSE-capable MCP server, we enabled external clients like Claude to:</p><ul><li>Discover and invoke Needl’s tools dynamically</li><li>Inject structured prompts and intermediate queries</li><li>Construct context-aware answers programmatically</li></ul><p>With this architecture, <em>we unlocked the full power of AskNeedl,</em> not just as a chatbot, but as a context engine any LLM could query.</p><h3>What is MCP?</h3><p><strong>Model Context Protocol (MCP)</strong> is a specification for exposing prompts, tools, and resources over a network interface.</p><p>In our case, MCP became the way we let outside models like Claude interact with AskNeedl’s engine — treating it less like a chatbot, and more like an extensible Q&amp;A backend.</p><blockquote>From the official <a href="https://modelcontextprotocol.io/introduction">MCP intro</a>:</blockquote><blockquote>“MCP enables agents and runtime environments to fetch model input context dynamically, from structured remote sources.”</blockquote><p>Through MCP, we turned our LLM expertise into <strong><em>remote-callable capabilities</em></strong>. This includes:</p><ul><li>Custom summarizers for feeds, documents defined on needl</li><li>Structured document analysis</li><li>Prompt templates for various Report Generation usecases</li><li><em>and so on…</em></li></ul><h3>Why Remote MCP?</h3><h4><strong>AskNeedl Before MCP:</strong></h4><ul><li>Prompt logic was bundled in our app backend</li><li>Context generation wasn’t composable</li><li>External models couldn’t leverage AskNeedl capabilities</li></ul><p>AskNeedl worked great when <strong><em>we</em></strong> controlled the prompt. But if another agent (e.g. Claude) wanted to drive the conversation and tap into our system?</p><p>No clean interface. No discoverable tools. No streaming.</p><h4><strong>AskNeedl After MCP:</strong></h4><p>MCP gave us a structured way to expose:</p><ul><li><strong>Prompt Templates</strong>: Like our `<strong><em>summarize_annual_report</em></strong>` prompt etc</li><li><strong>Tools</strong>: Like `<strong><em>ask_needl_web</em></strong>`, `<strong><em>ask_us_capital_markets</em></strong>`, `<strong><em>ask_india_capital_markets</em></strong>`, `<strong><em>ask_document</em></strong>`, `<strong><em>ask_feed</em></strong>` etc.</li><li><strong>User-context aware APIs</strong>: that return structured data, not plain strings</li></ul><p>This turned AskNeedl from a monolith into a <em>context provider</em>, capable of powering external agents through composable APIs.</p><blockquote>With MCP, AskNeedl became a backend for anyone building smart model workflows — not just our UI.</blockquote><p>The async-first, SSE-powered implementation means external clients can stream responses, invoke tools concurrently, and get partial answers immediately.</p><h3>📊 Architecture Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9OFfjQWua_IFcF_qfS39ew.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GJ6IogxNTWN4EKgKZpa6aQ.png" /></figure><h3>⚡ The Async-First Philosophy</h3><p>In a system where every request might:</p><ul><li>Hit a retrieval engine</li><li>Call external APIs</li><li>Access secured user data</li><li>Return streamed responses</li></ul><blockquote><strong>async<em> </em></strong>is not optional.</blockquote><p>We designed our server so that <strong>every layer is async</strong> — from request parsing to final tool execution — runs on `<strong>asyncio</strong>`. This ensures:</p><ul><li>High concurrency</li><li>Non-blocking I/O</li><li>Smooth streaming over SSE</li></ul><h4>Token Context with `contextvars`</h4><p>We use Python’s `contextvars` to isolate auth context across coroutines:</p><pre>import contextvars<br>token_ctx: contextvars.ContextVar[str] = contextvars.ContextVar(&quot;token&quot;)</pre><p>This allows us to:</p><ul><li>Store the user token at request entry</li><li>Access it later from deep inside a tool or prompt</li><li>Prevent accidental leakage between concurrent requests</li></ul><h3>🔧 Implementation Highlights</h3><h4>Authentication via Headers</h4><p>We use a`<strong>Bearer token</strong>` Authentication header as auth layer to secure tool access.</p><p><strong>Example header:</strong></p><pre>Authorization: Bearer my-api-key</pre><p>Our middleware extracts the token, stores it in an async-safe `<strong>contextvar</strong>`, and validates it. This design ensures:</p><ul><li>Auth is enforced before tool execution</li><li>Token is available across async layers</li><li>Invalid requests fail early with a clear error</li></ul><p><strong>Sample logic:</strong></p><pre>token = request.headers.get(&quot;Authorization&quot;, &quot;&quot;).replace(&quot;Bearer &quot;, &quot;&quot;)<br>token_ctx.set(token)<br>if not token:<br> return JSONResponse({&quot;error&quot;: &quot;Unauthorized&quot;}, status_code=401)</pre><h4>Tool Registration</h4><pre>@mcp.tool(name=&quot;ask_feed&quot;)<br>async def ask_feed(prompt: str, feed_id: str) -&gt; AskNeedlResponse:<br> token = token_ctx.get()<br> return await _ask_needl_request({&quot;prompt&quot;: prompt, &quot;feed_id&quot;: feed_id}, token)</pre><p><strong>Features</strong>:</p><ul><li>Param validation via `<strong>pydantic</strong>`</li><li>Auto-generated OpenAPI-compatible metadata</li><li>Token propagated via async-local storage</li><li>Error handling with decorators (`<strong>_tool_error_handler</strong>`)</li></ul><h4>Prompt Registration</h4><pre>@mcp.prompt(name=&quot;summarize_annual_report&quot;)<br>async def summarize_annual_report(message: str) -&gt; str:<br> return f&quot;{ANNUAL_REPORT_SUMMARY_PROMPT}\n\nUser Input:\n{message}&quot;</pre><p>This lets external agents construct well-scaffolded input prompts by simply passing a message string. The structure, tone, and format are handled remotely.</p><h4>SSE Transport</h4><pre>def create_sse_server(mcp: FastMCP):<br> app = mcp.http_app(path=&quot;/sse&quot;, transport=&quot;sse&quot;, middleware=[…])<br> return Starlette(routes=[Mount(&quot;/&quot;, app=app)])</pre><p>SSE streams tokens or structured payloads as they’re generated. Perfect for:</p><ul><li>Live inference feedback</li><li>Long-running tools</li><li>Reducing user wait time</li></ul><h4>Middleware + Auth</h4><pre>class AuthenticateRequestMiddleware(BaseHTTPMiddleware):<br> async def dispatch(self, request, call_next):<br> token = extract_token(request)<br> token_ctx.set(token)<br> if not validate_token(token):<br>   return JSONResponse({&quot;error&quot;: &quot;Unauthorized&quot;}, status_code=401)<br> return await call_next(request)</pre><ul><li>Protects all routes</li><li>Token stays local to coroutine</li><li>Logging integrated with `contextvars` for traceability</li></ul><h4>Logging</h4><p>We use structured logs throughout:</p><pre>logging.basicConfig(level=logging.INFO, format=&quot;%(asctime)s %(levelname)s %(message)s&quot;)<br>logger.info(f&quot;Calling tool {tool_name} with params: {params}&quot;)</pre><p>Why it matters:</p><ul><li>Tool invocations are auditable</li><li>Performance bottlenecks are visible</li><li>Failures can be correlated with user sessions</li></ul><h4>Best Practices Followed</h4><ul><li>`<strong>httpx.AsyncClient</strong>` for outbound requests</li><li>`<a href="http://twitter.com/wraps"><strong>@wraps</strong></a>` decorators for consistent exception handling</li><li><strong>Timeout tuning</strong> for long RAG flows</li><li>All routes mounted under `/<strong>sse</strong>` for clean transport separation</li><li>Gunicorn config uses `<strong>UvicornWorker</strong>` + thread tuning</li></ul><h3>🤔 Challenges &amp; Learnings</h3><ul><li><strong>Sparse Documentation</strong>: Very little online content or community knowledge around implementing remote MCP. Even LLMs struggled to assist.</li><li><strong>Hard to Debug</strong>: Without clear references, diagnosing transport issues or tool behavior required deep dives.</li><li><strong>Async Everything</strong>: Reliability came from fully embracing async at all layers — transport, tools, HTTP clients, middleware.</li><li><strong>Error Handling</strong>: Carefully wrapped all tool logic to prevent server crashes and propagate clear failure states.</li><li><strong>Frequent Library Updates</strong>: Had to track and adopt updates to `fastmcp` to improve SSE reliability and reduce connection drops.</li><li><strong>Token Management</strong>: Ensuring per-request isolation and secure propagation via `contextvars` was essential for correctness.</li></ul><h3>👨‍💻 Developer Guide: Building Your Own MCP Server</h3><p>This guide is for developers who want to build and deploy their own Remote MCP server. Whether you’re working on internal LLM orchestration, agent frameworks, or enterprise tooling — here’s a clear path.</p><h4>1. Set Up Your Project</h4><pre>mkdir my-mcp-server &amp;&amp; cd my-mcp-server<br>python -m venv venv &amp;&amp; source venv/bin/activate<br>pip install fastapi fastmcp httpx uvicorn starlette</pre><h4>2. Define Token Context</h4><p>Use `contextvars` to store per-request token safely.</p><pre># context.py<br>import contextvars<br>token_ctx = contextvars.ContextVar(&quot;token&quot;)</pre><h4>3. Auth Middleware</h4><pre># middlewares/auth.py<br>from fastapi.responses import JSONResponse<br>from starlette.middleware.base import BaseHTTPMiddleware<br>from context import token_ctx<br><br>class AuthenticateRequestMiddleware(BaseHTTPMiddleware):<br> async def dispatch(self, request, call_next):<br>  token = request.headers.get(&quot;Authorization&quot;, &quot;&quot;).replace(&quot;Bearer &quot;, &quot;&quot;)<br>  token_ctx.set(token)<br>  if not token:<br>   return JSONResponse({&quot;error&quot;: &quot;Unauthorized&quot;}, status_code=401)<br>  return await call_next(request)</pre><h4>4. Initialize FastMCP</h4><pre># server.py<br>from fastapi import FastAPI<br>from fastmcp import FastMCP<br>from starlette.routing import Mount<br>from starlette.applications import Starlette<br>from starlette.middleware import Middleware<br>from middlewares.auth import AuthenticateRequestMiddleware<br>from context import token_ctx<br><br>mcp = FastMCP(&quot;My Remote MCP&quot;)<br><br># Register tools or prompts here (see next step)<br><br>def create_sse_app():<br> mcp_app = mcp.http_app(<br> path=&quot;/sse&quot;,<br> transport=&quot;sse&quot;,<br> middleware=[Middleware(AuthenticateRequestMiddleware)],<br> )<br> return Starlette(routes=[Mount(&quot;/&quot;, app=mcp_app)], lifespan=mcp_app.router.lifespan_context)<br><br>app = FastAPI()<br><br>app.mount(&quot;/&quot;, create_sse_app())<br><br>if __name__ == &quot;__main__&quot;:<br> import uvicorn<br> <br> uvicorn.run(<br>  &quot;server:app&quot;,<br>  host=&quot;0.0.0.0&quot;,<br>  port=CONFIG.PORT,<br>  loop=&quot;asyncio&quot;,<br> )</pre><h4>5. Register Your Tools</h4><pre># tools.py<br>from context import token_ctx<br>from pydantic import BaseModel<br>import httpx<br>class ResponseModel(BaseModel):<br> result: str<br>@mcp.tool(name=&quot;echo_tool&quot;, description=&quot;Echoes the input message.&quot;)<br>async def echo_tool(message: str) -&gt; ResponseModel:<br> token = token_ctx.get()<br> return ResponseModel(result=f&quot;Echo: {message}, Token: {token}&quot;)</pre><h4>6. Run the Server (Production)</h4><pre>#!/bin/bash<br>set -e<br>TIMEOUT=180<br>WORKERS=1<br>THREADS=8<br>PORT=${PORT:-8000}<br>gunicorn -b 0.0.0.0:$PORT \<br> -k uvicorn.workers.UvicornWorker \<br> src.askneedl_mcp_server.server:app \<br> - timeout $TIMEOUT \<br> - workers $WORKERS \<br> - threads $THREADS \<br> - max-requests 1000 \<br> - max-requests-jitter 100 \<br> - log-level &quot;info&quot;</pre><h4>7. Local Testing</h4><p>One way is you can use the <a href="https://modelcontextprotocol.io/docs/tools/inspector">official MCP inspector tool</a> which offers a interactive interface to test and debug your remote MCP server’s liveness, resources, prompts, and tools — all without writing extra code. It supports direct inspection of locally developed or packaged servers via npx, making it ideal for rapid iteration and validation during development.</p><p>Another way to test your MCP server locally especially with LLM agents like Claude, you can use the `mcp-remote` npm package. This acts as a <strong>shim/proxy</strong> that:</p><ul><li>Speaks MCP over `stdio` to the local LLM client</li><li>Connects to your MCP server over HTTP + SSE</li></ul><p>Even if your MCP server is hosted remotely, the Claude desktop app do not allow remote integrations unless you’re on Claude Max plan. However, using `mcp-remote`, you can bridge that gap on any plan.</p><p>Here’s how:</p><pre>{<br> &quot;mcpServers&quot;: {<br>  &quot;my_mcp_server&quot;: {<br>   &quot;command&quot;: &quot;npx&quot;,<br>   &quot;args&quot;: [<br>    &quot;-y&quot;,<br>    &quot;mcp-remote@latest&quot;,<br>    &quot;http://0.0.0.0:8000/sse&quot;,<br>    &quot;8000&quot;,<br>    &quot;--allow-http&quot;,<br>    &quot;--transport&quot;,<br>    &quot;sse-only&quot;,<br>    &quot;--header&quot;,<br>    &quot;Authorization: Bearer ${API_KEY}&quot;<br>   ],<br>   &quot;env&quot;: {<br>    &quot;API_KEY&quot;: &quot;&lt;your_api_key&gt;&quot;<br>   }<br>  }<br> }<br>}</pre><p><strong>How this works:</strong></p><ul><li>`mcp-remote` launches a local stdio listener</li><li>Forwards every MCP request to your actual FastAPI+SSE MCP server</li><li>Handles headers (like `Authorization`) automatically</li><li>Makes your remote tools/prompts callable from Claude</li></ul><p>This is a great way to test your tools locally before deploying them.</p><p>Learn more: <a href="https://www.npmjs.com/package/mcp-remote">npmjs.com/package/mcp-remote</a></p><h4>Tips for Production</h4><ul><li>Use `gunicorn` with `UvicornWorker`</li><li>Monitor memory usage during long SSE sessions</li><li>Use pydantic models for input validation</li><li>Integrate structured logging (JSON format)</li><li>Automate tool discovery with decorators</li></ul><h4>Bonus: Dev Tips</h4><ul><li><strong>Organize by responsibility:</strong> keep tools, prompts, auth, config in separate modules</li><li><strong>Start with one working tool</strong>: prove end-to-end SSE flow early</li><li><strong>Log everything:</strong> especially inputs/outputs at the tool level</li><li><strong>Stay close to the `fastmcp` changelog:</strong> fixes and features ship fast</li><li><strong>Write client simulators</strong>: test from both LLM and raw HTTP/SSE</li></ul><h4>Bonus: Tips for Scaling Further</h4><ul><li>✅ Add observability: log every tool invocation with user/session ID</li><li>✅ Use typed payloads everywhere with `pydantic`</li><li>✅ Validate request tokens against a central auth service</li><li>✅ Add rate-limiting per API key for multi-tenant safety</li><li>✅ Provide tool discovery via OpenAPI schema or `/metadata` endpoint</li></ul><p>These help ensure the MCP server is secure, stable, and extensible for broader use.</p><h3>🤖 Conclusion</h3><p>MCP transformed AskNeedl from a chatbot into a platform.</p><ul><li>Now, external agents can discover and invoke our tools.</li><li>Prompt logic lives where it should — in a managed service.</li><li>Streaming outputs feel fast and usable.</li><li>All of it is async, robust, and production-tested.</li></ul><p>And most importantly: it’s not just about sending better prompts. It’s about <strong>unlocking the full value of your LLM system</strong> by separating concerns, adding structure, and thinking like a protocol designer.</p><p>If you’re building LLM infra, it might be time to ship your own MCP server.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=759eb503e829" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>