<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Netanel Avraham on Medium]]></title>
        <description><![CDATA[Stories by Netanel Avraham on Medium]]></description>
        <link>https://medium.com/@netanelavraham?source=rss-e7d782cfe630------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ePzo2SHL0RfjV4l_ye3mzA.png</url>
            <title>Stories by Netanel Avraham on Medium</title>
            <link>https://medium.com/@netanelavraham?source=rss-e7d782cfe630------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 16:38:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@netanelavraham/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Build an LLM Gateway for Token Quotas with the AI SDK]]></title>
            <link>https://medium.com/elementor-engineers/your-ai-application-shouldnt-know-how-quota-works-936333136ded?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/936333136ded</guid>
            <category><![CDATA[vercel-ai-sdk]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[ai-infrastructure]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Sun, 26 Jul 2026 13:09:17 GMT</pubDate>
            <atom:updated>2026-09-24T15:51:40.915Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*ovKDUsPBZAkonblIhSYdkQ.png" /></figure><h4>One gateway owns quota and billing. The app just calls the model.</h4><h3>When quota becomes an application problem</h3><p>Every AI feature in our product runs on subscription credits. Each model call has to land on the right subscription, feature, and user, and deduct the right amount.</p><p>That is easy to describe. It is hard to keep true when you have to maintain several LLM services.</p><p>Usage reporting lived inside application code. Every new service repeated the same work: hook into the stream, calculate token usage, map tokens to credits, call the quota API, and handle interrupted streams.</p><p>It became copy-paste infrastructure. Fixes in one service did not reach others, and new AI surfaces could ship without quota wiring.</p><p>We did not build a gateway because centralized architecture sounded nice. Existing gateways already solve routing and provider access. We built it because quota was becoming a problem every application had to solve again.</p><p>The split is simple. <strong>The app passes attribution context</strong>: who is calling, which feature, which run. <strong>The gateway owns</strong> subscription lookup, eligibility check, token-to-credit math, and usage reporting.</p><p>The app should not know whether quota is checked before the call or settled after. It just makes the LLM call.</p><h3>Calling an LLM is easy. Charging the right subscription is not.</h3><p>The AI SDK is good at calling models. It does not track usage against your product’s subscription system.</p><p>With streamText, token counts are not available until the response completes. Every caller that wants accurate quota has to wait for that, convert usage into billing units, attach subscription and feature context, report without blocking the response, and handle interrupted streams the same way everywhere.</p><p>That is not a simple wrapper. It is easy to get subtly wrong, and every LLM service has to get it right on its own.</p><p>We were not fighting the SDK. We were fighting the gap between “call a model” and “charge the right subscription for that call.”</p><h3>Why existing solutions were not enough</h3><p>The obvious first fix is a shared npm package: helper functions, a standard metadata shape, maybe a reportQuotaAfterStream() utility. That helps. It did not solve the problem.</p><p>A package standardizes behavior when you use it, but it cannot enforce the path. Any service with provider credentials can bypass it accidentally.</p><p>If your only problem is duplicated boilerplate, use a package. We had a different requirement: every completed model call needed to pass through a centralized usage path. A library could make the right thing easier, but it could not enforce it.</p><p>We then looked for something we could adopt rather than build.</p><p>The Vercel AI SDK does not provide subscription quota management. Vercel AI Gateway handles provider routing and credentials, but it does not own your product concepts: subscription IDs, feature attribution, or custom billing rules.</p><p>An OpenAI-compatible proxy was another option, but it introduced a different problem. The AI SDK’s model interface is not the same as an OpenAI HTTP API. Tools, streaming, and structured outputs would need to be translated into OpenAI-style requests and back. A custom /generate endpoint would work, but every application would stop calling streamText.</p><p>We needed <strong>every call to go through a centralized quota</strong> path without changing how applications use the AI SDK. Nothing we evaluated did both.</p><h3>The trick: don’t build another API. Build a model.</h3><p>Fixing quota properly meant rewriting every LLM surface we already had, not patching the next copy. We did not add another HTTP API for apps to call. <strong>We made the gateway implement the SDK’s model interface instead.</strong> The gateway behaves like a model from the application’s perspective.</p><p>We wanted every application to call LLMs through one centralized gateway while keeping the existing AI SDK programming model: streamText, tools, structured outputs, and multi-step agent loops. Those goals sound incompatible.</p><p>A gateway wants to sit between every app and the provider. The AI SDK wants to call a model through its own abstraction.</p><p>You are not building an LLM API. <strong>You are implementing a model provider.</strong> The application-side package implements the AI SDK’s <a href="https://ai-sdk.dev/providers/community-providers/custom-providers">LanguageModel</a> interface.</p><pre>streamText()<br>  → LanguageModel (gateway client)<br>  → HTTP POST (/v1/generate or /v1/stream)<br>  → LLM gateway<br>      ┌─ before call ─────────────────┐<br>      │ metadata enrichment           │<br>      │ quota eligibility check       │<br>      └───────────────┬───────────────┘<br>                      ↓<br>              upstream model → Provider<br>                      ↓<br>      ┌─ after call ──────────────────┐<br>      │ token usage from completed call │<br>      │ quota consume POST (async)      │<br>      └───────────────────────────────┘</pre><p>The gateway is not an API your application calls. <strong>It is the model your application calls.</strong></p><h3><strong>Keep the application code the same</strong></h3><p>Before:</p><pre>const result = streamText({<br>  model: provider(&#39;gemini-3-flash-preview&#39;),<br>  system: systemPrompt,<br>  messages,<br>  tools,<br>});</pre><p>After:</p><pre>const result = streamText({<br>  model: llmGateway.provider(&#39;vertex/gemini-3-flash-preview&#39;),<br>  system: systemPrompt,<br>  messages,<br>  tools,<br>  providerOptions: {<br>    llmGateway: {<br>      metadata: { userId, feature, runId },<br>    },<br>  },<br>});</pre><p>The application still calls streamText. It does not know where credentials live, how usage becomes quota, or which service receives the usage event. The app requests a model by ID; the gateway decides whether it is allowed and maps it upstream. The metadata in providerOptions is <strong>attribution context, not quota logic</strong>.</p><p>That follows the <a href="https://ai-sdk.dev/docs/ai-sdk-core/middleware#configuring-per-request-custom-metadata">SDK pattern for per-request middleware metadata</a>: the app passes context in providerOptions, and gateway middleware reads it on the other side of the call. The gateway looks up the subscription server-side and runs the model only if the subscription still has credits.</p><h3>Turning an in-process interface into a network protocol</h3><p>The move is implementing LanguageModel across the network, not inventing a custom LLM API.</p><p>LanguageModel is a runtime interface, not a wire protocol. We had to decide which call options serialize cleanly, how stream parts encode over SSE, how errors cross the boundary, and how abort signals propagate.</p><pre>Application<br>  ↓ LanguageModelCallOptions<br>  ↓ serialize supported fields<br>JSON / SSE protocol<br>  ↓<br>reconstruct upstream call<br>  ↓ LanguageModel</pre><p>Only data crosses the wire, not live SDK objects. Callbacks, streams, and provider handles stay in the process that created them. The client serializes the model call to JSON and POSTs it. The gateway parses that JSON and runs a normal upstream AI SDK call.</p><p>HTTP failures come back as model errors the client SDK already understands. Stream failures show up as errors on the ReadableStream the application consumes.</p><p>Generate: POST /v1/generate. Stream: POST /v1/stream, SSE frames parsed into LanguageModelStreamPart. Abort propagates from client to upstream.</p><pre>async doStream(params) {<br>  const response = await postJson(&#39;/v1/stream&#39;, buildRequestBody(params), params.abortSignal);<br>  // Parse SSE frames from the gateway into LanguageModelStreamPart chunks.<br>  return { stream: sseToLanguageModelStream(response.body) };<br>}</pre><p>Paths are versioned. Client and server share a protocol package.</p><p>This is not a provider-neutral protocol. It is AI SDK-native. We traded protocol independence for keeping the application on streamText.</p><h3>Move quota from application code into middleware</h3><p>We moved usage reporting from application hooks into gateway middleware.</p><p>Before:</p><pre>Application → call model → wait for usage → POST quota</pre><p>After:</p><pre>Application<br>  → LanguageModel (gateway client)<br>  → HTTP POST (/v1/generate or /v1/stream)<br>  → LLM gateway<br>          → wrapLanguageModel<br>              → quota middleware   ← reports usage after the call<br>              → upstream model</pre><p>It never waits on usage or talks to the quota API. On the gateway server, the upstream model is wrapped with <a href="https://ai-sdk.dev/docs/ai-sdk-core/middleware">wrapLanguageModel</a>, the SDK&#39;s built-in hook for middleware around doGenerate and doStream:</p><pre>// LLM gateway<br>const model = wrapLanguageModel({<br>  model: upstream,<br>  middleware: [metadataMiddleware, quotaMiddleware],<br>});</pre><pre>return model.doStream(callOptions);</pre><p>On <strong>generate</strong> calls, middleware reports usage after doGenerate returns with token counts.</p><p>On <strong>stream</strong> calls, middleware passes chunks through immediately, watches for the finish event, converts usage into billing units, and POSTs to the quota service without blocking the response.</p><p>Subscription lookup and metadata enrichment still happen in the gateway before the upstream call. Usage reporting happens after it, in middleware that every model invocation shares. Quota rules can change inside the gateway without touching application call sites.</p><h3>The gateway becomes the AI infrastructure layer</h3><p>Quota was the first requirement. It is not the only thing that belongs between the app and the provider.</p><p>Every model call already passes through the gateway’s wrapLanguageModel middleware stack.</p><p>That is the hook for cross-cutting concerns you do not want copy-pasted into every service. The same stack that reports usage can host other layers:</p><ul><li><strong>Observability</strong> — Langfuse traces, structured logging, request correlation</li><li><strong>Stream lifecycle</strong> — resumable streams, disconnect handling, replay after a dropped connection</li><li><strong>Policy</strong> — rate limits, audit logging, content filters</li></ul><p>Each layer wraps doGenerate and doStream once. Applications keep calling streamText. They do not know which layers run or in what order.</p><p>Today, our gateway routes directly to provider adapters:</p><pre>Application → LanguageModel proxy → LLM gateway → Provider</pre><p>Over time, we can move provider routing and credentials behind Vercel AI Gateway:</p><pre>Application → LanguageModel proxy → LLM gateway → Vercel AI Gateway → Provider</pre><p>The application-facing contract does not change. Quota, attribution, and subscription policy remain in our gateway, while provider connectivity becomes an infrastructure concern.</p><h3>The tradeoffs</h3><ul><li><strong>Extra hop.</strong> Every call goes app → gateway → provider. That adds a few ms on our internal network. Model latency is seconds, so we have not felt it.</li><li><strong>Single point of failure.</strong> If the gateway is down, callers are down. Apps hold no provider credentials, so there is no direct fallback.</li><li><strong>Coupled deploys.</strong> Client and gateway share a protocol package. Changes to request or stream encoding usually ship together until that format stabilizes.</li><li><strong>Best-effort quota.</strong> Credits are checked before the call. Usage is POSTed after the model finishes, without blocking the response. If that POST fails, the user still has the answer.</li></ul><p>Skip this if you have one service, one model, and no quota to enforce.</p><p>Build it when token quota is a product requirement across multiple callers, and you need unchanged streamText call sites.</p><h3>The application should only know what it needs to know</h3><p>We did not add quota to the application. We made the gateway the model.</p><p>The application still knows who is calling and what feature is running. It still chooses a model and calls streamText. It does not know where provider credentials live, how subscription eligibility is checked, or how tokens become credits.</p><p>That is the boundary we wanted: the application speaks the AI SDK, and the gateway owns everything that happens around the model call.</p><p>The gateway is not another API for calling models. It is the model.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=936333136ded" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/your-ai-application-shouldnt-know-how-quota-works-936333136ded">How to Build an LLM Gateway for Token Quotas with the AI SDK</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Shipping MCP Apps in Production]]></title>
            <link>https://medium.com/elementor-engineers/shipping-mcp-apps-in-production-45dab11b358c?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/45dab11b358c</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[frontend-architecture]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[mcp-app]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Mon, 04 May 2026 13:45:19 GMT</pubDate>
            <atom:updated>2026-05-05T06:45:29.592Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TmsB-CqTBS3TDnxQwXEjqw.png" /></figure><h4>Building the host side: from inline HTML to React apps</h4><blockquote><strong>Note:</strong> This post assumes you’ve already played with MCP Apps and focuses on the host-side implementation — rendering, sandboxing, and communicating with MCP Apps. If you’re building an MCP App itself (the UI inside the iframe), check the <a href="https://modelcontextprotocol.io/extensions/apps/build">official build guide</a>.</blockquote><p>The first MCP App feels easy. The spec points you at ui://, you ship some HTML, done. Then you add a second flow. Then a third. And suddenly the problem isn’t MCP — it’s that your UI is just… strings inside your server.</p><p>It worked fine until the UI stopped feeling like UI.</p><p>Changing a label meant shipping the MCP server again. Adding a new flow meant more inline HTML, more deployment friction, and slower iteration every time.</p><h3>Why Inline HTML Broke at Scale</h3><p>The MCP Apps spec defines a ui:// resource scheme<em>, </em>servers register HTML resources, hosts fetch and render them.</p><p>The first implementation followed the spec exactly: inline HTML/CSS/JS strings bundled in the MCP server.</p><pre>// What we started with<br>const clarifyQuestionsAppHtml = `&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;&lt;style&gt;${CSS_STRING}&lt;/style&gt;&lt;/head&gt;<br>&lt;body&gt;<br>  ${HTML_STRING}<br>  &lt;script&gt;${JS_STRING}&lt;/script&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;<br>`;</pre><p>It worked, until it really didn’t.</p><ul><li><strong>No TypeScript</strong> → No types. Just strings.</li><li><strong>No design system</strong> → Every component is custom, no shared tokens.</li><li><strong>No hot reloading</strong> → Change = rebuild the MCP server</li><li><strong>Poor DX</strong> → Every UI tweak feels like backend work</li></ul><p>The approach didn’t scale.</p><p>By the third app, it was obvious that UI can’t stay as strings bundled with the MCP server, not if you want fast iteration and more than one app.</p><h3>Same-Origin React Apps</h3><p>The solution: serve first-party MCP Apps as same-origin pages instead of inline HTML strings.</p><p>The sequence works like this: the MCP server isn’t what loads /mcp-apps/*. The tool result carries resourceUri. The host sets the iframe&#39;s src, and the browser fetches same-origin routes from there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z7j_pXZpXG5PVeb6ODBEIA.png" /></figure><p><strong>Two paths for MCP Apps:</strong></p><ul><li><strong>Third-party apps</strong>: Host fetches HTML via ui:// resource, renders in srcdoc iframe</li><li><strong>First-party apps</strong>: Host loads same-origin URL directly in iframe src</li></ul><p>For first-party apps, resourceUri points to a same-origin Next.js route instead of a ui:// resource:</p><pre>// Tool definition on the MCP server<br>// HOST_ORIGIN = your app&#39;s public origin, e.g. process.env.HOST_ORIGIN<br>server.registerTool(&#39;ask-clarifying-questions&#39;, {<br>  description: &#39;Ask the user clarifying questions before proceeding&#39;,<br>  inputSchema: {<br>    questions: z.array(questionSchema).min(1).max(4)<br>      .describe(&#39;Array of questions to ask the user&#39;),<br>  },<br>  _meta: {<br>    ui: { resourceUri: `${HOST_ORIGIN}/mcp-apps/questions` }<br>  },<br>}, async ({ questions }) =&gt; {<br>  return {<br>    content: [{ type: &#39;text&#39;, text: JSON.stringify({ success: true, questions }) }],<br>  };<br>});</pre><p>The host detects same-origin URLs (by comparing the URL origin to the host origin) and adjusts the iframe sandbox accordingly. The app is just a React component using the ext-apps SDK. We use Next.js because it’s our existing stack. A static React build works equally well.</p><h3>Why SSR Breaks MCP Apps</h3><p>Skip { ssr: false } and the app crashes immediately. MCP Apps communicate via postMessage with the parent frame, but during server-side rendering, there is no parent frame. Any reference to window.parent throws. Anything that touches window at module import time fails for the same reason, often before useApp mounts.</p><p>Each MCP App route uses this pattern:</p><pre>// src/app/mcp-apps/questions/page.tsx<br>&#39;use client&#39;;<br>import dynamic from &#39;next/dynamic&#39;;</pre><pre>const ClarifyQuestionsApp = dynamic(<br>  () =&gt; import(&#39;./components/ClarifyQuestionsApp&#39;).then(mod =&gt; mod.ClarifyQuestionsApp),<br>  { ssr: false },<br>);</pre><pre>export default function QuestionsPage() {<br>  return &lt;ClarifyQuestionsApp /&gt;;<br>}</pre><p>The dynamic() wrapper also gives you code splitting for free — each app is its own chunk, loaded only when that tool fires. The main bundle doesn’t pay for apps the user never triggers.</p><h3>The App Component Pattern</h3><p>Here’s the actual pattern we use for MCP Apps:</p><pre>// src/app/mcp-apps/questions/components/ClarifyQuestionsApp.tsx<br>import { useCallback, useRef, useState } from &#39;react&#39;;<br>import type { App } from &#39;@modelcontextprotocol/ext-apps&#39;;<br>import { useApp } from &#39;@modelcontextprotocol/ext-apps/react&#39;;<br><br>export function ClarifyQuestionsApp() {<br>  const [questions, setQuestions] = useState&lt;Question[]&gt;([]);<br>  const [completed, setCompleted] = useState(false);<br>  <br>  // Tool hooks need a stable App reference across renders<br>  const appRef = useRef&lt;App | null&gt;(null);<br><br>  const { error } = useApp({<br>    appInfo: { name: &#39;ClarifyQuestionsApp&#39;, version: &#39;1.0.0&#39; },<br>    capabilities: {},<br>    onAppCreated: (app) =&gt; {<br>      appRef.current = app;<br><br>      app.ontoolinput = async ({ arguments: args }) =&gt; {<br>        const parsed = args as { questions?: Question[] };<br>        if (parsed?.questions) setQuestions(parsed.questions);<br>      };<br><br>      // Restore after refresh — host replays saved tool result on init<br>      app.ontoolresult = async ({ _meta }) =&gt; {<br>        if (_meta?.appStatus === &#39;completed&#39;) setCompleted(true);<br>      };<br>    },<br>  });<br><br>  const handleSubmit = useCallback(async (answers: AnswersMap) =&gt; {<br>    if (!appRef.current) return;<br><br>    await appRef.current.sendMessage({<br>      role: &#39;user&#39;,<br>      content: [{ type: &#39;text&#39;, text: JSON.stringify(answers) }],<br>    });<br>    setCompleted(true);<br>  }, []);<br><br>  if (error) return null;<br>  if (completed) return null;<br>  if (questions.length === 0) return null;<br><br> // QuestionWizard is your form component — it renders the questions and collects answers.<br>  return &lt;QuestionWizard questions={questions} onSubmit={handleSubmit} /&gt;;<br>}</pre><p>This pattern wires up init, tool input, and restore paths. User actions go back through the ext-apps SDK.</p><p>Once the apps were running, the next problem was communication.</p><h3>Where the Spec Meets Real Product UX</h3><p>The SDK gives you two ways for apps to communicate back:</p><ul><li>sendMessage() sends a chat message that appears in conversation and can trigger AI follow-ups</li><li>callServerTool() calls a tool on the MCP server to fetch data or trigger server-side actions</li></ul><p>But a deployment confirmation flow needed something different: tell the host to reload the page after applying changes. That’s not a chat message, it shouldn’t appear in conversation, and it’s not a server action — the server has no way to reload the host’s page.</p><p>We needed host-side signals: out-of-band actions that the host intercepts and handles without involving chat or the server.</p><p>Since ext-apps doesn’t have sendSignal() yet, we encode signals as prefixed JSON inside sendMessage and intercept them on the host before they reach the chat:</p><pre>// mcp-app-signals.ts<br>const SIGNAL_PREFIX = &#39;[SIGNAL]:&#39;;<br><br>export enum SignalType {<br>  STATUS_CHANGE = &#39;status_change&#39;,<br>}<br><br>export const createSignal = (type: SignalType, payload: object): string =&gt;<br>  `${SIGNAL_PREFIX}${JSON.stringify({ type, ...payload })}`;<br><br>export const parseSignal = (text: string) =&gt; {<br>  if (!text?.startsWith(SIGNAL_PREFIX)) return null;<br>  return JSON.parse(text.slice(SIGNAL_PREFIX.length));<br>};<br><br>// In the app: fire a page reload without a real chat message<br>await app.sendMessage({<br>  role: &#39;user&#39;,<br>  content: [{ <br>    type: &#39;text&#39;, <br>    text: createSignal(SignalType.STATUS_CHANGE, { status: &#39;reload_required&#39; })<br>  }],<br>});</pre><p>The host intercepts the prefixed string before it reaches the chat:</p><pre>// Host: strip signals before they hit the conversation<br>function handleAppMessage(text: string) {<br>  const signal = parseSignal(text);<br>  if (signal) {<br>    executeSignalAction(signal); // e.g. reload the page<br>    return;<br>  }<br>  appendToChat(text);<br>}</pre><p>This pattern came from production requirements the spec doesn’t cover yet. It’s simple enough to replace once native signal support lands.</p><p>With communication solved, the next challenge was timing.</p><h3>The Timing Problem</h3><p>When applying changes, we show a confirmation button. The pain wasn’t just long responses — it was timing. In agentic flows with several tool calls in one turn, if the UI mounts while more calls are still pending, users can confirm before the full plan finishes. We needed a way to hold the UI until the turn is done, the agent stops chaining tools, the user sees the full explanation, and only then surfaces the action.</p><blockquote><em>So we added one small thing — </em><em>renderTiming, with two options:</em></blockquote><pre>| renderTiming  | When it shows                       | Typical use           |<br>|---------------|-------------------------------------|-----------------------|<br>| inline        | As soon as the tool runs            | Questions, forms      |<br>| end-of-turn   | After the agent finishes the turn   | Confirmation buttons  |</pre><pre>server.registerTool(&#39;apply-pending-changes&#39;, {<br>  description: &#39;Apply pending changes and prompt for confirmation&#39;,<br>  inputSchema: {<br>    changeId: z.string().describe(&#39;Unique identifier for the pending change&#39;),<br>  },<br>  _meta: {<br>    ui: {<br>      resourceUri: `${HOST_ORIGIN}/mcp-apps/confirm`,<br>      renderTiming: &#39;end-of-turn&#39;,<br>    },<br>  },<br>}, async ({ changeId }) =&gt; {<br>  await applyChanges(changeId);<br>  return { content: [{ type: &#39;text&#39;, text: &#39;Changes applied to preview.&#39; }] };<br>});</pre><p>Users read the full response first, then see the action button.</p><p><strong>Spec contribution:</strong> This field is not part of the MCP Apps spec yet. I opened <a href="https://github.com/modelcontextprotocol/ext-apps/pull/553">PR #553</a><strong> </strong>to propose adding it, check it out if you’re building a host.</p><h3>State Persistence</h3><p>Users refresh pages mid-flow. If the app loses state, they’ll either retry (best case) or leave (worst case).</p><p>We persist appState alongside the message. On iframe init, replay saved tool input/result once in oninitialized so you don’t duplicate handshakes across reloads.</p><p>Host side — replay saved state when the iframe comes back:</p><pre>// Host side: replay saved state<br>bridge.oninitialized = () =&gt; {<br>  // Send saved state back to the app<br>  bridge.sendToolInput({ arguments: savedToolInput });<br>  bridge.sendToolResult({ <br>    content: savedToolResult, <br>    _meta: { appStatus: savedStatus, appResult: savedResult } <br>  });<br>};<br><br>// App side: check restored status<br>createdApp.ontoolresult = async (result) =&gt; {<br>  const status = result._meta?.appStatus;<br>  <br>  if (status === &#39;completed&#39;) {<br>    setCompleted(true);<br>    return;<br>  }<br>  <br>  if (status === &#39;reload_required&#39;) {<br>    setStatus(&#39;pending&#39;);<br>  }<br>};</pre><p>The app reads appStatus to decide what to show: skip the reload prompt if they already passed it, or show confirmation when still pending.</p><p>One more production concern: security.</p><h3>Two Sandbox Modes</h3><p>Sandbox isn’t one-size-fits-all. We ended up with two modes depending on where the app comes from:</p><p><strong>Third-party apps (inline HTML):</strong></p><pre>&lt;iframe sandbox=&quot;allow-scripts&quot; srcdoc=&quot;...&quot; &gt;&lt;/iframe&gt;</pre><ul><li>No allow-same-origin → runs in a null origin</li><li>No access to cookies, storage, or the parent</li><li>Safe default for anything you don’t fully control</li></ul><p><strong>First-party apps (our React routes):</strong></p><pre>&lt;iframe sandbox=&quot;allow-scripts allow-same-origin&quot; src=&quot;...&quot;&gt;&lt;/iframe&gt;</pre><ul><li>Needs allow-same-origin for React, fonts, and normal browser behavior</li><li>Shares origin with the host → cookies, storage, everything</li><li>Treat this as fully trusted code only</li></ul><p><strong>Security note: </strong>Always validate postMessage origins strictly in both modes.</p><p>Same-origin iframes share cookie context with the host whenever the iframe URL matches your site’s origin. The sandbox attributes are not a substitute for a proper CSP boundary — see the <a href="https://github.com/modelcontextprotocol/ext-apps/blob/main/docs/csp-cors.md">CSP &amp; CORS guide</a> for configuring connectDomains, resourceDomains, and stable origins for apps that make network requests.</p><h3>Results</h3><p>The clarification questions flow that started this refactor is now three React components, hot-reloadable, wired into the design system.</p><p>It finally feels like frontend work again — instead of carving UI out of server strings.</p><p>Deployment confirmation went from messy string manipulation to a typed React component with explicit state. Adding new apps is now just frontend work: route, component, tool registration</p><h3>Key Takeaways</h3><ul><li>Same-origin &gt; inline HTML when you control the app</li><li>{ ssr: false } or your app crashes</li><li>Signals = hack until the spec catches up</li><li>renderTiming fixes premature UI</li><li>Persist state or users lose the flow on refresh</li></ul><p>The spec gives you a solid foundation. Production fills in the rest.</p><p>MCP Apps let you ship interactive UI alongside your AI tools — forms, confirmations, wizards — without leaving the chat experience. If you’re building an agent, you’ll hit this sooner or later.</p><p>If you’ve hit similar patterns or found better ones, I’d love to hear about them. Feel free to drop a comment.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45dab11b358c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/shipping-mcp-apps-in-production-45dab11b358c">Shipping MCP Apps in Production</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Token Optimization Strategies for AI Agents]]></title>
            <link>https://medium.com/elementor-engineers/optimizing-token-usage-in-agent-based-assistants-ffd1822ece9c?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffd1822ece9c</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[context-window]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Sat, 02 Aug 2025 15:16:48 GMT</pubDate>
            <atom:updated>2025-08-13T05:48:43.090Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/877/1*dkwquWpKkkYg9u0PuOp4Zw.png" /></figure><h4><strong>Practical tips to reduce token usage in LLM systems without slowing things down</strong></h4><p><strong>LLMs don’t charge by the hour, they charge by the token. And with every wasted token, the cost quietly stacks up</strong></p><p>When your system is juggling dozens of tools, long conversations, and ever-growing context, those tokens add up fast, draining your budget and slowing things down.</p><p>So just before you hit that frustrating context_length_exceeded error, here’s how to optimize token usage without dumbing down the results.</p><h3>Use the Right Model for the Job</h3><p>Need a conversation title or a simple classification? Hand it off to a cheap, fast model like gpt-4.1-nano. Save the heavyweight Claude-4-Opus-Ultra-Pro-Max for the tasks that actually need it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RATa55W2GHwmlLNPIYwxeg.png" /><figcaption><a href="https://pricepertoken.com/">Price per Token</a></figcaption></figure><h3>Leverage <a href="https://platform.openai.com/docs/guides/prompt-caching">Prompt Caching</a></h3><p>Got system prompt parts that never change? Move them to the top. C<strong>ached tokens are 75% cheaper to process</strong>.</p><p>The model efficiently caches the initial tokens of recurring prompts. When input shifts later on (e.g., the user message), only those changed tokens trigger reprocessing (like Docker layers). So, stack the unchanging context up front and let caching do its thing. Free optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*4zfQr7Cth19YIAa6KjkK3w.png" /></figure><h3>Lean Tools, Minimal Prompts</h3><h4>Tools Filtering</h4><p>When the agent picks a tool, it reads <strong>every</strong> available option. Even unused tools get tokenized and billed. Keep the tool list lean and filter based on relevance before passing it to the model.</p><h4>Trim the System Prompt</h4><p>System prompts often contain sections like “when using the email tool…” that only matter sometimes. Conditionally include those parts only when the tool is in the filtered list. No point in burning tokens on instructions for tools the model isn’t using.</p><h3>Think Like a Model</h3><p>Name tools clearly and describe them precisely. Vague names or fuzzy descriptions confuse the model, leading to wrong calls, wasted tokens, and slower responses. Clarity upfront saves retries and your users&#39; patience.</p><p>Want a proven structure? I follow <a href="https://github.com/netanelavr/cursorrules/blob/main/.cursor/rules/mcp-tool-definition.mdc">this rule</a> that defines exactly how to name and describe MCP tools so agents know when and why to use them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_8PdzHgft25xFvmPkkIsyw.png" /><figcaption>What a good tool description includes</figcaption></figure><h3>Don’t YOLO Error Responses</h3><p>Every retry is a fresh request and a fresh charge. Instead of tossing back a generic “400 Bad Request,” return a structured, helpful message: what went wrong, what’s needed, and how to fix it. That reduces retries, saves tokens, and keeps the agent from looping endlessly trying to guess the next move.</p><h3>Context Strategy</h3><h4>Smaller Is Smarter</h4><p>Massive context windows are tempting, but more tokens ≠ better output. In fact, shorter, well-scoped inputs often yield sharper responses. Clear instructions and relevant examples go further than stuffing the whole transcript into the prompt.</p><p>Well-crafted context can even let you use a lighter, cheaper model without sacrificing performance, so you save on both tokens and model tier.</p><h4>Trim Long Conversations Thoughtfully</h4><p>For multi-turn chats, you don’t need to replay the entire saga. Consider:</p><ul><li>Keeping just the last N messages</li><li>Summarizing old turns</li><li>Retaining only key decision points or user instructions</li></ul><p><strong>That said, dropping too much may confuse the model and strip away key context.</strong></p><h3>Output Tokens Are Expensive</h3><p>Each output token costs ~4x more than the input. You can:</p><ul><li>Tell the model to be concise</li><li>Limit response length</li><li>Use max_tokens to set a hard cap</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*BFOMIUB7mGV-GFIp-Os1Kg.png" /></figure><h3><a href="https://platform.openai.com/docs/guides/batch">Batch API</a> for Background Jobs</h3><p>If you’re willing to wait up to 24 hours for results, you can get end-of-season token pricing with a 50% discount and higher rate limits, better suited for jobs, evaluations, etc.</p><h3>Use Structured Outputs</h3><p>Instead of padding the system prompt with verbose example responses, use OpenAI’s <a href="https://platform.openai.com/docs/guides/structured-outputs?api-mode=responses">structured output</a> schemas. You get consistent formatting, fewer hallucinations, and a lot fewer input tokens.</p><h3><strong>Wrapping It Up</strong></h3><p>Optimizing token usage isn’t about cutting corners. It’s about being smart. With the right strategies, you can keep quality high while saving on both cost and latency.</p><p><strong>Got your own token-saving tricks? Drop them in the comments. And if this helped, give it a clap or two :)</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffd1822ece9c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/optimizing-token-usage-in-agent-based-assistants-ffd1822ece9c">Token Optimization Strategies for AI Agents</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The AI Combo That Writes Your Frontend Tests]]></title>
            <link>https://medium.com/elementor-engineers/the-ai-combo-that-makes-frontend-testing-effortless-00690462d0d2?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/00690462d0d2</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[playwright-automation]]></category>
            <category><![CDATA[mcps]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Fri, 30 May 2025 11:10:49 GMT</pubDate>
            <atom:updated>2025-06-07T18:58:21.344Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kH-hkaOKlk8ZTOG3AbM3xw.png" /></figure><h4>How Cursor and Playwright MCP autonomously explored my app, mapped every API call, created mocks, and generated E2E tests in minutes</h4><p><strong>AI that writes code?</strong> That’s already old news<em>.</em></p><p><strong>AI that dives into your app, acts like a QA assistant, maps every click, and spits out ready-to-run frontend E2E tests while you make coffee? That’s exactly what Cursor + Playwright MCP did for me.</strong></p><p>No guidance, no hand-holding. Just Cursor running Claude Sonnet 4 with Playwright MCP, exploring every corner of the app, mapping APIs, capturing network traffic, and transforming those discoveries into smart test scenarios.</p><p>The kind of work that typically takes <strong>days</strong> of coming up with all the test scenarios, creating mocks, and test specs was completed in <strong>minutes</strong> while I wasn’t even watching.</p><h3>“Move It to the Next Sprint”</h3><p>You know the drill: Sprint planning arrives, E2E tests are mentioned, everyone nods knowingly, and then… nothing. There’s always a more urgent ticket. A critical bug. A feature that “just needs to be pushed out first.” Two months later, your app is running in production with no E2E coverage.</p><p>Our AI Site Planner was no different. But instead of moving the “Playwright E2E tests” ticket to the backlog, <a href="https://medium.com/u/730534c1b460">Mati Horovitz</a> came up with yet another great idea: What if we use Playwright MCP to enable AI to autonomously explore the entire app and generate tests without any guidance?</p><h3>The 30-Second MCP Setup</h3><p>Playwright MCP is a lightweight server that turns a browser into an API your LLM can call. Think of it as a USB-C cable connecting Cursor’s “brain” and Chrome’s “hands”, where the LLM “thinks” and the MCP acts.</p><p>Add Playwright MCP to <strong>mcp.json</strong> at your project root (or ~/.cursor/mcp.json):`</p><pre> &quot;playwright&quot;: {<br>      &quot;command&quot;: &quot;npx&quot;,<br>      &quot;args&quot;: [<br>        &quot;@playwright/mcp@latest&quot;,<br>        &quot;--isolated&quot;,<br>       ],<br>      &quot;host&quot;: &quot;localhost&quot;,<br>      &quot;port&quot;: 3000 // where your backend service is running<br>    }</pre><p>That’s it. This tells Cursor how to connect to Playwright MCP, essentially giving AI direct control over a browser. For more details, check the official <a href="https://github.com/microsoft/playwright-mcp">Playwright MCP</a> repo.</p><h3>The Magic Prompt</h3><p>I opened Cursor’s AI chat and gave it this prompt:</p><blockquote>Let’s create playwright tests for the website-planner.</blockquote><blockquote>Start by exploring the app at its home page while recording the network.<br>We’ll probably have auth issues first so once it happens, I’ll handle it and let you know when to continue, wait for my approval.</blockquote><blockquote>Once we’re in, start interacting with it and basically create your own website.</blockquote><blockquote>While doing that, record the network traffic so you’ll understand the APIs and will be able to generate accurate mocks.</blockquote><blockquote>Use the playwright mcp!<br>All local services are already up so no need to run them</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/1*G32GxrDgxog579toe4eJrA.png" /></figure><h3>Handling Authentication</h3><p>Sure enough, we hit the authentication wall immediately and the browser was redirected to a login page, as expected. What was remarkable was that <strong>it paused, acknowledging the instruction to wait for my intervention.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_tBo5suSobj4Fxt8jBgYDA.png" /></figure><p>I entered my credentials. Cursor observed the entire authentication flow and recorded the authentication responses to create realistic mocks later.</p><p><strong>Note: This was running on a staging environment.</strong> <strong>Always use a non-production environment for such tests.</strong></p><h3>YOLO Mode</h3><p>With auth sorted, I flipped on <em>YOLO mode</em> (Cursor runs without asking permission). No “Can I run this tool?”, just pure AI curiosity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t8xFdgoGJzU9djd3hkQrzA.png" /></figure><p>I hit enter and stepped away from my desk. I wanted to see if AI could actually understand my app without me hovering like an overprotective parent, guiding every click and decision, and it worked better than I expected.</p><h3>What Happened Next</h3><p>I came back to find Cursor had completely explored the application. It discovered entry points, started conversation flows, used suggested buttons, and basically tested the entire app, covering all the key interactions, all while recording every network request.</p><p><strong>Here’s the crazy part: Cursor wasn’t just following a script. It was reasoning about what it saw and made its own decisions.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*Wdou4Hd1FFYIvF3rhghhgw.gif" /></figure><p>After exploration, I asked it to summarize everything it learned and create a readme file. Cursor immediately provided a comprehensive overview of the app’s structure and APIs, the perfect foundation for generating tests.</p><h3>Why Network Recording is Essential</h3><p>To understand why Playwright MCP’s network recording capabilities are crucial, it helps to grasp how modern web applications work.</p><p>Modern apps are essentially API clients. Every user interaction kicks off network requests: form submissions, data loading, and real-time updates. Without understanding these API calls, your tests are just clicking buttons in the dark.</p><p>Playwright can intercept and record every network request your application makes: HTTP requests, WebSocket connections for real-time features, static asset loading, and third-party API calls.</p><h3>Why Mocks Matter</h3><p>Your frontend tests shouldn’t depend on a live backend service or external services. why?</p><ul><li><strong>Speed: </strong>Network calls add seconds to test execution</li><li><strong>Reliability: </strong>External APIs can be down or rate-limited</li><li><strong>Consistency: </strong>Live data changes, making tests flaky</li><li><strong>Cost: </strong>Some vendors bill per call</li><li><strong>Environment: </strong>Tests should work offline and in CI/CD</li></ul><p>By recording real network traffic, AI could generate mocks that behave like the real thing. These mocks return realistic response structures with proper status codes and headers, and preserve the complex data relationships and dependencies that exist in your actual APIs.</p><h3>The Results Were Impressive</h3><p>In 15 minutes, while I was making coffee, AI had:</p><ul><li><strong>Mapped 20+ API endpoints</strong></li><li><strong>Figured out the entire authentication flow</strong></li><li><strong>Documented user flows with Mermaid diagrams</strong></li><li><strong>Documented the data models</strong></li><li><strong>Generated realistic mocks for all network interactions</strong></li><li><strong>Created test suites that made sense</strong></li></ul><p><em>See screenshots below (anonymized, of course)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/1*wE51u5SeDVwxuAGoLyn0Kw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/835/1*Mhetwn8hS0Nly09wDOKnUA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*qYgZFrx6lpzBJEz50CcUWw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*VtoyVYs09W1xidqU_MEV0w.png" /></figure><p><strong>Time I would’ve spent:</strong> 2–3 days<br><strong>Time I spent:</strong> 15 minutes + minor tweaks afterward + coffee break</p><h3>Why This Changes Everything</h3><p>The time savings were clear, but for me, the jump in quality is a game-changer. This wasn’t just recording clicks, <strong>AI was taking snapshots, analyzing the codebase, and reasoning about what users would logically do next.</strong></p><p>It would pause at a dropdown, understand the options, reason about which combinations made sense, and then test different scenarios. When AI can explore your app like a human, all while simultaneously reading your code and testing UI, the traditional approach doesn’t just feel slow, it&#39;s outdated.</p><p>This approach is powerful for reducing effort and achieving excellent baseline coverage. Still, <strong>human oversight remains key for complex issues, tricky user paths, or validating specific business rules. Think of it as a force multiplier, not a complete replacement (yet!).</strong></p><h3><strong>Your Turn</strong></h3><p>If you’re thinking, ‘This sounds too good to be true,’ I thought the same thing. But here’s your experiment:</p><ol><li>Copy the <strong>mcp.json</strong> snippet</li><li>Paste the prompt and adjust to your use case</li><li>Handle authentication once</li><li>Enable YOLO mode</li><li>Let AI do its thing</li></ol><p><strong>I’d love to hear about your experiments with this approach!</strong> Drop a comment below with your results, challenges you faced, or creative ways you’ve used Playwright MCP or any other AI tools to improve your testing workflows.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=00690462d0d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/the-ai-combo-that-makes-frontend-testing-effortless-00690462d0d2">The AI Combo That Writes Your Frontend Tests</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Easiest Way to Track Kubernetes Deployments in Grafana]]></title>
            <link>https://medium.com/elementor-engineers/the-easiest-way-to-track-kubernetes-deployments-in-grafana-7e6411978b75?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e6411978b75</guid>
            <category><![CDATA[production]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Sun, 11 May 2025 09:00:06 GMT</pubDate>
            <atom:updated>2025-05-31T11:46:34.016Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iiHGt5jZq-UTARdu2pq-rA.png" /></figure><h4><strong>Automatic deployment annotations with a simple </strong>Prometheus<strong> query</strong></h4><p><strong>Do these questions sound familiar?</strong></p><ul><li>“Did we deploy around that time?”</li><li>“Was this caused by the new version?”</li><li>“Which commit was it?”</li></ul><p>If you’re nodding your head, you’re not alone — being able to connect metrics to deployments <strong>saves critical time</strong> during incidents.</p><h4><strong>The Simple Solution</strong></h4><p>We wanted that visibility in our services without modifying our app, updating CI/CD pipelines, or maintaining extra code. So we found the simplest solution: <strong>A single Prometheus query + a tiny Grafana tweak.</strong></p><p>You get automatic annotations for every new deployment, with a link to the relevant GitHub release. And the best part? If you’re using K8s, <strong>you probably already have the data.</strong></p><h4>How It Works</h4><p>We use this PromQL query to detect deployment changes based on container image tags:</p><pre>changes(<br>  max(<br>    label_replace(<br>      kube_pod_container_info{pod=~&quot;your-api-name.*&quot;, namespace=&quot;your-namespace&quot;, container=&quot;your-container&quot;},<br>      &quot;version&quot;,<br>      &quot;$1&quot;,<br>      &quot;image&quot;,<br>      &quot;.*:(.*)(@sha256.*)?$&quot;<br>    )<br>  ) by (version, container)<br>) &gt; 0</pre><p><strong>What it does -</strong></p><ul><li>Extracts the version tag from your container images</li><li>Groups by version and container</li><li>Uses the <a href="https://prometheus.io/docs/prometheus/latest/querying/functions/#changes">changes()</a> function to detect a new version</li></ul><p>Every time a new version is deployed, this query fires, and Grafana adds an annotation automatically.</p><p>Here’s how it looks in action — you can clearly see a drop in request rate right after a new deployment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ubnD3ZX5hW47lVoThQW-vA.png" /></figure><h4><strong>Where the Data Comes From</strong></h4><p>This relies on kube_pod_container_info metric from <a href="https://github.com/kubernetes/kube-state-metrics/blob/main/docs/metrics/workload/pod-metrics.md?plain=1#L17"><strong>kube-state-metrics</strong></a>, a lightweight Kubernetes add-on that exposes Prometheus metrics based on your cluster’s state.</p><p>If you’re not already using it, you should be — it provides tremendous observability value with minimal overhead.</p><h4><strong>Bonus - </strong>Link Annotations to GitHub Releases</h4><p>In your Grafana annotation, make the title a clickable link to your GitHub release:</p><pre>&lt;a href=&quot;https://github.com/your-org/your-repo/releases/tag/{{version}}&quot;&gt;{{version}}&lt;/a&gt;</pre><p>Now, each deployment annotation links straight to its GitHub release. <strong>No guessing. Just context.</strong></p><p>Here’s how the annotation is configured in Grafana:<br><em>(You can access it via the dashboard settings → Annotations tab)</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/993/1*ESy7rwypogIhPgqCEOyddA.png" /></figure><h4><strong>What’s In It For You</strong></h4><ul><li>See exactly when deployments happen alongside your metrics</li><li>No code changes, no CI tweaks</li><li>Direct links to GitHub release notes</li><li>Set it once and forget it — annotations appear automatically</li></ul><p><strong>Ready to try it?</strong> If you’re already using kube-state-metrics, you’re 90% there. Just —</p><ul><li>Copy the query</li><li>Adjust the labels</li><li>Add the annotation in Grafana</li><li>Done :)</li></ul><p><strong>I’m always up for new observability tricks, so if you have one, feel free to share it in the comments.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e6411978b75" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/the-easiest-way-to-track-kubernetes-deployments-in-grafana-7e6411978b75">The Easiest Way to Track Kubernetes Deployments in Grafana</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[30 Lines of YAML That Killed Our Alert Noise]]></title>
            <link>https://medium.com/elementor-engineers/30-lines-of-yaml-that-killed-our-alert-noise-c224d9a43d90?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/c224d9a43d90</guid>
            <category><![CDATA[production]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[alertmanager]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[incident-response]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Sun, 27 Apr 2025 14:57:52 GMT</pubDate>
            <atom:updated>2025-06-06T20:08:39.356Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f952fPKJxLxBfkJx3wFl_g.png" /></figure><h4>Making critical problems visible with <strong>Alertmanager</strong> Inhibit Rules</h4><p>If you’ve ever managed a production system, you know the pain of alert fatigue. When a major incident hits, your monitoring channel floods with notifications. It’s not one alert, it’s dozens. The signal gets lost in the noise.</p><p>We just wanted to focus on what mattered during incidents. So we implemented a simple solution with Alertmanager inhibit rules. Nothing strategic or complex. Just a way to quiet the noise.</p><h3>The Problem: Alert Storms in Production</h3><p>It happens every time. An API crashes, and suddenly our Slack explodes. Some alerts tell us what’s wrong (“OpenAI Error Rate”), while others just show symptoms (“Error Rate Increase”, “Low Request Rate”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jA4gBuL6CS5vEW7e" /></figure><p>During our last major incident, we had dozens of alerts for a single failure, and we spent precious minutes scrolling through notifications, trying to figure out what was going on.</p><h3>Enter Inhibit Rules</h3><p>Alertmanager includes a feature called <a href="https://prometheus.io/docs/alerting/latest/configuration/#inhibit_rule">inhibit rules</a>. Nothing groundbreaking, just a way to say, “If this critical alert is firing, don’t show me these related ones.”</p><p>The idea is simple — when we know the API is down, we don’t need ten different alerts telling us about the symptoms. We just need the one that matters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*ljMYThzWREinZSROaUtCmw.png" /></figure><h3>How We Set It Up</h3><p>The implementation is just a few lines of YAML in our alerting config. Each rule basically says: “When this critical alert fires, silence these less important ones that share the same context.”</p><p>Here’s some of the configuration we added:</p><p><em>Note: YAML syntax can slightly change between versions. If you’re setting this up, make sure to </em><a href="https://github.com/prometheus/alertmanager/blob/main/doc/examples/simple.yml"><em>check the official examples</em></a><em> for your version.</em></p><pre>inhibit_rules:<br>  # Silence API error symptoms if no pods running<br>  - source_matchers:<br>      - alertname = APINoRunningPods<br>      - severity = critical<br>    target_matchers:<br>      - alertname =~ &quot;APIErrorRate|APIRouteErrorRateIncrease&quot;<br>    equal: [&#39;scope&#39;]<br><br>  # Silence OpenAI performance metrics if errors occur<br>  - source_matchers:<br>      - alertname = OpenAIErrorRate<br>      - severity = critical<br>    target_matchers:<br>      - alertname =~ &quot;OpenAIResponseDuration|OpenAIRequestCount&quot;<br>    equal: [&#39;scope&#39;]<br><br>  # Silence low traffic alerts when API error rates are high<br>  - source_matchers:<br>      - alertname = APIErrorRate<br>      - severity = critical<br>    target_matchers:<br>      - alertname =~ &quot;LowRequestRate|LowInsertRate&quot;<br>    equal: [&#39;scope&#39;]<br><br>  # Silence duration and execution count alerts for a failing job<br>  - source_matchers:<br>      - alertname = JobErrorCounter<br>      - severity = critical<br>    target_matchers:<br>      - alertname =~ &quot;JobDuration|JobExecutedCounter&quot;<br>    equal: [&#39;scope&#39;, &#39;jobName&#39;]</pre><h3>What This Actually Does</h3><p>When our API error rate is up, we get one clear alert: “APIErrorRate.” All the cascading error alerts are automatically suppressed. If our OpenAI integration fails, we see the critical error, not the dozen metrics showing slow response times or failed requests.</p><p>The rules are smart enough only to silence related alerts. If something else breaks in another system, those alerts still come through loud and clear.</p><h3>Real-Time Clarity, Right When It Matters</h3><p>The first incident after implementing these rules told us everything. Our Slack channel, which would normally be flooded with alerts, showed just three critical notifications. We identified the problem in seconds. No scrolling, no piecing together clues from a dozen different alerts. Just the essential information, right when it was needed most.</p><p><strong>The root cause was immediately visible.</strong></p><h3>Small Config, Big Impact</h3><p>The entire system took less than an hour to implement. The YAML? Maybe 30 lines. The result?</p><ul><li>Identified root cause in seconds, not minutes</li><li>Faster, more focused incident response</li><li>Postmortems concentrate on solutions, not noise</li></ul><h3>Final Thought</h3><p>There are plenty of sophisticated alert management platforms out there. This isn’t one of them.</p><p>Just a few lines of YAML, a simple config file, and a quieter Slack channel. But in those critical moments when systems fail and customers are impacted, that clarity makes all the difference.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c224d9a43d90" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/30-lines-of-yaml-that-killed-our-alert-noise-c224d9a43d90">30 Lines of YAML That Killed Our Alert Noise</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Avoid Prometheus mess in NestJS]]></title>
            <link>https://medium.com/elementor-engineers/avoid-prometheus-mess-in-nestjs-1ea368e3e21e?source=rss-e7d782cfe630------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ea368e3e21e</guid>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[nestjs]]></category>
            <category><![CDATA[prometheus]]></category>
            <dc:creator><![CDATA[Netanel Avraham]]></dc:creator>
            <pubDate>Thu, 25 Jul 2024 07:51:15 GMT</pubDate>
            <atom:updated>2025-11-13T21:47:34.753Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KTKBoTanWwEzPESrDKuzpg.png" /></figure><h4>Simplifying <strong>Prometheus </strong>metrics reporting with a global metrics reporter in NestJS</h4><blockquote><em>The </em><strong><em>Global Metrics Reporter</em></strong><em> is now open-source as nestjs-metrics-reporter! Check it out on </em><a href="https://github.com/netanelavr/nestjs-metrics-reporter"><em>GitHub</em></a><em> and give it a try!</em></blockquote><h3>Introduction</h3><p>Transforming a complex monitoring process into a seamless experience can greatly enhance developer productivity. In our NestJS application, we faced this challenge head-on by setting up a monitoring solution.</p><p>Initially, we opted for a quick setup to get things running. However, as our application grew and more metrics started to pile up, it became clear that we needed a better solution. Our goal was to simplify the process of adding metrics, moving from updating multiple files to just <strong>a single line of code.</strong></p><h3>The Quick Setup</h3><h4>Initial Steps</h4><p>Initially, we prioritized speed by integrating <a href="https://github.com/willsoto/nestjs-prometheus">@willsoto/nestjs-prometheus</a>. This package streamlined Prometheus metrics integration into our NestJS application, offering:</p><ul><li><strong>Automatic Endpoint Registration</strong>: A /metrics for Prometheus integration.</li><li><strong>Default Metrics Collection</strong>: Immediate insights into CPU usage and memory consumption.</li><li><strong>Customizable Default Settings</strong>: Easy configuration of labels and prefixes for consistent metric categorization.</li></ul><p>It was the perfect solution to get us started.</p><h4>Pros:</h4><ul><li><strong>Easy Setup</strong>: Add Prometheus metrics with minimal code.</li><li><strong>Out-of-the-box Metrics</strong>: Pre-configured metrics without extra setup.</li><li><strong>Customizable Defaults</strong>: Configure labels and prefixes effortlessly.</li></ul><h4>Cons:</h4><ul><li><strong>Boilerplate Code</strong>: Adding new metrics requires repetitive setup.</li><li><strong>Scalability Issues</strong>: Managing numerous metrics becomes cumbersome.</li></ul><h3>Growing Pains and Limitations</h3><p>I’ll start with an example to illustrate the process of adding a new metric —</p><pre>// Creating a new provider <br><br>const requestsCounterProvider = makeCounterProvider({<br>  name: REQUESTS_COUNTER,<br>  help: &#39;Number of requests&#39;,<br>  labelNames: [&#39;controller&#39;, &#39;method&#39;],<br>});</pre><pre>// Adding the provider to the module<br><br>@Module({<br>  providers: [<br>    requestsCounterProvider,<br>    CoolService,<br>  ],<br>  exports: [CoolService],<br>})<br>export class CoolModule {}</pre><pre>// Injecting the metric to the service<br><br>@Injectable()<br>export class CoolService {<br>  constructor( @InjectMetric(REQUESTS_COUNTER) private readonly requestCounter: Counter&lt;string&gt; ) {}<br><br>  public doSomething(controller: string, method: string): void {<br>    this.requestCounter.inc({ controller, method });<br>  }<br>}</pre><p>Managing a variety of metrics could make your service look like this -</p><pre>@Injectable()<br>export class MetricsOverloadedService {<br>  constructor(<br>    @InjectMetric(&#39;COOL_METRIC_GAUGE&#39;) public readonly metricCoolGauge: Gauge&lt;string&gt;,<br>    @InjectMetric(&#39;COOL_COOL_METRIC_GAUGE&#39;) public readonly metricCool2Gauge: Gauge&lt;string&gt;,<br>    @InjectMetric(&#39;COOL_COOL_COOL_METRIC_GAUGE&#39;) public readonly metricCool3Gauge: Gauge&lt;string&gt;,<br>    @InjectMetric(&#39;COOL_COOL_COOL_METRIC_COUNTER&#39;) public readonly metricCool4Counter: Counter&lt;string&gt;,<br>    @InjectMetric(&#39;COOL_COOL_COOL_COOL_COOL_METRIC_COUNTER&#39;) public readonly metricCoolCounter: Counter&lt;string&gt;,<br>    // ... potentially many more<br>  ) {<br>  }<br>}</pre><p>As our application scaled, we faced challenges with manual metric injection and excessive boilerplate code. This led us to explore a more efficient solution.</p><h3>Introducing <a href="https://github.com/siimon/prom-client">prom-client</a></h3><p>At its core, <a href="https://github.com/willsoto/nestjs-prometheus">nestjs-prometheus</a> leverages <a href="https://github.com/siimon/prom-client">prom-client</a>, a robust Prometheus client for Node.js. This library is the backbone of our metrics collection, and supports various metric types (counter, gauge, histogram, and summary), providing flexibility and control over our metrics collection.</p><h3>The Global Metrics Reporter</h3><p>Determined to find a better way, we created a global metrics reporter. This solution would allow us to report metrics anywhere in our application without manual injection, simplifying metric reporting and improving the developer experience. Let’s walk through the implementation:</p><h3><strong>Understanding the Global Reporter </strong>Architecture</h3><p>Before we dive into the implementation details, let’s look at the high-level architecture of the global metrics reporter. The diagram below illustrates how different components in the application interact with reporting, managing, and exposing metrics for Prometheus.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ZqRpKNjGwbc3Tkhq_1nxA.png" /></figure><h3>Step-by-Step Implementation</h3><h4>Step 1: Implementing the Metrics Service</h4><p>We began by developing a MetricsService with <a href="https://github.com/siimon/prom-client">prom-client</a> directly. Imagine being able to manage and update all your metrics from a single point of control — that’s exactly what we aimed to achieve.</p><pre>@Injectable()<br>export class MetricsService {<br> private readonly counter: { [key: string]: Counter&lt;string&gt; } = {};<br> private readonly gauge: { [key: string]: Gauge&lt;string&gt; } = {};<br><br> constructor( @Inject( Registry ) private readonly registry: Registry ) {}<br><br> public incCounter( key: string, labels?: Record&lt;string, string | number&gt; ): void {<br>  if ( ! this.counter[ key ] ) {<br>    this.counter[ key ] = new Counter( {<br>      name: key,<br>      help: `Counter for ${ key }`,<br>      labelNames: labels ? Object.keys( labels ) : [],<br>      registers: [ this.registry ],<br>    } );<br>  }<br>  this.counter[ key ].inc( labels );<br> }<br><br> public setGauge( key: string, value: number, labels?: Record&lt;string, string | number&gt; ): void {<br>  if ( ! this.gauge[ key ] ) { <br>    this.gauge[ key ] = new Gauge( {<br>      name: key,<br>      help: `Gauge for ${ key }`,<br>      labelNames: labels ? Object.keys( labels ) : [],<br>      registers: [ this.registry ],<br>     } );<br>  }<br>  this.gauge[ key ].set( labels, value );<br> }<br>}</pre><h4>Step 2: Creating a Global Reporter Service</h4><p>Next, we created a ReporterService that acts as a wrapper around the MetricsService and uses the OnApplicationBootstrap lifecycle hook to automatically initialize the metrics when the application starts. This service allows us to report metrics anywhere in our codebase without injecting the metric.</p><pre><br>@Injectable()<br>export class ReporterService implements OnApplicationBootstrap {<br> private static metricsService: MetricsService;<br> <br> constructor( private readonly metrics: MetricsService ) {}<br> <br> onApplicationBootstrap() {<br>  ReporterService.metricsService = this.metrics;<br> }<br><br> static counter( key: string, labels?: Record&lt;string, string | number&gt; ): void {<br>  ReporterService.metricsService.incCounter( key, labels );<br> }<br><br> static gauge( key: string, value: number, labels?: Record&lt;string, string | number&gt; ): void {<br>  ReporterService.metricsService.setGauge( key, value, labels );<br> }<br>}</pre><p>While it was possible to consolidate metrics management and reporting into a single service, we preferred to separate concerns to ensure each service handles its specific responsibilities:</p><ul><li><strong>MetricsService:</strong> Focuses on managing and providing the metrics.</li><li><strong>ReporterService:</strong> Responsible for reporting the metrics.</li></ul><p>This separation made our system modular and easier to maintain. It’s like having one friend who’s great at cooking and another who’s great at eating — they each do their part perfectly.</p><h4>Step 3: Defining the Reporter Module</h4><p>We created a ReporterModule and wrapped it with NestJS <a href="https://docs.nestjs.com/modules#global-modules">@Global</a> decorator. This ensures the service can be easily imported and used in any module without adding it as a provider.</p><pre>@Global()<br>@Module( {<br>  imports: [ MetricsModule ],<br>  providers: [ ReporterService, MetricsService ],<br>  exports: [ ReporterService ],<br>} )<br><br>export class ReporterModule {}</pre><p><strong>Step 4: Metrics Controller</strong></p><p>We created a MetricsController to expose the /metrics endpoint and retrieve metrics data from the registry. This endpoint serves as the gateway for Prometheus to scrape metrics from your application.</p><pre>@ApiTags( &#39;Metrics&#39; )<br>@Controller( &#39;metrics&#39; )<br>export class MetricsController {<br>  constructor( @Inject( Registry ) private readonly registry: Registry ) {}<br><br>  @Get()<br>  async getMetrics() {<br>    return await this.registry.metrics();<br>  }<br>}</pre><p><strong>Step 5: Report :)</strong></p><p>Just import the global ReporterService and use it to report the necessary metrics.</p><pre>@Injectable()<br>export class RequestsInterceptor implements NestInterceptor {<br>  intercept(context: ExecutionContext, next: CallHandler): Observable&lt;any&gt; {<br>    const { controllerName, methodName } = this.extractControllerInfo(context);<br>    <br>    ReporterService.counter(REQUESTS_COUNTER, {<br>      controller: controllerName, <br>      method: methodName,<br>    });<br>  }<br>}</pre><h3>Understanding the Registry as a Singleton</h3><p>The <a href="https://github.com/siimon/prom-client">prom-client</a> Registry serves as the cornerstone of our metrics infrastructure, acting as a singleton to centralize all metrics and prevent duplication. Using the useFactory method we ensure a single instance of the Registry is shared across the application.</p><h3>Conclusion</h3><p>Building a global metrics reporter in NestJS isn’t just monitoring — it’s about enhancing the developer experience. By consolidating metric management into a global service, we eliminated manual injection, reduced boilerplate code, and improved maintainability. This approach ensures cleaner code and smoother scalability as our application grows.</p><h3>Key Takeaways:</h3><ul><li><strong>Start Simple:</strong> Utilize a quick setup package.</li><li><strong>Scale Wisely:</strong> Evolve your monitoring solution as your application scales.</li><li><strong>Enhance Developer Experience</strong>: Streamline metric reporting to boost productivity and clean code.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mBRMmqhTMbb-lvm8teOrWw.png" /></figure><p>I hope this article provided useful tips for managing metrics in your NestJS applications. Thanks for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ea368e3e21e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/elementor-engineers/avoid-prometheus-mess-in-nestjs-1ea368e3e21e">Avoid Prometheus mess in NestJS</a> was originally published in <a href="https://medium.com/elementor-engineers">Elementor Engineers</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>