<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by thataidev on Medium]]></title>
        <description><![CDATA[Stories by thataidev on Medium]]></description>
        <link>https://medium.com/@siddharth.bisht.work?source=rss-6fbdfb40ae92------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QTpcINxv5f8nJkwaOyfMVQ.png</url>
            <title>Stories by thataidev on Medium</title>
            <link>https://medium.com/@siddharth.bisht.work?source=rss-6fbdfb40ae92------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 00:28:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@siddharth.bisht.work/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[When AI Agents Call the Wrong Tool: The Hidden Problem With Tool Selection]]></title>
            <link>https://medium.com/@siddharth.bisht.work/when-ai-agents-call-the-wrong-tool-the-hidden-problem-with-tool-selection-778bf37e7baa?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/778bf37e7baa</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 15:21:02 GMT</pubDate>
            <atom:updated>2026-11-13T15:21:02.176Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gh3S-sT2eQvR03fm-p0Vng.png" /></figure><p>One of the most impressive things about an AI agent is its ability to use tools.</p><p>It can search a database, call an API, read a file, or execute a workflow without requiring a developer to hardcode every decision.</p><p>But there’s a catch.</p><p>An agent can have access to all the right tools and still choose the wrong one.</p><p>Imagine asking an AI assistant to check whether an order has been shipped. Instead of calling the order-status API, it searches an unrelated knowledge base. Or it calls the correct API but sends the wrong parameters.</p><p>The tools work. The model responds. Yet the task fails.</p><p>This is the hidden challenge of <strong>tool selection in AI agents</strong>.</p><h3>Why Does Tool Selection Go Wrong?</h3><p>When a model uses function calling or tool calling, it selects from the tools made available to it, based on their descriptions, parameters, and the current context.</p><p>If two tools have similar descriptions, the model may confuse them. If a tool’s purpose isn’t clearly explained, the model may misunderstand when to use it.</p><p>Poor context can make the decision even harder. The model may not have enough information to distinguish between two possible actions, or it may make assumptions about missing parameters.</p><p>And sometimes, the tool is selected correctly but the arguments are invalid.</p><p>These are different failure modes, and treating them all as generic AI errors makes debugging harder.</p><h3>How Do You Make Tool Calling More Reliable?</h3><p>Start with clear tool definitions.</p><p>A tool description should explain what the tool does, when it should be used, what information it requires, and what it should not be used for. Specific names and unambiguous parameter schemas can reduce confusion.</p><p>Next, validate tool arguments before execution. Use schema validation, type checks, authorization rules, and business logic to prevent malformed or unauthorized calls from reaching external systems.</p><p>For complex workflows, don’t expose every available tool at once if the agent only needs a small subset. <strong>Tool routing</strong> can help narrow the options based on the user’s request or the current workflow stage.</p><p>Finally, evaluate tool selection separately from the final answer. Track whether the agent chose the correct tool, supplied valid arguments, handled errors, and achieved the intended outcome.</p><p>A fluent final response doesn’t prove that the agent used the right process.</p><h3>Final Thoughts</h3><p>Giving an AI agent more tools doesn’t automatically make it more capable.</p><p>What matters is whether it can choose the right tool, use it correctly, and recognize when it needs clarification.</p><p>Reliable tool calling isn’t just about connecting a model to an API. It’s about designing the boundaries between probabilistic decisions and predictable software.</p><p>Because an intelligent agent is only as useful as the actions it takes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=778bf37e7baa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Infinite Loop Problem in AI Agents]]></title>
            <link>https://medium.com/@siddharth.bisht.work/the-infinite-loop-problem-in-ai-agents-437697ad37b8?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/437697ad37b8</guid>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 15:16:01 GMT</pubDate>
            <atom:updated>2026-11-09T15:16:01.497Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4mKTqIqwtAskuYdaRXM6EQ.png" /></figure><p>An AI agent receives a task.</p><p>It calls a tool. The tool returns a result. The agent decides what to do next, calls another tool, and continues working.</p><p>So far, everything looks good.</p><p>But what happens when the agent keeps repeating the same action, retries a failed request indefinitely, or gets trapped between two steps that never move the task forward?</p><p>Your agent is still running. Your API is still receiving requests. Your token bill is still growing.</p><p>But the task isn’t getting done.</p><p>Welcome to the <strong>infinite loop problem in AI agents</strong>.</p><h3>Why Do AI Agents Get Stuck?</h3><p>Unlike a traditional script with a fixed sequence of steps, an AI agent often chooses its next action based on the latest model response and tool output.</p><p>That flexibility creates several possible failure modes.</p><p>The agent might misunderstand a tool response and repeat the same call. It might keep searching because it never considers the available evidence sufficient. Or it might alternate between two tools without making meaningful progress.</p><p>A poorly handled error can make things worse. If a tool keeps returning the same failure and the agent treats every failure as a reason to try again, the workflow can continue indefinitely.</p><p>The model may be capable of reasoning, but reasoning alone doesn’t guarantee that a task will terminate.</p><h3>Why Retries Aren’t Always the Answer</h3><p>Retries are useful when failures are temporary, such as a brief network interruption or a rate-limit response.</p><p>But blindly retrying every failure can create a retry storm, increase latency, waste tokens, and put additional pressure on an already struggling service.</p><p>A reliable agent needs to distinguish between failures that may recover and failures that require a different approach.</p><p>For example, a temporary timeout might justify a limited retry with exponential backoff. An invalid tool argument should usually trigger correction or a clear failure — not the same request repeated endlessly.</p><h3>How to Prevent Agent Loops</h3><p>Developers can introduce several layers of control.</p><p><strong>Maximum step limits:</strong> Set a hard cap on the number of reasoning or tool-execution steps in a single task.</p><p><strong>Timeouts and budgets:</strong> Limit total execution time, token consumption, and tool calls so a stalled agent cannot consume unlimited resources.</p><p><strong>Bounded retries:</strong> Use retry limits and exponential backoff for transient failures. Avoid repeating requests that are unlikely to succeed without a change.</p><p><strong>Progress checks:</strong> Detect repeated tool calls, identical errors, or cycles that produce no meaningful change in state.</p><p><strong>Explicit stopping conditions:</strong> Define what successful completion looks like and when the agent should stop, ask for clarification, or return a failure.</p><p><strong>Observability:</strong> Trace model calls and tool executions so developers can identify where the loop began and why it continued.</p><p>These controls are especially important for autonomous agents that can interact with external APIs, modify data, or trigger costly operations.</p><h3>Final Thoughts</h3><p>An AI agent doesn’t become reliable simply because it can plan and use tools.</p><p>It becomes reliable when its execution is bounded, its failures are handled deliberately, and its progress can be measured.</p><p>Give an agent enough freedom to solve a problem — but never give it unlimited time, unlimited retries, and unlimited resources.</p><p>Because a system that never stops working isn’t necessarily productive.</p><p>Sometimes, it’s just stuck.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=437697ad37b8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Rot: When More Context Makes AI Worse]]></title>
            <link>https://medium.com/@siddharth.bisht.work/context-rot-when-more-context-makes-ai-worse-a15b245fdc3e?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/a15b245fdc3e</guid>
            <category><![CDATA[ai-model-training]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[context-rot]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 15:16:01 GMT</pubDate>
            <atom:updated>2026-11-05T15:16:01.552Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J8CpkWR4TAZ96vxd_3g4Wg.png" /></figure><p>We spent a lot of time teaching AI models to handle longer conversations.</p><p>Bigger context windows. More documents. Longer chat histories. Entire codebases added to a single prompt.</p><p>It sounds like a straightforward improvement: if an AI has more information, shouldn’t it give better answers?</p><p>Not always.</p><p>Sometimes, giving a model more context makes it harder for the model to find what actually matters. This is one of the challenges developers encounter when building long-context AI applications, often discussed under the term <strong>context rot</strong>.</p><h3>The Problem With Too Much Context</h3><p>Imagine asking an AI coding assistant to fix a bug in your application.</p><p>At first, you provide the relevant function and the error message. The answer is focused and useful.</p><p>Now imagine adding hundreds of files, old conversations, unrelated documentation, and several outdated implementation notes.</p><p>The model has more information, but not necessarily more useful information.</p><p>Important details can get buried. Conflicting instructions may compete for attention. Relevant evidence may be overlooked among irrelevant text.</p><p>A large context window tells you how much information a model can accept. It doesn’t guarantee that the model will use every part of that information equally well.</p><h3>Why Context Quality Matters</h3><p>Several factors can make a long prompt less effective.</p><p><strong>Irrelevant information</strong> increases the amount of material the model must process without helping it answer the question.</p><p><strong>Conflicting information</strong> can appear when old documentation or previous decisions disagree with the current requirements.</p><p><strong>Poor information placement</strong> can make important details harder to use, depending on the model and task.</p><p><strong>Context dilution</strong> happens when useful facts are surrounded by so much unrelated material that the response becomes less focused.</p><p>These issues can affect coding assistants, RAG pipelines, AI agents, and long-running conversations.</p><h3>How Developers Can Reduce Context Rot</h3><p>The solution isn’t simply to buy access to a model with a larger context window.</p><p>It’s to improve <strong>context engineering</strong>.</p><p>Retrieve information relevant to the current task instead of automatically including every available document. Use semantic search and metadata filters to narrow down results. Apply effective chunking so retrieved passages preserve their meaning.</p><p>Summarize older conversation history while retaining important decisions, constraints, and unresolved questions. Remove duplicated content and distinguish current instructions from historical information.</p><p>For coding agents, this might mean providing the relevant files, project conventions, recent errors, and acceptance criteria instead of dumping the entire repository into every request.</p><p>And don’t assume a context strategy works just because it sounds sensible. Evaluate it using representative tasks, measuring answer quality, retrieval relevance, latency, and token cost.</p><h3>Final Thoughts</h3><p>The future of AI isn’t just about models that can process more information.</p><p>It’s about systems that can identify the right information at the right time.</p><p>A larger context window gives an AI more room to work. Good context engineering helps it use that room wisely.</p><p>Because when everything is included, the most important thing can still get lost.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a15b245fdc3e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Agents Need Guardrails, Not Just Intelligence]]></title>
            <link>https://medium.com/@siddharth.bisht.work/ai-agents-need-guardrails-not-just-intelligence-b73d3ea5ad9f?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/b73d3ea5ad9f</guid>
            <category><![CDATA[prompt-injection]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[aiguardrails]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Mon, 02 Nov 2026 14:51:01 GMT</pubDate>
            <atom:updated>2026-11-02T14:51:01.740Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HX9emTkiLUTj6eTnXzYmXg.png" /></figure><p>An AI agent can plan a task, call tools, retrieve information, and take action with surprisingly little human intervention.</p><p>That sounds powerful — and it is.</p><p>But imagine an agent that keeps retrying a failed payment request, calls the same API repeatedly, or follows an incorrect instruction from an untrusted webpage.</p><p>Suddenly, intelligence isn’t the only thing that matters.</p><p>The real question becomes: <strong>How do you keep an AI agent within safe and predictable boundaries?</strong></p><h3>Why AI Agents Can Be Unpredictable</h3><p>A traditional program usually follows explicitly defined control flow. An AI agent may decide which tool to call, what information to retrieve, and what step to take next based on <a href="https://www.ibm.com/think/topics/reasoning-model">model-generated reasoning or plans.</a></p><p>That flexibility makes agents useful, but it introduces new failure modes.</p><p>The agent might choose the wrong tool, misinterpret a tool’s output, enter a loop, or act on misleading information. In systems with access to external services, these mistakes can have real consequences.</p><p>A well-written system prompt helps, but it isn’t a security boundary by itself.</p><h3>What Do Guardrails Actually Do?</h3><p><a href="https://www.ibm.com/think/topics/ai-guardrails"><strong>AI guardrails</strong> </a>are controls that restrict or validate what an AI system can do.</p><p>They should exist at multiple layers.</p><p><strong>Input validation</strong> checks incoming requests for malformed data, prohibited actions, or unexpected content.</p><p><strong>Tool permissions</strong> limit which tools an agent can access and which operations it can perform. An agent that only needs to read customer records shouldn’t automatically receive permission to delete them.</p><p><strong>Output validation</strong> checks whether generated responses or tool arguments match expected schemas and business rules.</p><p><strong>Execution limits</strong> prevent runaway behavior through maximum step counts, timeouts, retry limits, and token budgets.</p><p><strong>Human approval</strong> can be required before consequential actions, such as issuing a refund, deleting records, or sending an external communication.</p><p>These controls reduce risk without requiring the model to make every safety decision correctly.</p><h3>Don’t Confuse Prompt Instructions With Enforcement</h3><p>Telling an agent, “Never perform a dangerous action,” is useful guidance. It isn’t sufficient protection.</p><p>A robust application enforces permissions in the code and services surrounding the model. The agent should not be able to bypass authorization simply by generating a convincing explanation.</p><p>Developers should also account for <a href="https://en.wikipedia.org/wiki/Prompt_injection"><strong>prompt injection</strong></a>, where untrusted content attempts to influence the model’s behavior. A retrieved document or webpage should be treated as data — not automatically as an instruction to follow.</p><p>Finally, log important decisions, test adversarial inputs, and monitor repeated failures. Guardrails need ongoing evaluation as tools, prompts, and models evolve.</p><h3>Final Thoughts</h3><p>The best AI agent isn’t necessarily the one with the most autonomy.</p><p>It’s the one that can complete useful tasks while staying within clearly defined limits.</p><p>Give agents the freedom to reason, but keep permissions, validation, and critical decisions under reliable control.</p><p>Because in production, an intelligent system that cannot be controlled is not a reliable system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b73d3ea5ad9f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Observability: What Happens After Your AI Goes Live?]]></title>
            <link>https://medium.com/@siddharth.bisht.work/ai-observability-what-happens-after-your-ai-goes-live-75beb1cb5487?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/75beb1cb5487</guid>
            <category><![CDATA[ai-observability]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[https]]></category>
            <category><![CDATA[llm-observability]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Sat, 31 Oct 2026 14:51:01 GMT</pubDate>
            <atom:updated>2026-10-31T14:51:01.466Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gFW2xwXj6bpzRBL6AVzBPw.png" /></figure><p>Getting an AI application to work in a demo feels great.</p><p>Your chatbot answers questions. Your AI agent calls tools. Your RAG pipeline retrieves relevant documents.</p><p>Everything looks good.</p><p>Then real users arrive.</p><p>Responses become slower. Token costs increase. Retrieval quality drops. The model starts giving inconsistent answers, and nobody can immediately explain why.</p><p>Welcome to the part of AI engineering that demos rarely show: <a href="https://www.ibm.com/think/topics/observability"><strong>production observability</strong>.</a></p><h3>Why Traditional Monitoring Isn’t Enough</h3><p>In a traditional application, monitoring CPU usage, memory consumption, error rates, and response times can reveal a lot about system health.</p><p>AI applications need those metrics too, but they introduce additional questions.</p><p>Was the wrong document retrieved? Did the model ignore the context? Did a tool call fail? Did the prompt become unnecessarily large? Did the model produce an incorrect answer even though the API returned a successful response?</p><p>A request can return <a href="https://www.ibm.com/docs/en/cics-ts/6.x?topic=http-responses-codes">HTTP 200</a> and still deliver a terrible AI experience.</p><p>That’s why <a href="https://www.ibm.com/think/topics/llm-observability"><strong>LLM observability</strong></a> goes beyond infrastructure monitoring.</p><h3>What Should We Observe?</h3><p>A useful AI observability setup tracks several layers.</p><p><strong>Latency and errors:</strong> Measure end-to-end response time, model latency, timeout rates, and failed tool calls.</p><p><strong>Token usage and cost:</strong> Monitor input tokens, output tokens, and spending across models, users, and workflows. Unexpected increases can reveal inefficient prompts or unnecessary agent loops.</p><p><strong>Retrieval quality:</strong> For RAG applications, examine which documents were retrieved, their relevance, and whether the final answer was grounded in them.</p><p><strong>Response quality:</strong> Use evaluation metrics to assess factual accuracy, relevance, instruction-following, and safety. Automated LLM-as-a-judge evaluations can help, but they should be calibrated against human-reviewed examples.</p><p><strong>Traceability:</strong> Distributed tracing helps connect a single user request to its individual steps — retrieval, model calls, tool execution, retries, and final response.</p><p>Together, these signals make it easier to understand not just whether a request failed, but why the experience was poor.</p><h3>Observability Is More Than Collecting Logs</h3><p>Collecting thousands of logs doesn’t automatically make a system observable.</p><p>The real value comes from connecting signals to meaningful questions.</p><p>If response times increase, can you identify whether the bottleneck is retrieval, the model provider, or a tool call?</p><p>If answer quality drops after a model update, can you compare evaluation results across versions?</p><p>If costs suddenly spike, can you trace the increase to a particular workflow?</p><p>One important detail: prompts and model responses can contain sensitive user data. Observability systems should use appropriate redaction, access controls, and retention policies rather than logging everything indiscriminately.</p><h3>Final Thoughts</h3><p>AI applications don’t stop changing once they’re deployed. Model behavior, user inputs, external data, and system dependencies can all affect performance over time.</p><p>That’s why observability belongs in the design from day one, not as an emergency addition after something breaks.</p><p>Building an AI feature is one challenge.</p><p>Understanding how it behaves in the real world is another — and that’s what makes it production-ready.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75beb1cb5487" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Evaluation: Why Your AI Can Get Worse Without Anyone Noticing]]></title>
            <link>https://medium.com/@siddharth.bisht.work/ai-evaluation-why-your-ai-can-get-worse-without-anyone-noticing-52970ae956e5?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/52970ae956e5</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-evaluation]]></category>
            <category><![CDATA[developer]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 14:46:01 GMT</pubDate>
            <atom:updated>2026-10-29T14:46:01.651Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SAKQc1LwaGCC_fn3B8Tqdw.png" /></figure><p>There’s something tricky about shipping an AI feature.</p><p>You test it. The answers look good. Your evaluation scores are acceptable, and everything seems ready for production.</p><p>Then, a few weeks later, users start reporting strange responses.</p><p>The model misunderstands simple questions. The chatbot misses important context. Answers that used to be accurate suddenly feel unreliable.</p><p>And your application hasn’t even crashed.</p><p>Welcome to one of the biggest challenges in AI engineering: <a href="https://www.quality-assurance-solutions.com/Regression.html"><strong>model quality regression</strong>.</a></p><h3>AI Doesn’t Need to Crash to Fail</h3><p>Traditional software testing often gives us a clear signal. A function returns the wrong value, an API throws an error, or a test fails.</p><p>AI systems are different.</p><p>A response can be grammatically correct, return a successful HTTP status, and still be factually wrong or irrelevant.</p><p>Even small changes can affect quality. A new system prompt, an updated model version, a different embedding model, or a change in document chunking can alter the output of an entire RAG pipeline.</p><p>That’s why testing an AI application once isn’t enough.</p><h3>What Is an AI Evaluation Pipeline?</h3><p>An <a href="https://arize.com/glossary/evaluation-pipeline/"><strong>AI evaluation pipeline</strong> </a>measures whether a model or AI application continues to meet its expected quality standards.</p><p>It starts with an evaluation dataset: a collection of representative inputs and expected behaviors. These examples should include normal requests, edge cases, ambiguous questions, and situations where the system should refuse to answer.</p><p>The application then generates responses that can be evaluated against specific criteria, such as:</p><ul><li>Factual accuracy and relevance</li><li>Instruction-following</li><li>Context grounding in RAG systems</li><li>Tool selection and execution in AI agents</li><li>Safety and appropriate refusal behavior</li></ul><p>Some checks can be deterministic. Others may use an LLM-as-a-judge, human reviewers, or a combination of both.</p><p>No single metric captures everything, so evaluations should match the actual risks and goals of the application.</p><h3>Treat Model Changes Like Software Changes</h3><p>Imagine upgrading the model powering your customer support assistant.</p><p>The new version is faster and cheaper, but it starts giving less accurate answers to billing questions.</p><p>Without regression testing, that issue might reach thousands of users before anyone notices.</p><p>A better approach is to run the same evaluation dataset against the old and new versions, compare results, and investigate meaningful regressions before deployment.</p><p>This is where <a href="https://deepchecks.com/glossary/llm-regression-testing/"><strong>LLM regression testing</strong></a>, versioned prompts, and continuous evaluation become valuable.</p><p>You can also monitor production feedback and use carefully reviewed examples to improve future evaluation datasets.</p><h3>Final Thoughts</h3><p>AI quality isn’t a permanent achievement. It can change whenever the model, prompt, data, or surrounding system changes.</p><p>The goal isn’t to prove that your AI works today. It’s to build a process that helps you notice when it stops working tomorrow.</p><p>Because reliable AI isn’t just about getting good answers.</p><p>It’s about knowing whether those answers are still good.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52970ae956e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Hallucinations: When an LLM Sounds Confident but Gets It Wrong]]></title>
            <link>https://medium.com/@siddharth.bisht.work/ai-hallucinations-when-an-llm-sounds-confident-but-gets-it-wrong-2ada84d7e87a?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ada84d7e87a</guid>
            <category><![CDATA[development]]></category>
            <category><![CDATA[ai-hallucination]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 14:46:01 GMT</pubDate>
            <atom:updated>2026-10-27T14:46:01.478Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2lhkWc_xR0wMj0CT1K6fsg.png" /></figure><p>One of the strangest things about working with AI is that it can give you a completely wrong answer with complete confidence.</p><p>Ask an LLM a technical question, and it might explain a function that doesn’t exist, invent an API endpoint, or provide a convincing answer backed by a fake reference.</p><p>The response sounds right. The problem is, it isn’t.</p><p>This is called an <a href="https://www.ibm.com/think/topics/ai-hallucinations"><strong>AI hallucination</strong>.</a></p><h3>Why Do LLMs Hallucinate?</h3><p><a href="https://www.ibm.com/think/topics/large-language-models">Large language models</a> don’t retrieve truth from a perfect database. They generate responses by predicting likely next tokens based on patterns learned during training and the context they receive.</p><p>That makes them incredibly useful, but it also means they can produce plausible information without having reliable evidence to support it.</p><p>Hallucinations can happen because of incomplete training data, ambiguous prompts, outdated knowledge, or missing context. Even a well-designed prompt cannot guarantee factual accuracy.</p><h3>Can RAG Reduce Hallucinations?</h3><p>One common approach is <a href="https://www.ibm.com/think/topics/retrieval-augmented-generation"><strong>Retrieval-Augmented Generation (RAG)</strong></a>.</p><p>Instead of relying only on what the model learned during training, a RAG system retrieves relevant information from external sources, such as documentation, internal knowledge bases, or verified company data.</p><p>The retrieved context gives the model evidence to work with.</p><p>But here’s the important part: <strong>RAG reduces some hallucinations; it doesn’t eliminate them.</strong></p><p>The retrieval system might return irrelevant documents, the model might misinterpret the evidence, or the answer might contain claims that the sources never supported.</p><p>That’s why retrieval quality and generation quality both need evaluation.</p><h3>How Do We Make AI More Reliable?</h3><p>In production, AI reliability requires more than a clever prompt.</p><p>Developers can use source citations, structured outputs, grounding checks, confidence thresholds, and automated evaluation datasets. For high-risk use cases, human review may also be necessary.</p><p>An AI evaluation pipeline can test whether responses are factually correct, relevant to the question, and supported by the retrieved context.</p><p>It’s also worth testing edge cases: missing information, conflicting sources, misleading input, and questions the system simply cannot answer.</p><p>Sometimes, the best response an AI can give is: <em>“I don’t have enough information to answer that reliably.”</em></p><h3>Final Thoughts</h3><p>The goal isn’t to build an AI that always sounds intelligent. It’s to build one that knows when its answer needs evidence — and when it should admit uncertainty.</p><p>Because in software, a confident response isn’t the same as a correct response.</p><p>And when AI moves from experimentation into production, that difference matters more than ever.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ada84d7e87a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is Backpressure in AI Systems? When Requests Arrive Faster Than They Can Be Processed]]></title>
            <link>https://medium.com/@siddharth.bisht.work/what-is-backpressure-in-ai-systems-when-requests-arrive-faster-than-they-can-be-processed-302e78e20ba1?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/302e78e20ba1</guid>
            <category><![CDATA[backpressure]]></category>
            <category><![CDATA[ai-systems]]></category>
            <category><![CDATA[load-shedding]]></category>
            <category><![CDATA[continuous-batching]]></category>
            <category><![CDATA[gpu]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Sat, 24 Oct 2026 15:21:01 GMT</pubDate>
            <atom:updated>2026-10-24T15:21:01.652Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jdHUGjc_QBcSS9fczU3_Ew.png" /></figure><p>Imagine your AI application is working perfectly.</p><p>Then traffic increases.</p><p>Requests start piling up, response times climb, and GPU resources become saturated. Eventually, even simple questions take too long to answer.</p><p>The model might be fine. <strong>The system is struggling to keep up with demand.</strong></p><p>This is where <a href="https://www.geeksforgeeks.org/computer-networks/back-pressure-in-distributed-systems/"><strong>backpressure</strong> </a>becomes important.</p><h3>What Is Backpressure?</h3><p>Backpressure is a mechanism that helps a system manage incoming work when downstream components cannot process it quickly enough.</p><p>In an <a href="https://www.ibm.com/think/topics/llm-inference"><strong>LLM inference pipeline</strong></a>, requests may pass through an API gateway, a request queue, an inference server, and a GPU-backed model.</p><p>If requests arrive faster than the inference server can handle them, the queue grows. Without proper controls, this can increase <strong>latency, memory consumption, timeouts, and request failures</strong>.</p><h3>How Do AI Systems Handle It?</h3><p>Engineers use techniques such as <strong>bounded queues, </strong><a href="https://www.tibco.com/glossary/what-is-api-throttling"><strong>request throttling</strong></a><strong>, rate limiting, </strong><a href="https://www.tatapower.com/glossary/l/load-shedding"><strong>load shedding</strong></a><strong>, </strong><a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/"><strong>admission control</strong></a><strong>, and </strong><a href="https://www.sandgarden.com/learn/dynamic-batching"><strong>dynamic batching</strong></a>.</p><p>For example, a bounded queue prevents unlimited request accumulation. Admission control can reject or defer work when capacity is limited, while <a href="https://inferencex.semianalysis.com/glossary/continuous-batching"><strong>continuous batching</strong> </a>helps inference servers process multiple requests efficiently.</p><p>The goal isn’t to accept every request at any cost.</p><p>It’s to protect the system so it can continue serving requests reliably.</p><h3>Final Thoughts</h3><p>Scaling AI isn’t just about adding more GPUs or choosing a larger model.</p><p>It’s also about managing the flow of work through the system.</p><p><strong>A reliable AI application knows not only how to process requests, but also when to slow down before everything falls behind.</strong> ⚙️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=302e78e20ba1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is an AI Memory System? How AI Agents Remember Past Interactions]]></title>
            <link>https://medium.com/@siddharth.bisht.work/what-is-an-ai-memory-system-how-ai-agents-remember-past-interactions-2dd46dd5f60c?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/2dd46dd5f60c</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[memory-systems]]></category>
            <category><![CDATA[metadata-filtering]]></category>
            <category><![CDATA[vector-database]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Wed, 21 Oct 2026 15:11:02 GMT</pubDate>
            <atom:updated>2026-10-21T15:11:02.298Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PaE9lDRLTooK2wuwxrunNQ.png" /></figure><p>Ever had to explain the same thing to an AI assistant every time you opened a new conversation?</p><p>That’s one of the limitations AI memory systems aim to solve.</p><p>An <a href="https://atlan.com/know/ai-memory-system/"><strong>AI memory system</strong></a> helps an AI application retain and retrieve relevant information across interactions, rather than treating every request as an isolated conversation.</p><h3>How Does AI Memory Work?</h3><p>AI memory isn’t necessarily a model remembering everything it has ever seen. It’s often an engineered system that stores selected information and retrieves it when needed.</p><p>Different types of memory serve different purposes:</p><ul><li><strong>Short-term memory:</strong> Maintains the current conversation or task state.</li><li><strong>Long-term memory:</strong> Stores useful information for future interactions.</li><li><strong>Episodic memory:</strong> Records past events or interactions.</li><li><strong>Semantic memory:</strong> Stores facts and knowledge that can be retrieved later.</li></ul><p>Developers may combine <strong>conversation history, summarization, embeddings, </strong><a href="https://datascience.stackexchange.com/questions/123181/how-do-vector-databases-work-for-the-lay-coder?__cf_chl_tk=liZx0UseUL7oUJe2DPolOX.8F4GnkDrupN5wToH_6b0-1792390031-1.0.1.1-37UEXIj65TvNIATx.7qfeAWAd2VLlqaBwnco.nRxz.k"><strong>vector databases</strong></a><strong>, </strong><a href="https://www.geeksforgeeks.org/artificial-intelligence/metadata-filtering-in-langchain/"><strong>metadata filtering</strong></a><strong>, and retrieval pipelines</strong> to implement these capabilities.</p><h3>Why Does It Matter?</h3><p>For <strong>AI agents, personalized chatbots, and multi-step workflows</strong>, memory can improve continuity and reduce repeated instructions.</p><p>But storing more information isn’t always better. Poor retrieval can introduce irrelevant context, while careless storage can create privacy and security risks.</p><p>That’s why effective AI memory requires <strong>context management, access controls, retention policies, and memory evaluation</strong>.</p><h3>Final Thoughts</h3><p>A useful AI assistant doesn’t need to remember everything.</p><p>It needs to remember — or retrieve — the right information when it matters.</p><p><strong>Better AI memory isn’t about storing more. It’s about making the right context available at the right time.</strong> 🧠</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2dd46dd5f60c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why AI Agents Fail in Production: Retries, Timeouts, and Recovery]]></title>
            <link>https://medium.com/@siddharth.bisht.work/why-ai-agents-fail-in-production-retries-timeouts-and-recovery-879286095e39?source=rss-6fbdfb40ae92------2</link>
            <guid isPermaLink="false">https://medium.com/p/879286095e39</guid>
            <category><![CDATA[session-timeout]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[production]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[thataidev]]></dc:creator>
            <pubDate>Mon, 19 Oct 2026 15:06:01 GMT</pubDate>
            <atom:updated>2026-10-19T15:06:01.751Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JDm9grOSxI7XRTNvO8eUhw.png" /></figure><p>An <a href="https://www.ibm.com/think/topics/ai-agent-orchestration">AI agent</a> works perfectly in a demo.</p><p>It calls an API, gets the right response, and completes the task.</p><p>Then production happens.</p><p>An API times out. A tool returns an unexpected response. The model repeats an action that has already succeeded.</p><p>Suddenly, a simple workflow becomes a reliability problem.</p><h3>The Problem Isn’t Always the Model</h3><p>AI agents depend on multiple components: <a href="https://www.ibm.com/think/topics/large-language-models"><strong>LLMs</strong></a><strong>, APIs, databases, external tools, and </strong><a href="https://www.everpuredata.com/knowledge/what-is-orchestration-layer.html"><strong>orchestration layers</strong></a>. Any of them can fail, and one failure can interrupt the entire workflow.</p><p>That’s why production <a href="https://www.geeksforgeeks.org/artificial-intelligence/what-is-ai-engineer/">AI engineering</a> needs more than good prompts.</p><p>Developers use <a href="https://www.geeksforgeeks.org/system-design/timeout-strategies-in-microservices-architecture/"><strong>timeouts</strong></a> to prevent requests from waiting indefinitely and <strong>retry logic</strong> to recover from temporary failures. <a href="https://en.wikipedia.org/wiki/Exponential_backoff"><strong>Exponential backoff</strong></a> gradually increases the delay between retries, while <strong>circuit breakers</strong> help prevent repeated calls to an unhealthy service.</p><p>But retries have a catch.</p><p>If an agent retries a payment or another state-changing operation, it might execute the action twice. This is where <strong>idempotency keys</strong> and duplicate-request handling become important.</p><h3>How Do We Build More Reliable Agents?</h3><p>Reliable AI workflows combine <strong>bounded retries, error classification, structured tool outputs, state tracking, logging, and distributed tracing</strong>.</p><p>They also need clear stopping conditions. Otherwise, an agent may repeatedly call tools, consume tokens, and increase <strong>LLM inference costs</strong> without making meaningful progress.</p><h3>Final Thoughts</h3><p>An AI agent isn’t production-ready just because it succeeds when everything goes right.</p><p>It’s production-ready when developers have designed for what happens when things go wrong.</p><p><strong>Reliability isn’t about preventing every failure. It’s about recovering without turning one failure into five more.</strong> 🧠⚙️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=879286095e39" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>