<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Saman Siadati on Medium]]></title>
        <description><![CDATA[Stories by Saman Siadati on Medium]]></description>
        <link>https://medium.com/@samansiadati?source=rss-7ba39b76d27e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*GapQiQqyjPGA0IGjnI-tgw.jpeg</url>
            <title>Stories by Saman Siadati on Medium</title>
            <link>https://medium.com/@samansiadati?source=rss-7ba39b76d27e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 04:32:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@samansiadati/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AI Evaluation]]></title>
            <link>https://medium.com/aibizx/ai-evaluation-3e197c72a1b5?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e197c72a1b5</guid>
            <category><![CDATA[ai-evaluation]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 04:37:16 GMT</pubDate>
            <atom:updated>2026-11-16T04:37:42.678Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building Evaluation Habit That Scales</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oM0TLSQWfPDEbeBR" /><figcaption>Photo by <a href="https://unsplash.com/@horizonflights?utm_source=medium&amp;utm_medium=referral">Horizon flights</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>Testing a prompt once is a start. Here’s how to turn it into a routine that catches problems before your users do, as your system and team grow.</em></blockquote><p>Earlier in this series we covered how to test a prompt and how to evaluate a RAG system, each with a small test set and a handful of measures. Those are good first steps. But a test set you built in an afternoon doesn’t stay useful by itself. Models change, prompts get tweaked, documents are updated, new features arrive, and the system you tested last month isn’t quite the one running today.</p><p>This article is about the next step: making evaluation a habit that keeps working as things grow. It’s less about clever measures and more about structure, ownership and routine.</p><h3>Why evaluation decays</h3><p>Skipping evaluation causes obvious trouble. A more subtle problem is evaluation that was done once and then left to rot.</p><ul><li><strong>The test set goes stale.</strong> Real users start asking things your cases never covered.</li><li><strong>Nobody owns it.</strong> It lives in one person’s folder, and when they move on, it’s forgotten.</li><li><strong>It’s run by hand,</strong> so it’s run when someone remembers, which is not when it matters.</li><li><strong>The passing score becomes the goal.</strong> People tune the system to the test, and the test stops telling the truth.</li><li><strong>It measures what’s easy,</strong> not what matters to users.</li></ul><p>A scalable habit tackles each of these.</p><h3>Three levels of maturity</h3><p>It helps to know where you are, and what the next step looks like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*n2lSwNo3n-c6j5e0UXs0Ng.png" /></figure><p>The goal of this article is to get you to level 3 without building something heavy.</p><h3>The five parts of the habit</h3><h3>1. A test set with an owner and a version</h3><p>Treat the test set like code. Keep it in the same repository as your prompts and application, so it’s versioned and reviewed. Give it a named owner, even if that’s one person, who is responsible for keeping it healthy.</p><p>A useful structure is one file of cases, each with the information you need:</p><pre>{&quot;id&quot;: &quot;refund-001&quot;, &quot;category&quot;: &quot;policy&quot;, &quot;question&quot;: &quot;Can I return a gift card?&quot;, &quot;must_contain&quot;: [&quot;cannot be returned&quot;], &quot;must_not_contain&quot;: [&quot;refund&quot;], &quot;answerable&quot;: true, &quot;added&quot;: &quot;2026-10-02&quot;, &quot;source&quot;: &quot;support ticket&quot;}<br>{&quot;id&quot;: &quot;bike-001&quot;, &quot;category&quot;: &quot;out_of_scope&quot;, &quot;question&quot;: &quot;Do you sell bicycles?&quot;, &quot;must_contain&quot;: [&quot;can&#39;t help&quot;], &quot;answerable&quot;: false, &quot;added&quot;: &quot;2026-10-02&quot;, &quot;source&quot;: &quot;invented&quot;}</pre><p>The fields worth having:</p><ul><li><strong>A category,</strong> so you can see results by type of question, not just in total</li><li><strong>Checks,</strong> such as phrases that must or must not appear</li><li><strong>Whether it’s answerable,</strong> so you can test refusals</li><li><strong>When it was added, and where it came from,</strong> so you can see how the set has grown and whether it reflects real use</li></ul><h3>2. Categories, so you can see where you’re weak</h3><p>A single pass rate hides problems. A system that scores 92 percent overall may be failing completely on one kind of question. Group your cases into categories that match how the system is used, for example:</p><ul><li>Everyday questions</li><li>Questions needing several documents</li><li>Out-of-scope questions it should decline</li><li>Sensitive topics that need care</li><li>Attempts to break the rules</li><li>Formatting and structure requirements</li><li>Questions in other languages</li></ul><p>Then report results for each category. A drop in one is an early warning.</p><h3>3. Automatic runs on every change</h3><p>An evaluation that runs only when someone remembers isn’t a habit. Run it automatically whenever something that could change behaviour changes: a prompt edit, a code change, a model switch, a new document set. We saw how to wire an evaluation gate into a pipeline in the CI/CD article. Here’s a small runner that reports by category and fails when quality drops:</p><pre>import json<br>import sys<br>from collections import defaultdict<br><br><br>def run(cases_path, answer_fn, min_overall=0.9, min_category=0.8):<br>    totals = defaultdict(lambda: [0, 0])      # category -&gt; [passed, total]<br>    failures = []<br>    with open(cases_path) as f:<br>        for line in f:<br>            case = json.loads(line)<br>            answer = answer_fn(case[&quot;question&quot;]).lower()<br>            ok = all(p.lower() in answer for p in case.get(&quot;must_contain&quot;, []))<br>            ok = ok and not any(p.lower() in answer for p in case.get(&quot;must_not_contain&quot;, []))<br>            category = case[&quot;category&quot;]<br>            totals[category][1] += 1<br>            if ok:<br>                totals[category][0] += 1<br>            else:<br>                failures.append(case[&quot;id&quot;])<br>    passed = sum(p for p, _ in totals.values())<br>    total = sum(t for _, t in totals.values())<br>    print(f&quot;Overall: {passed}/{total} ({passed / total:.0%})&quot;)<br>    healthy = passed / total &gt;= min_overall<br>    for category, (p, t) in sorted(totals.items()):<br>        print(f&quot;  {category}: {p}/{t} ({p / t:.0%})&quot;)<br>        healthy = healthy and (p / t &gt;= min_category)<br>    if failures:<br>        print(&quot;Failed:&quot;, failures)<br>    sys.exit(0 if healthy else 1)</pre><p>The answer_fn is whatever calls your system. This is a sketch I haven&#39;t run, and the simple phrase checks are crude, but the structure is the point: results by category, a floor for each, and a non-zero exit code that stops a bad release.</p><h3>4. Monitoring in production</h3><p>Your test set covers what you thought of. Production shows what you didn’t. Pick a few signals and watch them over time:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*rVpJfl8rRl-FbGi2RiqqQg.png" /></figure><p>Sample real conversations on a schedule, with proper regard for privacy, and have someone read them. Nothing replaces reading real examples.</p><h3>5. A loop that feeds failures back in</h3><p>This is what keeps the set alive. Every real failure becomes a test case:</p><ol><li>A user complaint or a reviewed conversation shows a bad answer.</li><li>Someone writes a test case for it, with the right expected behaviour.</li><li>The case is added to the set, with its source recorded.</li><li>The system is fixed, and the new case keeps it from regressing.</li></ol><p>Over time, your test set becomes a record of everything you’ve learned about how the system fails.</p><h3>Choosing the right checks</h3><p>Phrase matching is a starting point. As your system matures, use a mix:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*iRLYZu-Sc562XhbPoGQEUQ.png" /></figure><p>If you use a model as a judge, check it against human scores on a sample, keep the judge’s prompt and model version fixed, and re-check it periodically, because changing the judge changes your scores.</p><h3>Avoiding the traps</h3><p><strong>Tuning to the test.</strong> If you keep editing the system until the set passes, you’ve fitted it to the set. Keep a hidden slice of cases that you don’t look at while you tune, and check it at the end.</p><p><strong>Leakage.</strong> If a test question also appears as an example in your prompt, or in the documents the system retrieves from, it will pass for the wrong reason.</p><p><strong>Chasing one number.</strong> When a score becomes a target, people optimise the score and not the quality. Look at categories and at real conversations too.</p><p><strong>Noise mistaken for change.</strong> Model output varies. A one-point drop might be chance. Run important comparisons more than once, and look at which cases flipped.</p><p><strong>Averages hiding harm.</strong> A tiny category, such as sensitive topics, can matter more than its share of the total. Give high-stakes categories their own, stricter threshold.</p><p><strong>An eval set that no longer matches real use.</strong> Compare your categories with what users actually ask, and rebalance.</p><h3>Who does what</h3><p>Evaluation works best when responsibility is clear.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*mx_zQVEZzCBbYWNTML5mnA.png" /></figure><p>One person can hold several of these roles in a small team. What matters is that someone does each.</p><h3>A short evaluation plan you can copy</h3><p>Write this on one page, and keep it with the project:</p><ol><li><strong>What we’re evaluating:</strong> the system, and what “good” means for it</li><li><strong>The test set:</strong> where it lives, who owns it and how it grows</li><li><strong>Categories and thresholds:</strong> including stricter ones for high-stakes areas</li><li><strong>When it runs:</strong> on every prompt, code, model or document change</li><li><strong>Checks used:</strong> rules, references, people, model judge, and how the judge is calibrated</li><li><strong>Production signals:</strong> which we watch, and who reviews samples and how often</li><li><strong>The feedback loop:</strong> how failures become test cases</li><li><strong>What happens when it fails:</strong> who decides, and what the rollback plan is</li></ol><h3>A checklist</h3><ol><li>Does your test set live in version control, with a named owner?</li><li>Are cases grouped into categories, with results reported for each?</li><li>Does the evaluation run automatically on every relevant change?</li><li>Do you have a stricter threshold for high-stakes categories?</li><li>Do you watch a few production signals, and read real conversations?</li><li>Does every real failure become a test case?</li><li>Have you kept a hidden slice to guard against tuning to the test?</li><li>Is any model judge calibrated against human scores?</li></ol><h3>The takeaway</h3><p>Evaluation that scales is mostly structure: a versioned, owned test set grouped by category, run automatically on every change, backed by monitoring in production, and fed by real failures. Do that, and you’ll catch most problems before your users do, and learn something from the ones you don’t.</p><h3>Where we go next</h3><p>Next, we’ll look at bias in AI: where it comes from, how it shows up in the real world and simple ways to check for it.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e197c72a1b5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/ai-evaluation-3e197c72a1b5">AI Evaluation</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Memory]]></title>
            <link>https://medium.com/aibizx/agent-memory-eb5634e2a300?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/eb5634e2a300</guid>
            <category><![CDATA[ai-agent-memory]]></category>
            <category><![CDATA[ai-agent-development]]></category>
            <category><![CDATA[ai-agnets]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 20:48:41 GMT</pubDate>
            <atom:updated>2026-11-13T20:48:41.172Z</atom:updated>
            <content:encoded><![CDATA[<h4>Short-Term, Long-Term and What to Store</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tkb8pbZbrMuZBPu9" /><figcaption>Photo by <a href="https://unsplash.com/@mrsunburnt?utm_source=medium&amp;utm_medium=referral">Xu Haiwei</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>A model starts every conversation knowing nothing about you. Memory is what makes an agent feel continuous, and what makes it risky. Here’s how it works, what’s worth remembering and how to keep it safe.</em></blockquote><p>Earlier in this series we learned that a chat model doesn’t really remember anything. Each time you send a message, the whole conversation is sent again, and the model reads it fresh. Once the conversation ends, or outgrows the context window, what was said is gone.</p><p>An agent that works for you over days or weeks can’t live like that. It needs to remember what you asked, what it already tried and what you prefer. That’s the job of <strong>memory</strong>, and it’s one of the least understood parts of agent design. This article explains the two kinds, the main design patterns and, importantly, what you should and shouldn’t remember.</p><h3>Two kinds of memory</h3><h3>Short-term memory: the current task</h3><p>Short-term memory is what the agent holds during one conversation or task: the messages so far, the tool results, its plan and what it has done. In practice this is simply the context window, the text the model sees on each step.</p><p>It’s reliable and automatic, but it has hard limits:</p><ul><li><strong>It’s limited in size.</strong> Long tasks fill the window.</li><li><strong>It costs money,</strong> because everything in it is sent, and billed, on every step.</li><li><strong>It vanishes</strong> when the session ends.</li><li><strong>Long contexts can be used unevenly,</strong> with detail in the middle sometimes ignored.</li></ul><h3>Long-term memory: across sessions</h3><p>Long-term memory is information saved outside the model, so it can be brought back later: your preferences, past decisions, facts about your projects, summaries of earlier work. The model itself doesn’t change. Your system stores things in a database and puts the relevant ones back into the prompt when needed.</p><p>That’s the key point: <strong>long-term memory is a retrieval problem.</strong> You decide what to save, where to keep it and what to bring back into the context for each new task.</p><h3>The kinds of things worth remembering</h3><p>Researchers often borrow terms from human memory to describe what an agent might store:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*XBtW2MmGie3x2s0HssI-qA.png" /></figure><p>You don’t need all three. Pick what makes your agent more useful, and keep it small.</p><h3>How it works in practice</h3><p>Most long-term memory systems follow the same cycle:</p><ol><li><strong>During or after a conversation, decide what’s worth keeping.</strong> Often a model is asked to pull out the important facts or write a short summary.</li><li><strong>Store it,</strong> typically as short text entries in a database, often with an embedding so it can be searched by meaning, plus metadata such as the date, the user and the source.</li><li><strong>At the start of a new task, retrieve the relevant memories</strong> with a search, using the same retrieval ideas from the RAG series.</li><li><strong>Put them in the prompt,</strong> clearly labelled as remembered information.</li><li><strong>Update or retire them</strong> as things change.</li></ol><p>Here’s a deliberately simple sketch of the idea. The call_model function stands in for your model call, and a real system would use a proper database with permissions and encryption:</p><pre>import json<br>from datetime import date<br><br>memory = {}   # user_id -&gt; list of short facts (use a real database in practice)<br><br>def remember(user_id, conversation, call_model):<br>    prompt = (<br>        &quot;From the conversation below, list any lasting facts or preferences about the &quot;<br>        &quot;user that would help in future conversations. Return a JSON list of short &quot;<br>        &quot;strings. Exclude passwords, financial details, health information and &quot;<br>        &quot;anything the user asked you not to keep. Return [] if there is nothing.\n\n&quot;<br>        f&quot;&lt;conversation&gt;\n{conversation}\n&lt;/conversation&gt;&quot;<br>    )<br>    facts = json.loads(call_model(prompt))<br>    for fact in facts:<br>        memory.setdefault(user_id, []).append({&quot;fact&quot;: fact, &quot;saved&quot;: str(date.today())})<br><br>def recall(user_id, limit=5):<br>    # A real system would search by relevance to the new task.<br>    entries = memory.get(user_id, [])[-limit:]<br>    return &quot;\n&quot;.join(f&quot;- {e[&#39;fact&#39;]} (saved {e[&#39;saved&#39;]})&quot; for e in entries)<br><br>def forget(user_id):<br>    memory.pop(user_id, None)    # always provide a way to delete</pre><p>Notice three things: the extraction prompt tells the model what <em>not</em> to keep, each memory is dated and there’s a forget function. Those aren&#39;t nice-to-haves. They&#39;re the basics of doing this responsibly.</p><h3>Two common designs for conversation memory</h3><p><strong>A rolling summary.</strong> As a conversation grows, keep the recent messages in full and replace older ones with a short summary. It keeps the context small while preserving the thread.</p><p><strong>Extract and retrieve.</strong> Pull out specific facts and store them separately, then retrieve only the relevant ones when needed. It’s more work, but it scales to long relationships and many topics.</p><p>Many systems use both: summaries within a session, and extracted facts across sessions. Some cloud platforms offer memory as a managed service. For example, AWS’s AgentCore has a memory capability with short-term session context and optional long-term memory, so you don’t have to build the storage yourself. You still have to decide what’s worth keeping.</p><h3>What to store, and what to leave out</h3><p>This is the part that deserves the most thought. More memory isn’t better.</p><p><strong>Good candidates:</strong></p><ul><li>Stable preferences the user has stated, such as format, tone and language</li><li>Context that’s costly to re-explain, such as the structure of a project</li><li>Decisions and their reasons</li><li>Outcomes of past attempts, so the agent doesn’t repeat failures</li></ul><p><strong>Think twice about, or avoid:</strong></p><ul><li><strong>Passwords, keys and payment details.</strong> Never store these in an agent’s memory.</li><li><strong>Sensitive personal information,</strong> such as health, finances, identity documents and anything about other people. If you must keep it, you need a clear reason, consent and strong protection.</li><li><strong>Guesses about the user.</strong> An agent that stores “seems anxious about money” has stored an inference, not a fact. Keep to what was stated.</li><li><strong>Temporary details,</strong> such as “is travelling this week,” which will go stale quickly.</li><li><strong>Anything the user asked you to forget.</strong></li><li><strong>Other people’s details,</strong> which aren’t yours to keep.</li></ul><p>A useful test before saving anything: <em>Would the user be comfortable seeing this stored, and would it really improve the next conversation?</em> If not, leave it out.</p><h3>The risks</h3><p>Memory turns a one-off interaction into a lasting record, and that creates risks worth planning for.</p><p><strong>Privacy.</strong> A memory store is a pile of personal information. Treat it accordingly: encrypt it, limit who can read it, set retention periods and follow the rules that apply to personal data where you operate.</p><p><strong>Stale or wrong memories.</strong> Facts change. If the agent confidently acts on an outdated memory, it can look foolish or do harm. Date every memory, and let newer information override older.</p><p><strong>Memory poisoning.</strong> Memories can be written from content the agent read, such as a web page or an email. An attacker can plant text that the agent saves, such as “the user always wants invoices sent to this address,” and it then influences every future task. This is prompt injection with a long-lasting effect. Don’t save instructions found in untrusted content, and review what’s being stored.</p><p><strong>Over-personalisation.</strong> An agent that brings up old details at odd moments feels intrusive, and can be unsettling. Use memories only where they change the answer.</p><p><strong>Bias and false inferences.</strong> A stored judgement about a person can quietly shape how the agent treats them forever.</p><p><strong>Cross-user leakage.</strong> Memories must be kept apart per user, and per tenant in a business setting. A bug that mixes them is a serious breach.</p><p><strong>Cost.</strong> Retrieved memories take space in the prompt, and every token is billed. Retrieve only what’s relevant.</p><h3>Giving users control</h3><p>Good memory design puts the person in charge.</p><ul><li><strong>Make it visible.</strong> Let people see what’s been remembered.</li><li><strong>Let them edit and delete,</strong> individual items or everything.</li><li><strong>Be transparent</strong> about what’s saved and why.</li><li><strong>Offer an off switch,</strong> or a temporary mode where nothing is stored.</li><li><strong>Honour deletion fully,</strong> including from backups and any search indexes, within a reasonable time.</li></ul><h3>How to test memory</h3><p>Test it like any other feature, using the habits from the evaluation articles.</p><ol><li><strong>Does it remember what it should?</strong> State a preference, end the session, start a new one and check.</li><li><strong>Does it forget what it should?</strong> Ask it to delete something, and check it’s gone.</li><li><strong>Does it ignore irrelevant memories?</strong> Ask something unrelated and check it doesn’t drag old details in.</li><li><strong>Does it handle updates?</strong> Change a preference and check the new one wins.</li><li><strong>Can it be poisoned?</strong> Put planted instructions in a document the agent reads, and check they aren’t saved.</li><li><strong>Is it isolated?</strong> Check that one user’s memories never appear for another.</li></ol><h3>A short checklist</h3><ol><li>Have you decided what the agent needs to remember, and kept it small?</li><li>Do you store only stated facts, not guesses?</li><li>Are passwords, payment details and sensitive data excluded?</li><li>Is each memory dated, and can newer information replace older?</li><li>Can users see, edit and delete their memories?</li><li>Do you refuse to save instructions found in untrusted content?</li><li>Are memories encrypted, access-controlled and kept separate per user?</li><li>Have you tested remembering, forgetting and poisoning?</li></ol><h3>The takeaway</h3><p>Short-term memory is the context window, and long-term memory is information you save and retrieve. The craft is in choosing what to keep: stated facts and useful context, kept small, dated and under the user’s control, and never secrets or guesses. Remember that a memory store is both a feature and a liability, and design it that way.</p><h3>Where we go next</h3><p>That completes our short series on agents. Next, we’ll turn to evaluation and safety in more depth, starting with how to build an AI evaluation habit that scales.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eb5634e2a300" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/agent-memory-eb5634e2a300">Agent Memory</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Model Context Protocol (MCP)]]></title>
            <link>https://medium.com/aibizx/the-model-context-protocol-mcp-dd40af46b6a6?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd40af46b6a6</guid>
            <category><![CDATA[aws-bedrock]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[mcp-protocol]]></category>
            <category><![CDATA[mcp-client]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 20:37:11 GMT</pubDate>
            <atom:updated>2026-11-13T20:37:11.359Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why Everyone Is Talking About It</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Qw2YmgY3-riCx_b-" /><figcaption>Photo by <a href="https://unsplash.com/@growtika?utm_source=medium&amp;utm_medium=referral">Growtika</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>A standard plug for connecting AI applications to tools and data. What problem it solves, how it works, and the security questions you should ask before you plug anything in.</em></blockquote><p>If you’ve read anything about AI agents lately, you’ve seen the letters MCP. They stand for the <strong>Model Context Protocol</strong>, and they’re attached to announcements from AI labs, cloud providers, coding tools and plenty of startups.</p><p>The idea behind it is simple, and the excitement makes more sense once you see the problem it solves. This article explains MCP in plain language, shows how the pieces fit together, and covers what to watch out for.</p><h3>The problem: too many custom connections</h3><p>In the last two articles we saw that agents use tools, and that each tool needs a description and some code. Now imagine you have several AI applications: a chat assistant, a coding tool and an internal agent. And imagine you have several things they should connect to: your calendar, your documents, your ticket system and your database.</p><p>Without a standard, every application needs its own custom connection to every tool. Three apps and four tools means twelve separate integrations, each built and maintained by someone. Add a new tool and you build it three times.</p><p>That’s the same problem the world had with connecting devices before USB. Every gadget needed its own cable. A common plug meant any device could work with any computer.</p><p>MCP is meant to be that common plug for AI. Anthropic introduced it in late 2024 as an open standard, and many other vendors have since adopted it.</p><h3>The idea</h3><p>With MCP, a tool provider writes one <strong>server</strong> that exposes its capabilities in a standard way. Any AI application that speaks MCP can then connect to it. Build once, and it works everywhere.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*vBWy0-RGUZy_otvQuFHiTA.png" /></figure><p>A host can connect to many servers at once, and a server can be used by many hosts.</p><h3>What a server can offer</h3><p>An MCP server can provide three main kinds of thing:</p><p><strong>Tools.</strong> Actions the model can ask to run, such as “create a calendar event” or “search tickets.” This is the function calling we covered last time, in a standard wrapper.</p><p><strong>Resources.</strong> Data the application can read and give to the model as context, such as files, records or documents.</p><p><strong>Prompts.</strong> Reusable templates for common tasks that the server provides, which users can choose.</p><p>Tools get most of the attention, but the three together are what make a server useful.</p><h3>How it works, briefly</h3><p>Under the surface, MCP uses a simple request and response message format, based on JSON. When a host connects to a server, they exchange information about what each can do. The host then learns the server’s tools and their descriptions, and makes those available to the model. When the model asks to use a tool, the host sends the request to the server, which runs it and returns the result.</p><p>There are two common ways for the messages to travel:</p><ul><li><strong>Locally,</strong> where the host starts the server as a program on your own computer and talks to it directly. Many desktop and coding tools work this way.</li><li><strong>Over the network,</strong> using HTTP, for servers that run remotely. This is how hosted and shared servers work.</li></ul><p>A remote server needs authentication, so that only the right people and applications can use it. The protocol includes provisions for this, and the details have developed over time. Check the current specification if you’re building one.</p><h3>Why it caught on</h3><p><strong>Build once.</strong> A company that offers a service can publish one server instead of building integrations for every AI app.</p><p><strong>Choice for users.</strong> You can switch AI applications without losing your connections.</p><p><strong>A growing library.</strong> There are now many servers available for common tools and services.</p><p><strong>Fits agents.</strong> It gives agents a consistent way to discover and use tools, which makes agent products quicker to build.</p><p><strong>Industry support.</strong> Major AI companies and cloud providers have added support. AWS, for example, has features that let you expose your own functions and APIs as MCP tools for agents, and AWS’s Kiro development environment supports MCP. It has also been reported that Anthropic moved the project to a neutral foundation under the Linux Foundation, but check the current governance details yourself.</p><h3>What it doesn’t do</h3><p>It’s worth being clear about the limits, because the buzz can blur them.</p><ul><li><strong>It doesn’t make a model smarter.</strong> It makes connections easier.</li><li><strong>It isn’t a security system.</strong> It’s a way to connect things. How safely you connect them is up to you.</li><li><strong>It doesn’t guarantee tool quality.</strong> A badly described or badly built tool is still badly described or badly built.</li><li><strong>It isn’t the only approach.</strong> Plain function calling with your own code, or a vendor’s own plugin system, can be simpler for one-off needs.</li><li><strong>It’s still maturing.</strong> The standard has evolved quickly, so details change between versions.</li></ul><h3>The security side: take this seriously</h3><p>MCP’s strength, how easily you can connect an AI application to powerful tools, is also its main risk. Connecting a tool means letting a model, and anything that can influence the model, act through it.</p><p><strong>Malicious or careless servers.</strong> An MCP server is software that runs with whatever access you give it, and it can run on your machine. Installing one from an unknown source is like installing any unvetted program. Use servers from sources you trust, read what they do and keep them updated.</p><p><strong>Over-broad permissions.</strong> A server that can read all your files, or act on all your accounts, can do harm if the model is steered wrong. Give each server the narrowest access that works.</p><p><strong>Prompt injection.</strong> Text that comes back through a tool, such as a web page, a document or a ticket, can contain hidden instructions. A model connected to powerful tools may follow them. We covered this in the article on prompt injection, and it’s the central risk with agents and MCP.</p><p><strong>Tool descriptions can be poisoned.</strong> The descriptions a server gives its tools are read by the model. A malicious server can hide instructions in them. Another reason to trust your sources.</p><p><strong>Tools that change after you approve them.</strong> A server can alter its tool descriptions later. Be wary of connections that update silently.</p><p><strong>Combining servers.</strong> The risks multiply when one application connects to a server that reads private data, another that handles untrusted content and a third that can send messages out. That’s the combination to avoid.</p><p><strong>Credentials.</strong> Don’t paste long-lived keys into configuration files you share or commit. Use the narrowest, shortest-lived credentials available.</p><p><strong>Approvals.</strong> Many hosts ask you to approve tool calls. Read what you’re approving, especially anything that sends, deletes or spends.</p><h3>Practical rules for using MCP safely</h3><ol><li><strong>Only connect servers you trust,</strong> and prefer official ones from the service’s own maker.</li><li><strong>Start with read-only tools.</strong></li><li><strong>Limit each server’s permissions</strong> to what the task needs.</li><li><strong>Keep a person in the loop</strong> for actions that change things.</li><li><strong>Don’t mix private data, untrusted content and outbound actions</strong> in one setup.</li><li><strong>Pin and review versions</strong> of the servers you run, and know when they change.</li><li><strong>Log tool calls,</strong> and review them.</li><li><strong>Treat everything a tool returns as untrusted text.</strong></li></ol><h3>When to build your own server</h3><p>Consider it when several AI applications need the same capability, when you want a service to be usable by other people’s tools or when your team wants one governed way to expose internal systems. For a single application with a couple of tools, plain function calling is simpler. If you do build one, apply the same discipline as for any tool: small, well-described functions, validated inputs, narrow permissions, authentication for anything remote and logging.</p><h3>A short checklist</h3><ol><li>Do you need a standard connection, or would simple function calling do?</li><li>Do you trust the source of each MCP server you plan to use?</li><li>Have you limited each server’s access to what the task needs?</li><li>Are risky actions approved by a person?</li><li>Have you avoided combining private data, untrusted content and outbound actions?</li><li>Do you know how updates to a server’s tools reach you?</li><li>Are credentials narrow and short-lived?</li><li>Are tool calls logged and reviewed?</li></ol><h3>The takeaway</h3><p>MCP is a common plug for connecting AI applications to tools and data, so a tool built once works with many applications. It solves a real integration problem and has been widely adopted. But it makes powerful connections easy, so it raises the stakes on trust, permissions and approvals. Use it, and treat every server as software you’re letting act on your behalf.</p><h3>Where we go next</h3><p>Next, we’ll look at agent memory: short-term and long-term, what to store and what to leave out.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd40af46b6a6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/the-model-context-protocol-mcp-dd40af46b6a6">The Model Context Protocol (MCP)</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tool Use and Function Calling]]></title>
            <link>https://medium.com/aibizx/tool-use-and-function-calling-593a4491630e?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/593a4491630e</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[amazon-bedrock]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Fri, 13 Nov 2026 02:36:15 GMT</pubDate>
            <atom:updated>2026-11-13T02:39:44.750Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>A Hands-on Guide with Python and AWS</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VLtkSW6qKITXhYaq" /><figcaption>Photo by <a href="https://unsplash.com/@zoeey97?utm_source=medium&amp;utm_medium=referral">Zoha Gohar</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>A language model can’t look up an order or send an email. But it can ask your code to. Here’s how that works, step by step, with a small example you can run.</em></p><p>In the last article we defined an agent as a system where a model decides what to do and uses tools to do it. That raises a very practical question: how does a model, which only produces text, actually use a tool?</p><p>The answer is simpler than it sounds, and understanding it clears up a lot of confusion about what agents can and can’t do. It’s called <strong>tool use</strong>, or <strong>function calling</strong>, and it’s one of the most important ideas in building with AI.</p><h3>The key idea: the model asks, your code acts</h3><p>A model never runs anything. It can’t call an API, open a file or query a database. All it can do is write text.</p><p>Tool use is an agreement about what that text looks like. You tell the model what tools exist. When it decides one would help, it writes a structured request, such as “please call get_order_status with order ID 1001.&quot; Your code sees that request, runs the real function and sends the result back. The model then reads the result and carries on.</p><p>So the model is like a manager who can only write notes. The tools are staff who do the work, and your code is the one carrying the notes back and forth.</p><h3>The five steps</h3><ol><li><strong>You describe the tools.</strong> For each one, you give a name, a plain-language description and the inputs it accepts.</li><li><strong>You send the model the user’s question and the tool descriptions.</strong></li><li><strong>The model decides.</strong> If it can answer directly, it does. If a tool would help, it replies with a tool request instead of a final answer.</li><li><strong>Your code runs the tool</strong> and gets a result.</li><li><strong>You send the result back to the model,</strong> which uses it to write the final answer, or to request another tool.</li></ol><p>Steps 3 to 5 can repeat. That repetition is the agent loop.</p><h3>What a tool description looks like</h3><p>The model never sees your code, only the description. It’s a short specification, usually in a standard format called JSON Schema. For an order lookup tool:</p><pre>{<br>  &quot;name&quot;: &quot;get_order_status&quot;,<br>  &quot;description&quot;: &quot;Look up the shipping status of an order by its numeric order ID.&quot;,<br>  &quot;input_schema&quot;: {<br>    &quot;type&quot;: &quot;object&quot;,<br>    &quot;properties&quot;: {<br>      &quot;order_id&quot;: {<br>        &quot;type&quot;: &quot;string&quot;,<br>        &quot;description&quot;: &quot;The order ID, for example 1001&quot;<br>      }<br>    },<br>    &quot;required&quot;: [&quot;order_id&quot;]<br>  }<br>}</pre><p>The exact field names differ between providers, but the content is the same: a name, a description and the inputs.</p><p>This description is doing the job of a prompt. The model decides when to use the tool, and what to pass it, based on these words alone. A vague description gives you a tool that gets used at the wrong times.</p><h3>A worked example</h3><p>Here’s the whole loop in Python. This example uses the Converse API in Amazon Bedrock, which has a standard way to express tools, but the pattern is the same elsewhere. The model ID is a placeholder, and I haven’t run this exact code, so check it against your provider’s documentation.</p><pre>import boto3<br><br><br>client = boto3.client(&quot;bedrock-runtime&quot;, region_name=&quot;us-east-1&quot;)<br>MODEL_ID = &quot;YOUR_MODEL_ID&quot;   # a model that supports tool use<br>ORDERS = {&quot;1001&quot;: &quot;shipped, due in 2 days&quot;, &quot;1002&quot;: &quot;being prepared&quot;}<br><br>def get_order_status(order_id):<br>    return {&quot;status&quot;: ORDERS.get(str(order_id), &quot;not found&quot;)}<br><br>tool_config = {<br>    &quot;tools&quot;: [<br>        {<br>            &quot;toolSpec&quot;: {<br>                &quot;name&quot;: &quot;get_order_status&quot;,<br>                &quot;description&quot;: &quot;Look up the shipping status of an order by its numeric order ID.&quot;,<br>                &quot;inputSchema&quot;: {<br>                    &quot;json&quot;: {<br>                        &quot;type&quot;: &quot;object&quot;,<br>                        &quot;properties&quot;: {&quot;order_id&quot;: {&quot;type&quot;: &quot;string&quot;}},<br>                        &quot;required&quot;: [&quot;order_id&quot;],<br>                    }<br>                },<br>            }<br>        }<br>    ]<br>}<br>messages = [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: [{&quot;text&quot;: &quot;Where is order 1001?&quot;}]}]<br>for _ in range(5):                     # a hard limit on the number of steps<br>    response = client.converse(modelId=MODEL_ID, messages=messages, toolConfig=tool_config)<br>    message = response[&quot;output&quot;][&quot;message&quot;]<br>    messages.append(message)<br>    if response[&quot;stopReason&quot;] != &quot;tool_use&quot;:<br>        print(message[&quot;content&quot;][0][&quot;text&quot;])      # a final answer<br>        break<br>    results = []<br>    for block in message[&quot;content&quot;]:<br>        if &quot;toolUse&quot; in block:<br>            call = block[&quot;toolUse&quot;]<br>            if call[&quot;name&quot;] == &quot;get_order_status&quot;:<br>                output = get_order_status(**call[&quot;input&quot;])<br>            else:<br>                output = {&quot;error&quot;: &quot;unknown tool&quot;}<br>            results.append({&quot;toolResult&quot;: {&quot;toolUseId&quot;: call[&quot;toolUseId&quot;], &quot;content&quot;: [{&quot;json&quot;: output}]}})<br>    messages.append({&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: results})</pre><p>Walk through what happens:</p><ol><li>The user asks, “Where is order 1001?”</li><li>The model replies with a tool request: call get_order_status with order_id set to &quot;1001&quot;. The response&#39;s stop reason says it stopped to use a tool.</li><li>The code runs the function and sends back the result inside a message.</li><li>The model reads “shipped, due in 2 days” and writes a normal sentence for the user.</li></ol><p>Notice the loop has a hard limit of five rounds. That’s not decoration. It’s what stops a confused model from calling tools forever.</p><h3>The details that matter</h3><p><strong>The model can request several tools at once.</strong> Some models ask for multiple tools in one turn. Your code should handle a list of requests, as the loop above does.</p><p><strong>The model’s arguments are untrusted input.</strong> It chooses the values, and it can get them wrong, or be steered by hostile text in the conversation. Validate every argument before you act on it: check types, ranges and allowed values.</p><p><strong>Tool results go back into the conversation.</strong> Everything a tool returns becomes text the model reads. If a result contains instructions, such as text from a web page, the model may follow them. This is prompt injection again. Treat tool output as untrusted, and return only what’s needed.</p><p><strong>Errors are part of the conversation.</strong> If a tool fails, send back a clear error message instead of crashing. The model can often recover, for example by trying again with corrected input or telling the user what went wrong.</p><p><strong>You decide what’s allowed.</strong> Your code is the gatekeeper. If the model asks for a tool it shouldn’t use, or in a way that breaks your rules, don’t run it.</p><h3>Writing good tools</h3><ul><li><strong>Make each tool do one clear thing.</strong> “Get order status” is easier for a model to use correctly than “manage orders.”</li><li><strong>Describe it well.</strong> Say what it does, when to use it, and what each input means. Give an example value.</li><li><strong>Keep the number of tools small.</strong> Models choose worse when they face a long menu. If you have dozens, group them or load only the ones that are relevant.</li><li><strong>Name things plainly.</strong> get_order_status beats ordstat2.</li><li><strong>Return short, useful results.</strong> Long, messy output costs tokens and confuses the model.</li><li><strong>Use fixed choices</strong> where you can, such as a list of allowed values, so the model can’t invent one.</li></ul><h3>Safety: keep the tools on a short leash</h3><p>Tools are where an AI system touches the real world, so this is where risks live.</p><ul><li><strong>Start read-only.</strong> Tools that look things up are far safer than tools that change things.</li><li><strong>Give each tool the narrowest permissions it needs,</strong> so a tricked model can only do limited damage.</li><li><strong>Require approval for risky actions,</strong> such as sending messages, deleting data or spending money. Show the person exactly what will happen.</li><li><strong>Limit the loop</strong> with a step limit, a time limit and a budget.</li><li><strong>Log every tool call and its result,</strong> with care for personal data, so you can see what happened.</li><li><strong>Make actions safe to repeat where you can.</strong> A model or a network may retry a call, and a refund should never happen twice.</li></ul><h3>Tool use versus structured output</h3><p>These two ideas are cousins. In both, the model produces data in a fixed shape. The difference is the purpose. With structured output, you want the data itself, such as a classification. With tool use, the data is a request for your code to act. Some people use the tool format purely to get structured output, which works well.</p><h3>A short checklist</h3><ol><li>Does each tool have a clear name and a good description with example values?</li><li>Do you validate every argument before running anything?</li><li>Is there a hard limit on the number of loop rounds?</li><li>Do tools run with the least permissions possible?</li><li>Do risky actions need human approval?</li><li>Do you treat tool results as untrusted text?</li><li>Do you handle errors by sending a clear message back?</li><li>Do you log tool calls?</li></ol><h3>The takeaway</h3><p>Tool use is a simple agreement: you describe your functions, the model asks for them in a structured way, and your code decides whether to run them and sends back the result. Everything an agent does in the real world passes through that gate, so that’s where to put your validation, limits and approvals.</p><h3>Where we go next</h3><p>Next, we’ll look at the Model Context Protocol, or MCP: a standard way to connect models to tools and data, and why so many people are talking about it.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=593a4491630e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/tool-use-and-function-calling-593a4491630e">Tool Use and Function Calling</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is an AI Agent?]]></title>
            <link>https://medium.com/aibizx/what-is-an-ai-agent-1788bcc65a2d?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/1788bcc65a2d</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 22:45:37 GMT</pubDate>
            <atom:updated>2026-11-12T22:45:37.003Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Plain-Language Guide</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_WKCgBx_FwjPKYUK" /><figcaption>Photo by <a href="https://unsplash.com/@julientromeur?utm_source=medium&amp;utm_medium=referral">julien Tromeur</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>“Agent” is the most overused word in AI right now. Here’s what it actually means, how it differs from a chatbot or a workflow, and how to start without getting burned.</em></p><p>If you’ve followed AI news this year, you’ve been told that agents are going to do your admin, run your support desk and maybe your whole company. You’ve also seen “agent” attached to products that are really just chatbots with a new sticker.</p><p>This article cuts through it. It gives you a working definition, a ladder from simple to complex systems, and a sensible way to begin. It’s the start of a short series on agents, written without any particular vendor in mind.</p><h3>A definition you can use</h3><p>Here’s a definition that holds up in practice:</p><p><strong>An AI agent is a system where a language model decides what to do next, uses tools to do it, looks at the result, and repeats until the job is done.</strong></p><p>The key phrase is <em>decides what to do next</em>. In a normal chat, you ask and the model answers. In an agent, the model is given a goal and some tools, and it works out the steps itself.</p><p>An example. Suppose you tell a system: “Find out why customer 4471 was charged twice and draft a reply.” An agent might:</p><ol><li>Look up the customer’s orders using a tool.</li><li>Notice two charges and check the payment records using another tool.</li><li>Find that one was a retry after a timeout.</li><li>Check the refund policy in your documents.</li><li>Draft a reply explaining what happened and offering a refund.</li></ol><p>Nobody told it those five steps. It chose them, one at a time, based on what it found.</p><h3>The ladder from chatbot to agent</h3><p>Not everything called an agent is one. It helps to see the levels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/778/1*lMkOBRIsCO-9lVRFhwjIIA.png" /></figure><p>Most useful business systems today sit at levels 2 and 3, and that’s fine. A well-built workflow is often more reliable and cheaper than an agent. Level 4 is for jobs where the steps genuinely can’t be predicted in advance.</p><h3>What an agent is made of</h3><p>Every agent, whichever tools it’s built with, has the same few parts.</p><p><strong>A model</strong> that reasons and makes decisions.</p><p><strong>Instructions</strong> that set its role, rules and limits: what it should do, what it must never do, when to stop.</p><p><strong>Tools</strong> that let it act: searching documents, querying a database, calling an API, running a calculation, sending a message.</p><p><strong>Memory.</strong> Short-term, so it keeps track of what it has done in this task, and sometimes long-term, so it can remember across sessions.</p><p><strong>A loop</strong> that ties it together: think, act, look at the result, repeat. This is what makes it an agent instead of a one-off call.</p><p><strong>Limits and oversight:</strong> step limits, time limits, permissions, approval rules and logging. These are not optional extras. They’re what makes an agent safe to run.</p><h3>What agents are good at</h3><ul><li><strong>Tasks with variable steps,</strong> where what to do next depends on what turns up, such as investigating an issue</li><li><strong>Gathering information from several places</strong> and putting it together</li><li><strong>Routine multi-step work</strong> that currently eats up staff time</li><li><strong>Research and analysis,</strong> where the model can search, read and follow leads</li><li><strong>Software tasks,</strong> such as finding a bug, making a change and running the tests</li></ul><h3>Where they struggle</h3><p>Be clear-eyed about the weaknesses, because they’re where the problems come from.</p><ul><li><strong>Reliability drops with every step.</strong> If each step is 95 percent reliable, a ten-step task succeeds well under 70 percent of the time. Errors compound.</li><li><strong>They can wander.</strong> An agent can loop, repeat itself or chase irrelevant leads, burning time and money.</li><li><strong>They can choose the wrong tool or give it bad input.</strong></li><li><strong>They’re hard to predict and to test.</strong> The same request can follow different paths.</li><li><strong>They cost more.</strong> Every step is a model call, so a ten-step task costs roughly ten times a single answer.</li><li><strong>They can be manipulated.</strong> Text an agent reads, such as a web page or email, can contain hidden instructions, as we covered in the article on prompt injection.</li><li><strong>They overreach if you let them.</strong> An agent with broad permissions can do broad damage.</li></ul><h3>When you should and shouldn’t build one</h3><p><strong>Build an agent when:</strong></p><ul><li>the path to the answer genuinely varies from case to case</li><li>the task is valuable enough to justify the cost</li><li>a mistake is cheap, or a person checks the result before it matters</li><li>you can describe what “done” looks like</li></ul><p><strong>Use a simpler design when:</strong></p><ul><li>you can draw the steps on a whiteboard in advance, so a workflow will do</li><li>a single prompt, perhaps with retrieval, already works</li><li>an error would be costly and no one will check</li><li>speed and predictability matter more than flexibility</li></ul><p>A good rule of thumb: <strong>start with the simplest thing that works, and add autonomy only where you need it.</strong></p><h3>How to start safely</h3><p>A sensible path for a first agent:</p><ol><li><strong>Pick a narrow, low-risk task.</strong> For example, drafting a reply for a person to approve, not sending it.</li><li><strong>Start read-only.</strong> Let it look things up and propose actions, but not take them.</li><li><strong>Give it the fewest tools it needs,</strong> each with the narrowest permissions.</li><li><strong>Set hard limits:</strong> a maximum number of steps, a time limit and a budget.</li><li><strong>Require approval</strong> for anything that sends, deletes, buys or publishes.</li><li><strong>Log every step,</strong> so you can see what it did and why, with proper care for personal data.</li><li><strong>Test with realistic tasks,</strong> including awkward and hostile ones, before real users see it.</li><li><strong>Review a sample of its work every week,</strong> at first.</li><li><strong>Expand slowly,</strong> only once it has earned trust on the smaller job.</li></ol><h3>Questions to ask about any “agent” product</h3><p>When a vendor says their product is an agent, these questions cut through the marketing:</p><ul><li>What decisions does the model make by itself, and which are fixed in advance?</li><li>What tools can it use, and with what permissions?</li><li>What stops it from looping or overspending?</li><li>Which actions need human approval?</li><li>How can I see what it did, step by step?</li><li>What happens when it fails?</li><li>What data does it see and keep?</li></ul><p>If the answers are vague, treat the “agent” label with caution.</p><h3>A short checklist</h3><ol><li>Can a fixed workflow do the job? If yes, use that.</li><li>Is the task narrow, valuable and low risk?</li><li>Does the agent have only the tools and permissions it needs?</li><li>Are there limits on steps, time and spend?</li><li>Do risky actions need a person’s approval?</li><li>Can you see and review everything it does?</li><li>Have you tested it on awkward and hostile inputs?</li></ol><h3>The takeaway</h3><p>An agent is a system where a model chooses its own steps, uses tools and repeats until a goal is reached. That flexibility is powerful and also risky. Use agents for tasks where the path really can’t be planned, keep their reach small, and build in limits and oversight from day one.</p><h3>Where we go next</h3><p>Next, we’ll look at how agents actually use tools: tool use and function calling explained, with a small worked example.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1788bcc65a2d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/what-is-an-ai-agent-1788bcc65a2d">What Is an AI Agent?</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GraphRAG]]></title>
            <link>https://medium.com/aibizx/graphrag-d51f888bdd87?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d51f888bdd87</guid>
            <category><![CDATA[graphrag]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 07:21:38 GMT</pubDate>
            <atom:updated>2026-11-12T07:21:38.699Z</atom:updated>
            <content:encoded><![CDATA[<h4>When Simple RAG Isn’t Enough</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*otWEJHOZUYbCU9e7" /><figcaption>Photo by <a href="https://unsplash.com/@majedswan?utm_source=medium&amp;utm_medium=referral">majed swan</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>Some questions depend on how things connect, not just which passage looks similar. Here’s what GraphRAG adds, what it costs, and a sensible way to decide whether you need it.</em></blockquote><p>A standard RAG system is very good at questions like “What’s our refund policy for annual plans?” The answer lives in one passage, the search finds it and the model reads it out.</p><p>Now try a different kind of question: “Which of our suppliers could delay the launch of Product X?” The answer isn’t in a single paragraph. Product X uses Component Y, which comes from Supplier Z, whose factory is affected by a delay mentioned in a separate report. To answer, you have to follow a chain of connections across several documents.</p><p>Simple RAG struggles with that. This article explains why, introduces <strong>GraphRAG</strong>, and gives you an honest view of when it’s worth the extra effort.</p><h3>Where simple RAG runs out</h3><p>Standard RAG retrieves the passages most similar to the question. That works when the answer sits in one place. It starts to fail when:</p><ul><li><strong>The answer needs several hops.</strong> You must connect fact A to fact B to fact C, and no single passage mentions all three.</li><li><strong>The question is about relationships.</strong> “Who reports to whom?” “Which projects depend on this system?”</li><li><strong>The question is about the whole collection.</strong> “What are the main themes across all these customer interviews?” There’s no single passage to retrieve, because the answer is a summary of everything.</li><li><strong>The names change.</strong> The same person, company or product appears under different names in different documents, and a similarity search doesn’t connect them.</li></ul><p>The common thread is that similarity search finds passages that <em>look like</em> the question. These questions need passages connected by <em>relationships</em>.</p><h3>What a knowledge graph is</h3><p>A <strong>knowledge graph</strong> stores information as things and the links between them:</p><ul><li><strong>Entities</strong> are the things: people, companies, products, places, concepts.</li><li><strong>Relationships</strong> are how they connect: “works for,” “is part of,” “supplies,” “depends on.”</li></ul><p>The basic unit is a triple: subject, relationship, object. For example: (Product X, uses, Component Y), (Component Y, supplied by, Supplier Z), (Supplier Z, located in, Region Q).</p><p>Once your information is stored this way, following a chain is easy. Start at Product X, walk to Component Y, then to Supplier Z, then to Region Q. Each step is a lookup, and there’s no need to hope that a search engine finds one passage mentioning all of them.</p><p>Here’s a tiny illustration in Python, with made-up data:</p><pre>triples = [<br>    (&quot;Product X&quot;, &quot;uses&quot;, &quot;Component Y&quot;),<br>    (&quot;Component Y&quot;, &quot;supplied_by&quot;, &quot;Supplier Z&quot;),<br>    (&quot;Supplier Z&quot;, &quot;located_in&quot;, &quot;Region Q&quot;),<br>    (&quot;Product W&quot;, &quot;uses&quot;, &quot;Component V&quot;),<br>]<br><br>def neighbours(entity):<br>    return [(rel, obj) for subj, rel, obj in triples if subj == entity]<br><br>def follow(entity, hops=3):<br>    path, current = [entity], entity<br>    for _ in range(hops):<br>        steps = neighbours(current)<br>        if not steps:<br>            break<br>        rel, current = steps[0]      # simplest case: follow the first link<br>        path.append(f&quot;-{rel}-&gt;&quot;)<br>        path.append(current)<br>    return &quot; &quot;.join(path)<br><br>print(follow(&quot;Product X&quot;))</pre><p>That prints the chain from Product X through its component and supplier to the region. Real graph databases do this at scale, with proper query languages, but the principle is the same.</p><h3>What GraphRAG is</h3><p>GraphRAG combines the two ideas. It builds a knowledge graph from your documents, and uses the graph, as well as text search, to find what the model needs.</p><p>There’s more than one design, but a typical one looks like this:</p><p><strong>Building the graph (done ahead of time):</strong></p><ol><li>A language model reads each chunk of your documents and extracts the entities and relationships it mentions.</li><li>These are assembled into a graph, merging different mentions of the same thing where possible.</li><li>Some designs also group the graph into clusters, called communities, of closely connected entities, and write a summary of each.</li></ol><p><strong>Answering a question:</strong></p><ul><li><strong>For specific questions about particular things,</strong> find the relevant entities, follow their connections to gather the related facts and passages, and give those to the model.</li><li><strong>For broad questions about the whole collection,</strong> use the community summaries, so the model can answer “what are the main themes?” from summaries of the whole, not from a handful of passages.</li></ul><p>A well-known 2024 paper from Microsoft Research, “From Local to Global: A Graph RAG Approach to Query-Focused Summarization,” described this approach and showed that it improved answers to global questions about a whole text collection, compared with plain RAG. It’s worth knowing that the paper’s tests concentrated on that kind of whole-collection question. It doesn’t show that graph methods beat plain RAG on simple lookups.</p><p>Cloud providers have started to offer GraphRAG as a feature. For example, Amazon Bedrock Knowledge Bases has had a GraphRAG option that uses Amazon Neptune. Check the current documentation for what’s available, because AWS’s knowledge base offerings have been changing this year.</p><h3>When it helps, and when it doesn’t</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*F4CvDjAV3JXAG9G7cd_Cog.png" /></figure><h3>The costs</h3><p>GraphRAG isn’t free, and the costs come in several forms.</p><p><strong>Indexing cost.</strong> Building the graph typically means a language model reads every chunk to extract entities and relationships. For a large collection, that’s a lot of model calls, and it’s repeated when documents change.</p><p><strong>Quality risk.</strong> Extraction can be wrong. A missed or invented relationship becomes a wrong fact in the graph. Merging entities, deciding that “IBM” and “International Business Machines” are one thing, is a hard problem.</p><p><strong>Complexity.</strong> There’s a graph database or graph store to run, a new pipeline to maintain, and more places for things to break.</p><p><strong>Freshness.</strong> Updating a graph when documents change is harder than re-embedding a few chunks.</p><p><strong>Debugging.</strong> When an answer is wrong, there are more layers to check: the documents, the extraction, the graph, the traversal and the generation.</p><h3>Cheaper things to try first</h3><p>Before you build a graph, work through simpler fixes. Many multi-hop failures come from retrieval problems that are easier to solve.</p><ol><li><strong>Improve chunking and context,</strong> as in the earlier articles, so each chunk carries enough information.</li><li><strong>Use hybrid search and a reranker,</strong> to find the right passages more reliably.</li><li><strong>Break the question down.</strong> Have a model split “Which suppliers could delay Product X?” into smaller questions, answer each with ordinary RAG and combine the results. It’s a form of prompt chaining, and it handles a surprising share of multi-hop questions.</li><li><strong>Use structured data directly.</strong> If the relationships already live in a database, such as products, parts and suppliers, let the model write a query, and skip the extraction step entirely.</li><li><strong>Add metadata,</strong> such as the product, supplier and region that each chunk mentions, and filter on it.</li><li><strong>Try an agent</strong> that can search several times, using what it finds to decide what to look for next.</li></ol><p>Only if these fall short on your own test questions is it worth building a graph.</p><h3>How to decide: a small experiment</h3><ol><li><strong>Write 20 or more multi-hop and whole-collection questions</strong> that matter to you, with known answers.</li><li><strong>Run them through your best plain RAG setup,</strong> with hybrid search, reranking and question decomposition. Record how many it gets right.</li><li><strong>Look at the failures.</strong> Are they really about connections between facts, or about retrieval you could still improve?</li><li><strong>If connections are the problem, try a graph approach on a small slice</strong> of your documents, perhaps a few hundred, and test the same questions.</li><li><strong>Compare quality against cost and effort,</strong> including indexing and upkeep.</li></ol><p>If the graph version clearly wins on questions that matter, scale it up. If the improvement is small, you’ve saved yourself a lot of work.</p><h3>Pitfalls</h3><ol><li><strong>Reaching for a graph too early.</strong> It’s the heaviest option, not the first.</li><li><strong>Trusting the extraction.</strong> Check a sample of the entities and relationships by hand.</li><li><strong>Ignoring duplicates.</strong> Several names for one entity fragment your graph.</li><li><strong>Forgetting permissions.</strong> A graph can connect facts from documents that different people are allowed to see. Make sure access control carries through.</li><li><strong>Letting it go stale.</strong> Plan for how updates reach the graph.</li><li><strong>No evaluation.</strong> Test with the same retrieval and answer measures as before, and add multi-hop questions.</li></ol><h3>A checklist</h3><ol><li>Do your important questions really depend on connections between facts?</li><li>Have you tried better chunking, hybrid search, reranking and question decomposition first?</li><li>Could the relationships come from an existing database instead of extraction?</li><li>Have you tested a small slice before committing the whole collection?</li><li>Do you know the cost of building and updating the graph?</li><li>Have you checked the extracted entities and relationships by hand?</li><li>Do permissions carry through to the graph?</li><li>Do your test questions include multi-hop and whole-collection cases?</li></ol><h3>The takeaway</h3><p>Simple RAG finds passages that look like the question. GraphRAG adds a map of how things connect, which helps with multi-hop and whole-collection questions. It costs more to build and maintain, so try the cheaper fixes first, and let a test on your own questions tell you whether you need it.</p><h3>Where we go next</h3><p>That completes our series on RAG. Next we’ll move on to agents without the hype, starting with a plain-language guide to what an AI agent is.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d51f888bdd87" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/graphrag-d51f888bdd87">GraphRAG</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating a RAG System]]></title>
            <link>https://medium.com/aibizx/evaluating-a-rag-system-dac2df8a9464?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/dac2df8a9464</guid>
            <category><![CDATA[rage]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-evaluation]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Thu, 12 Nov 2026 02:10:42 GMT</pubDate>
            <atom:updated>2026-11-12T02:10:42.946Z</atom:updated>
            <content:encoded><![CDATA[<h4>Retrieval Quality vs Answer Quality</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fbBUuNFqsGNJAVy4" /><figcaption>Photo by <a href="https://unsplash.com/@procopiopi?utm_source=medium&amp;utm_medium=referral">Omar:. Lopez-Rincon</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>“It seems to work” won’t survive real users. Here’s a simple way to measure whether your RAG system finds the right passages and whether it uses them well, and the numbers to track.</em></blockquote><p>In the last article we saw that RAG systems fail in seven different ways. The practical difficulty is that from the outside they all look the same: a wrong answer. To fix a problem, you first need to know which part of the pipeline caused it.</p><p>That’s what evaluation is for. A good evaluation splits the system into its two halves and measures each separately: <strong>retrieval</strong>, which finds the passages, and <strong>generation</strong>, which writes the answer from them. This article shows you how to do that with a small test set and a handful of simple measures.</p><h3>Why measure the two halves separately</h3><p>Suppose 70 percent of your answers are correct. That number tells you almost nothing about what to fix. The cause might be that retrieval misses the right passage a quarter of the time, or that retrieval is perfect and the model misreads what it’s given. The fixes are completely different, so you need to know which.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*1eIJnsb1WQDI_j7gKFnF1Q.png" /></figure><h3>Step 1: Build a test set</h3><p>Everything starts here, and it’s worth doing carefully. Aim for 30 to 100 questions. For each one, record:</p><ul><li><strong>The question,</strong> in the words a real user would use</li><li><strong>The passage or passages that contain the answer.</strong> Use chunk IDs or document sections.</li><li><strong>A reference answer,</strong> short and correct, written by someone who knows the material</li><li><strong>Whether the question is answerable.</strong> Include some that your documents cannot answer.</li></ul><p>A useful mix:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*Ow3J0bwLA_gAcQZ2s4eXrg.png" /></figure><p>Put it in a simple file, for example:</p><pre>id,question,expected_chunks,reference_answer,answerable<br>1,&quot;Can I return a gift card?&quot;,&quot;policy-07&quot;,&quot;No. Gift cards can&#39;t be returned, but they never expire.&quot;,yes<br>2,&quot;Do you sell bicycles?&quot;,&quot;&quot;,&quot;The documents don&#39;t cover this.&quot;,no</pre><p>Real questions are best. Look through support tickets, search logs and the questions people ask colleagues. Questions you invented yourself tend to be too clean.</p><h3>Step 2: Measure retrieval</h3><p>For each question, record which chunks your system retrieved, in order. Then compute a few measures, comparing them with the expected chunks.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*pUQr6d6DaJc7Z-HKDuKXiA.png" /></figure><p>Hit rate is the most important at the start: if the right chunk isn’t in what you send to the model, nothing downstream can save the answer. Here’s a small function to compute all three:</p><pre>def retrieval_metrics(results, k=5):<br>    &quot;&quot;&quot;results: [{&#39;expected&#39;: [&#39;policy-07&#39;], &#39;retrieved&#39;: [&#39;faq-02&#39;, &#39;policy-07&#39;, ...]}]<br>    Only include answerable questions.&quot;&quot;&quot;<br>    hits = 0<br>    reciprocal = 0.0<br>    precision_total = 0.0<br>    for r in results:<br>        top = r[&quot;retrieved&quot;][:k]<br>        positions = [i for i, chunk in enumerate(top) if chunk in r[&quot;expected&quot;]]<br>        if positions:<br>            hits += 1<br>            reciprocal += 1 / (positions[0] + 1)<br>        precision_total += sum(1 for chunk in top if chunk in r[&quot;expected&quot;]) / k<br>    n = len(results)<br>    return {<br>        &quot;hit_rate&quot;: hits / n,<br>        &quot;mrr&quot;: reciprocal / n,<br>        &quot;precision_at_k&quot;: precision_total / n,<br>    }</pre><p>If your hit rate at the number of chunks you send is low, stop here and fix retrieval. Judging the answers will only confuse you.</p><h3>Step 3: Measure generation</h3><p>Now look at the final answers. Three qualities matter, and each has a different cause when it’s weak.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*_19esJn5elkv1FQ0QFKf1Q.png" /></figure><p>And two more that are easy to forget:</p><ul><li><strong>Correct refusals.</strong> For questions the documents can’t answer, did it say so? Count how often it did.</li><li><strong>Citation accuracy.</strong> If you show sources, do they really support the statements they’re attached to?</li></ul><p>Faithfulness and correctness are different. An answer can be faithful to a wrong or outdated passage. And it can be correct while ignoring the passages, which means the model guessed right this time.</p><h3>Step 4: Decide how to score</h3><p>There are three ways, and they’re best used together.</p><p><strong>Automatic checks.</strong> Cheap and consistent. For example: does the answer contain the key fact? Is the cited document the expected one? Is it under the length limit? Did it refuse an unanswerable question? These handle a lot of the work.</p><p><strong>Human review.</strong> The gold standard for judgement. Have someone who knows the material score a sample on a simple scale, such as 0 for wrong, 1 for partly right and 2 for right, and mark any unsupported claims.</p><p><strong>A model as judge.</strong> You can ask another model to score answers against a rubric. It’s fast and scales to large sets, but it isn’t a neutral referee. It can favour longer answers, miss subtle errors and be inconsistent. Check its scores against a person’s on a sample before you trust it, and use it as an aid, not the last word.</p><p>Here’s an example of a faithfulness check you could give a judge model:</p><pre>You are checking an answer against source passages.</pre><pre>&lt;passages&gt;<br>...retrieved passages...<br>&lt;/passages&gt;</pre><pre>&lt;answer&gt;<br>...the system&#39;s answer...<br>&lt;/answer&gt;</pre><pre>List each factual claim in the answer. For each claim, say whether it is<br>SUPPORTED or NOT SUPPORTED by the passages. Then give a final verdict:<br>FAITHFUL if every claim is supported, otherwise UNFAITHFUL.<br>Return JSON: {&quot;claims&quot;: [{&quot;claim&quot;: &quot;...&quot;, &quot;supported&quot;: true}], &quot;verdict&quot;: &quot;...&quot;}</pre><p>Parse and validate the result as you would any structured output, and spot-check it by hand.</p><h3>Step 5: Put it in a report</h3><p>Summarise each run in one small table, so you can compare changes. Alongside the numbers, keep a short list of the specific questions that failed in each run. The failures are what you learn from.</p><h3>Step 6: Make it a routine</h3><ul><li><strong>Rerun the set after every change</strong> to chunking, embeddings, search, prompt or model.</li><li><strong>Look at the questions that changed</strong> from pass to fail, not just the totals.</li><li><strong>Add every real failure</strong> you find in production to the set.</li><li><strong>Keep a few questions aside</strong> that you never use for tuning, to check you haven’t fitted the system to your own test.</li><li><strong>Retest when documents change.</strong> New or revised content can break old answers.</li></ul><h3>Watching it in production</h3><p>A test set tells you how the system behaves on questions you thought of. Real users ask others. Once it’s live, monitor:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*onwWKK2xyBwrQujrjTsFdQ.png" /></figure><p>Sample real conversations regularly, with proper regard for privacy, and add the interesting ones to your test set.</p><h3>Common mistakes</h3><ol><li><strong>Measuring only the final answer.</strong> You can’t tell where to fix it.</li><li><strong>A test set that’s too small or too easy.</strong></li><li><strong>Writing the expected passages after seeing what the system retrieved.</strong> Decide first.</li><li><strong>Skipping unanswerable questions.</strong></li><li><strong>Trusting a judge model without checking it.</strong></li><li><strong>Looking only at averages.</strong> A 90 percent score can hide a failure on exactly the questions that matter.</li><li><strong>Never updating the set.</strong></li><li><strong>Judging faithfulness without the passages the model actually saw.</strong></li></ol><h3>A short checklist</h3><ol><li>Do you have 30 or more real questions with expected passages and reference answers?</li><li>Does the set include unanswerable questions?</li><li>Do you measure hit rate, MRR and precision for retrieval?</li><li>Do you measure faithfulness, correctness and correct refusals for generation?</li><li>Have you checked any judge model against human scores?</li><li>Do you rerun the set after every change, and look at what flipped?</li><li>Do you monitor real usage and feed failures back into the set?</li></ol><h3>The takeaway</h3><p>Measure retrieval and generation separately. Check first that the right passage reaches the model, then whether the model answers faithfully from it and declines when it should. A small, honest test set, rerun after every change, will teach you more than any amount of impression.</p><h3>Where we go next</h3><p>Next, we’ll look at what to do when simple RAG isn’t enough: GraphRAG, multi-hop questions and an honest view of the trade-offs.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dac2df8a9464" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/evaluating-a-rag-system-dac2df8a9464">Evaluating a RAG System</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why RAG Fails]]></title>
            <link>https://medium.com/aibizx/why-rag-fails-d2149cdabd21?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2149cdabd21</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Wed, 11 Nov 2026 01:12:47 GMT</pubDate>
            <atom:updated>2026-11-11T01:12:47.290Z</atom:updated>
            <content:encoded><![CDATA[<h4>7 Common Problems and How to Fix Each</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FTCs2J2hPEwjknAy" /><figcaption>Photo by <a href="https://unsplash.com/@coopery?utm_source=medium&amp;utm_medium=referral">Mohamed Nohassi</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>When a RAG system gives a bad answer, the cause is almost always one of seven things. Here’s how to tell which one, and what to do about it.</em></blockquote><p>You’ve built a RAG system. It works nicely in the demo. Then real people start asking real questions, and some of the answers are wrong, vague or oddly confident. The temptation is to blame the language model, or to start rewriting the prompt at random.</p><p>There’s a better way. A RAG system is a pipeline, and when it fails it fails in a particular place. If you can work out where, the fix is usually clear. This article walks through the seven most common failure points, how to recognise each, and what to do about it.</p><h3>First, find out where it broke</h3><p>Before you change anything, take one bad answer and ask three questions, in order:</p><ol><li><strong>Is the answer actually in your documents?</strong> If not, the system can’t be blamed for not finding it.</li><li><strong>Was the right passage retrieved?</strong> Look at the chunks that were sent to the model.</li><li><strong>Did the model use it properly?</strong> If the right chunk was there and the answer was still wrong, the problem is in generation.</li></ol><p>To answer them, you need to see what was retrieved. Log it for every question: the question, the chunks returned with their scores, and the final answer. Without that, you’re debugging blind. Be careful about personal data in those logs, and set a short retention period.</p><p>With that in place, here are the seven problems.</p><h3>Problem 1: The right passage was never retrieved</h3><p><strong>Symptom:</strong> The answer is in your documents, but it isn’t among the chunks the model saw.</p><p><strong>Common causes:</strong></p><ul><li>The question is worded very differently from the document</li><li>The search misses exact terms, such as codes and names, because it’s vector-only</li><li>Chunks are poorly formed, so their meaning is blurred</li><li>The embedding model doesn’t suit your content or language</li></ul><p><strong>Fixes:</strong></p><ul><li>Use hybrid search, so keywords and meaning both count</li><li>Rewrite or expand the query before searching</li><li>Test other chunk sizes and structure-aware splitting</li><li>Compare embedding models on your own questions</li></ul><h3>Problem 2: It was retrieved, but ranked too low</h3><p><strong>Symptom:</strong> The right chunk shows up at position 9, and you only send the top 5.</p><p><strong>Fixes:</strong></p><ul><li>Add a reranker, as covered in the last article</li><li>Retrieve more candidates before trimming</li><li>Check that your filters aren’t excluding the right document</li></ul><h3>Problem 3: The chunk has no context</h3><p><strong>Symptom:</strong> The retrieved passage says “The fee is $15,” and nobody, including the model, knows what the fee is for. Or it says “unless the item is damaged” without the rule it’s an exception to.</p><p><strong>Fixes:</strong></p><ul><li>Add the document title and section heading to every chunk</li><li>Use larger chunks, or parent-and-child chunking, so the model gets surrounding text</li><li>Keep tables and lists together</li><li>Add overlap between chunks</li></ul><h3>Problem 4: The documents are wrong, old or contradictory</h3><p><strong>Symptom:</strong> The answer matches the retrieved passage, but the passage is out of date. Or two documents disagree and the model picks one.</p><p><strong>Fixes:</strong></p><ul><li>Give each document an owner, a date and a version, and store them as metadata</li><li>Remove or archive old versions, so they can’t be retrieved</li><li>Prefer newer documents, using a date filter or boost</li><li>Show the source and date in the answer so people can judge it</li><li>Treat this as a content problem. No amount of clever retrieval fixes a policy that was never updated.</li></ul><p>This one catches many teams out, because the citation makes a stale answer look trustworthy.</p><h3>Problem 5: The model ignores or misuses what it’s given</h3><p><strong>Symptom:</strong> The right passage was retrieved, and the answer still contradicts it, skips it or blends it with the model’s own guesses.</p><p><strong>Common causes:</strong></p><ul><li>Too many passages, so the key one is lost among the others</li><li>Weak instructions that don’t tell the model to stick to the text</li><li>Long inputs, where information in the middle can be used less reliably</li><li>Conflicting passages</li></ul><p><strong>Fixes:</strong></p><ul><li>Send fewer, better passages</li><li>Tell the model clearly: “Answer only from the passages below. If the answer isn’t there, say so.”</li><li>Ask it to quote or cite the supporting passage for each claim, which makes unsupported claims easy to spot</li><li>Put the most relevant passages first</li><li>Use a lower temperature for factual answers</li><li>Try a stronger model, if the others don’t help</li></ul><h3>Problem 6: The question needs more than lookup</h3><p><strong>Symptom:</strong> Questions like “Summarise all complaints from last year,” “Which of our products has the most returns?” or “Compare these three contracts” get partial or wrong answers.</p><p><strong>Why:</strong> RAG finds a few similar passages. It isn’t built to read everything, count things or combine information across many documents.</p><p><strong>Fixes:</strong></p><ul><li>For counts and totals, use a database or spreadsheet and let the model write the query, instead of reading text</li><li>For multi-step questions, break them into smaller questions, answer each and combine the results</li><li>For questions that follow relationships between things, look at graph-based approaches, which we’ll cover later in the series</li><li>For summaries of whole collections, summarise in stages</li><li>Set expectations. Tell users what kinds of questions it handles best.</li></ul><h3>Problem 7: There’s no answer, and it makes one up</h3><p><strong>Symptom:</strong> Someone asks about something your documents don’t cover, and the system answers anyway, usually in a confident tone, using whatever loosely related passage it found.</p><p><strong>Fixes:</strong></p><ul><li>Instruct the model to say “I can’t find that in the documents,” and test that it does</li><li>Use a relevance cut-off. If even the best passage scores poorly, don’t answer from it.</li><li>Include questions your documents can’t answer in your test set, and count how often the system correctly declines</li><li>Offer a next step when it can’t answer, such as contacting a person</li></ul><p>This is the failure that damages trust most. A system that sometimes says “I don’t know” is far more useful than one that never does.</p><h3>A quick diagnostic table</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*AC0HBLs5PpdfSi5TNcVSlQ.png" /></figure><h3>Two more risks to keep in view</h3><p>These aren’t about quality, but they can fail just as badly.</p><p><strong>Permission leaks.</strong> If different people may see different documents, filter retrieval by what the asker is allowed to see, and test it with accounts that shouldn’t have access. Don’t rely on the model to withhold information.</p><p><strong>Prompt injection.</strong> Text inside your documents can contain instructions that try to steer the model. Treat retrieved text as untrusted, as covered in the article on prompt injection.</p><h3>Build a habit of testing</h3><p>The single most useful practice is a standing set of questions that you rerun after every change:</p><ul><li>30 to 50 real questions with known answers and known source passages</li><li>A few questions your documents can’t answer</li><li>A few that test permissions</li><li>Awkward questions: misspellings, vague wording, two questions in one</li></ul><p>For each, record whether the right chunk was retrieved, and whether the final answer was right. Those two columns tell you straight away whether a problem sits in retrieval or generation, and the next article covers how to measure each.</p><h3>A short checklist</h3><ol><li>Do you log the question, the retrieved chunks and the answer?</li><li>Do you check retrieval separately from generation?</li><li>Do chunks carry their title and heading?</li><li>Do documents have dates, owners and versions, with old ones retired?</li><li>Does the prompt tell the model to answer only from the passages, and to say when it can’t?</li><li>Do you test questions the documents can’t answer?</li><li>Do you filter by permissions, and test it?</li><li>Do you rerun a fixed test set after every change?</li></ol><h3>The takeaway</h3><p>When a RAG system fails, don’t guess. Look at what was retrieved. If the right passage wasn’t found, fix search and chunking. If it was found but buried, add reranking. If the content is wrong, fix the content. If the model misused it, tighten the instructions. And always make sure the system can say “I don’t know.”</p><h3>Where we go next</h3><p>Next, we’ll look at how to evaluate a RAG system properly, separating retrieval quality from answer quality, with simple metrics you can compute yourself.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2149cdabd21" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/why-rag-fails-d2149cdabd21">Why RAG Fails</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rerankers]]></title>
            <link>https://medium.com/aibizx/rerankers-357da7e42a08?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/357da7e42a08</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[reranker]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 20:16:27 GMT</pubDate>
            <atom:updated>2026-11-10T20:16:27.990Z</atom:updated>
            <content:encoded><![CDATA[<h4>Getting the Best Passages to the Top</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uHQshtaqrZvTLE1D" /><figcaption>Photo by <a href="https://unsplash.com/@emilwidlund?utm_source=medium&amp;utm_medium=referral">Emil Widlund</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><em>A first search finds candidates quickly. A reranker reads them more carefully and puts the best ones first. Here’s what it does, when it’s worth the extra step, and how to test it.</em></p><p>Imagine you ask a librarian for books on starting a small business. They pull twenty from the shelves in a minute: some excellent, some only loosely related, a couple that mention “business” in the title and nothing else. Then they sit down and skim each one properly, and hand you the best three.</p><p>That’s a reranker. It’s a second pass that takes the candidates found by your search and re-orders them by how well each one actually answers the question. In a RAG system, it’s often one of the most effective upgrades you can make.</p><h3>Why the first search isn’t enough</h3><p>The search step we’ve covered so far, whether by vector, by keyword or both, has to be fast. It looks through thousands or millions of chunks in a fraction of a second. To manage that, it uses shortcuts.</p><p>With vector search, the question and every chunk are turned into numbers <strong>separately</strong>. The chunk’s embedding was made long before your question existed, and it has to serve every possible question. The search then just measures how close two lists of numbers are. That captures general meaning well, but it misses fine detail. Two passages can look about equally relevant by that measure, even when only one really answers the question.</p><p>The result is a list where the right passage is usually near the top, but not always first, and a few poor matches sneak in.</p><h3>How a reranker works</h3><p>A reranker takes the question and one candidate passage <strong>together</strong>, and reads both at once to judge how well the passage answers the question. This kind of model is often called a cross-encoder. Because it sees the question and the text side by side, it can notice things the first search couldn’t, such as whether the passage addresses the specific condition you asked about.</p><p>The price is speed. Reading a question and a passage together is much slower than comparing two stored lists of numbers, and it has to be done for every candidate. So you can’t run it on your whole collection. You use it on a short list.</p><h3>The two-stage pipeline</h3><p>The standard design:</p><ol><li><strong>Retrieve a generous set of candidates,</strong> say 20 to 50, with your fast search, such as hybrid search.</li><li><strong>Rerank them,</strong> scoring each one against the question.</li><li><strong>Keep the best few,</strong> perhaps 3 to 5, and send those to the language model.</li></ol><p>The first stage is built for recall: don’t miss the right passage. The second is built for precision: put the right one on top.</p><h3>Why it helps a RAG system</h3><ul><li><strong>Better answers from the same documents.</strong> The model sees the most relevant passages and less noise.</li><li><strong>Fewer tokens.</strong> By keeping only the top few, you send less text to the model, which can save money and speed up the generation step.</li><li><strong>Less distraction.</strong> Irrelevant passages can lead a model astray. Cutting them out helps.</li><li><strong>Robustness.</strong> It can rescue questions where the first search put the best passage at position 12.</li></ul><h3>What it costs</h3><p><strong>Latency.</strong> Each reranking call adds time. For a chat feature where people wait, that matters. It’s usually tens to hundreds of milliseconds for a short list, but check with your model.</p><p><strong>Money.</strong> Reranking services typically charge per request or per number of documents scored. Some of that is offset by sending fewer tokens to the generation model.</p><p><strong>Complexity.</strong> One more component to run, monitor and test.</p><p>So it isn’t free, and it isn’t always needed.</p><h3>When it’s worth adding</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*AlrI8vyzodzvDEReRQY-Hg.png" /></figure><p>The honest way to decide is to measure.</p><h3>Your options</h3><p><strong>A hosted reranking service.</strong> Some platforms offer a rerank API that takes a question and a list of passages and returns them scored. Amazon Bedrock, for example, has a rerank capability with models from different providers. It’s the easiest route. Check the current documentation for models, limits and pricing.</p><p><strong>An open-source cross-encoder model,</strong> which you run yourself. More control and no per-call fees, but you manage the hosting.</p><p><strong>A language model as the reranker.</strong> You give a general model the question and the passages and ask it to rank them. It’s flexible and needs no extra component, but it’s slower and more expensive, and its ratings can be less consistent.</p><p><strong>Built-in options in managed services.</strong> Some managed knowledge bases and search services offer reranking as a setting. Check what yours supports.</p><h3>A small sketch</h3><p>Here’s the shape of a reranking step. The score function stands in for whichever method you use: it takes the question and a passage and returns a relevance score.</p><pre>def rerank(question, candidates, score, keep=5):<br>    &quot;&quot;&quot;candidates: a list of passage texts from your first-stage search.&quot;&quot;&quot;<br>    scored = [(score(question, passage), passage) for passage in candidates]<br>    scored.sort(key=lambda pair: pair[0], reverse=True)<br>    return [passage for _, passage in scored[:keep]]<br></pre><pre># The pipeline:<br># candidates = hybrid_search(question, k=30)          # stage 1: fast and wide<br># best = rerank(question, candidates, score, keep=5)   # stage 2: careful and narrow<br># answer = generate_answer(question, best)             # then ask the language model</pre><p>The point of the sketch is how little there is to it. The work is in choosing the scorer and testing it.</p><h3>How to test whether it helps</h3><p>Use the same test set approach as in the earlier articles.</p><ol><li><strong>Take your 30 to 50 questions,</strong> each with the chunk that holds the answer.</li><li><strong>Run the first-stage search alone,</strong> and record where the right chunk ranks.</li><li><strong>Add the reranker,</strong> and record where it ranks now.</li><li><strong>Compare</strong> how often the right chunk is in the top 3 and top 5, and its average position.</li><li><strong>Measure the extra time and cost,</strong> and decide whether the gain justifies them.</li><li><strong>Check the final answers,</strong> not just the ranking, to see whether the improvement shows up where it counts.</li></ol><h3>Tuning it</h3><p><strong>How many candidates to retrieve.</strong> Too few, and the right passage may never reach the reranker. Too many, and it’s slower and more expensive. Try 20, 30 and 50 and look at the results.</p><p><strong>How many to keep.</strong> Three to five is common. More gives the model more to work with, at higher token cost.</p><p><strong>Use a score cut-off.</strong> If even the top result scores poorly, the right answer may not be in your documents. Consider saying “I can’t find that” instead of answering from weak passages.</p><h3>Pitfalls</h3><ol><li><strong>A reranker can’t find what wasn’t retrieved.</strong> If the right passage isn’t in your candidates, no reranker will bring it in. Fix retrieval first.</li><li><strong>Length limits.</strong> Rerankers can only read so much per passage. Long chunks may be cut off, which hurts accuracy. Check the limit against your chunk size.</li><li><strong>Language.</strong> Check that your reranker handles the languages in your documents and questions.</li><li><strong>Over-trusting the scores.</strong> Scores are useful for ordering, and less reliable as absolute measures. Don’t treat 0.8 as “80 percent correct.”</li><li><strong>Adding it without testing.</strong> It sometimes helps a lot, sometimes a little. Measure.</li><li><strong>Ignoring latency.</strong> A big quality gain isn’t much use if the feature becomes too slow.</li></ol><h3>A short checklist</h3><ol><li>Does your first-stage search retrieve the right passage within the top 20 or 30 most of the time?</li><li>Have you tested at least one reranker against your own questions?</li><li>Did you compare quality gains with the added delay and cost?</li><li>Do your chunks fit within the reranker’s length limit?</li><li>Have you tuned how many to retrieve and how many to keep?</li><li>Do you handle the case where nothing scores well?</li></ol><h3>The takeaway</h3><p>A reranker is a careful second look at a fast first search. It reads the question and each candidate together, and moves the best passages to the top, so the model answers from better material. It adds time and cost, so test it on your own questions to see whether it earns its place.</p><h3>Where we go next</h3><p>Next, we’ll look at why RAG systems fail in practice: seven common problems and how to fix each.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=357da7e42a08" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/rerankers-357da7e42a08">Rerankers</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vector, Keyword or Hybrid Search]]></title>
            <link>https://medium.com/aibizx/vector-keyword-or-hybrid-search-d75dada581ac?source=rss-7ba39b76d27e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d75dada581ac</guid>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Saman Siadati]]></dc:creator>
            <pubDate>Tue, 10 Nov 2026 13:08:43 GMT</pubDate>
            <atom:updated>2026-11-10T13:08:43.351Z</atom:updated>
            <content:encoded><![CDATA[<h4>When Each Wins</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ge8Q2iiNMQl5RHEx" /><figcaption>Photo by <a href="https://unsplash.com/@hassaanhre?utm_source=medium&amp;utm_medium=referral">Hassaan Here</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><em>Searching by meaning finds answers that use different words. Searching by words finds exact terms that meaning-based search blurs. Most good systems use both.</em></blockquote><p>In the last few articles we’ve talked about embeddings and chunks as if meaning-based search were the only way to find things. It’s powerful, but it isn’t the only way, and it isn’t always the best one.</p><p>This article compares the three styles of search you’ll meet in RAG systems: keyword search, vector search and hybrid search. You’ll see where each one shines, where each one falls over, and how to combine them.</p><h3>Keyword search: matching the words</h3><p>Keyword search is the older technique, and it’s what most search boxes did for decades. It looks for documents that contain the words in your query, and ranks them by how strongly they match. A common scoring method, called BM25, rewards documents where the query’s words appear often, appear in a short passage, and are rare across the whole collection. A rare word that appears in a document is a strong signal. A common word such as “the” tells you nothing.</p><p><strong>Where it shines:</strong></p><ul><li><strong>Exact terms.</strong> Product codes, error messages, names, part numbers, legal clause numbers and acronyms.</li><li><strong>Rare words.</strong> If someone searches for a specific, unusual term, keyword search finds the documents that really contain it.</li><li><strong>Predictability.</strong> You can see why a document matched: it contained the words.</li><li><strong>Speed and cost.</strong> It’s mature, fast and cheap.</li></ul><p><strong>Where it struggles:</strong></p><ul><li><strong>Different words, same meaning.</strong> A search for “get my money back” won’t find a page titled “Refund policy” if none of those words appear in it.</li><li><strong>Typos and variations,</strong> unless the system handles them.</li><li><strong>Vague or conversational questions,</strong> which are full of words that don’t appear in the answer.</li></ul><h3>Vector search: matching the meaning</h3><p>Vector search, which we’ve covered in the embeddings article, turns the query and each chunk into lists of numbers that capture meaning, and finds the chunks closest to the query. It can connect “get my money back” with “refund.”</p><p><strong>Where it shines:</strong></p><ul><li><strong>Paraphrases and synonyms.</strong> The question and the answer can use completely different words.</li><li><strong>Natural questions.</strong> People can ask in everyday language.</li><li><strong>Related ideas.</strong> It can surface passages that are about the same thing without sharing any terms.</li><li><strong>Some cross-language matching,</strong> with a multilingual model.</li></ul><p><strong>Where it struggles:</strong></p><ul><li><strong>Exact identifiers.</strong> To an embedding model, “ERR-4471” and “ERR-4417” can look nearly the same. A search for a specific code may return the wrong one.</li><li><strong>Rare or new terms</strong> the model hasn’t seen much, such as your own product names.</li><li><strong>Precision.</strong> It returns the closest things, even when nothing is truly relevant, so results can look plausible without being right.</li><li><strong>Opacity.</strong> It’s harder to explain why a result came back.</li></ul><h3>A side-by-side example</h3><p>Imagine a small support collection with these three chunks:</p><ol><li>“Annual plans can be refunded in full within 30 days of purchase.”</li><li>“Error ERR-4471: the payment gateway timed out. Retry after five minutes.”</li><li>“Error ERR-4417: the card was declined. Ask the customer to use another card.”</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*AhXoRuKXNmoCB3f1cv9CZQ.png" /></figure><p>Each method is right where the other is shaky. That’s the case for combining them.</p><h3>Hybrid search: using both</h3><p>Hybrid search runs both kinds of search and merges the results. Documents that rank well in both rise to the top, and each method rescues the other’s blind spots. Most serious RAG systems end up here.</p><p>The tricky part is merging, because the two methods score on different scales. A keyword score might be 14.2 and a vector similarity might be 0.83, and you can’t just add them. There are two common ways to handle it.</p><h3>Option 1: Combine the rankings</h3><p>Ignore the scores, and use each result’s position in each list. A popular method is <strong>reciprocal rank fusion</strong>. Each document gets points based on its rank in each list, with higher ranks earning more, and the points are added together.</p><pre>def rrf(rankings, k=60):<br>    &quot;&quot;&quot;rankings: a list of ranked lists of document IDs, best first.&quot;&quot;&quot;<br>    scores = {}<br>    for ranking in rankings:<br>        for rank, doc_id in enumerate(ranking, start=1):<br>            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)<br>    return sorted(scores, key=scores.get, reverse=True)<br></pre><pre>keyword_results = [&quot;c2&quot;, &quot;c3&quot;, &quot;c1&quot;]<br>vector_results = [&quot;c3&quot;, &quot;c2&quot;, &quot;c1&quot;]<br>print(rrf([keyword_results, vector_results]))</pre><p>The constant k, often set to 60, softens the advantage of the very top ranks. The great appeal of this approach is its simplicity: no need to compare scores on different scales.</p><h3>Option 2: Normalise and blend the scores</h3><p>Rescale each method’s scores to a common range, such as 0 to 1, then take a weighted sum, for example 70 percent vector and 30 percent keyword. It gives you a dial to tune the balance, but it needs care, since the scores’ distributions differ between queries. Many search engines, including OpenSearch, have built-in support for this kind of normalised blending. Check the current documentation for how your platform does it.</p><p>Either way, the principle is the same: merge two views of relevance, and let documents that both views like win.</p><h3>When to choose which</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*JQrvAnvI_bzmj6h-mJiWWg.png" /></figure><p>Some managed services make this choice for you. For example, AWS’s managed knowledge base uses hybrid search all the time, with no semantic-only mode. Others let you pick.</p><h3>Add filters</h3><p>Whichever you use, filters on metadata make results far better. Restrict search to a department, a product, a date range or documents the user is allowed to see, before or while searching. Filtering after retrieval can leave you with too few results. Many vector databases support filtering inside the search itself.</p><h3>Further steps up</h3><p>Once you have good search, two techniques can polish it further:</p><ul><li><strong>Reranking.</strong> Retrieve a larger set, say the top 30, then use a more careful model to re-order them and keep the best five. It’s slower and costs more, but often improves quality noticeably. We’ll look at it in the next article.</li><li><strong>Query rewriting.</strong> Use a model to turn a messy question into a clearer search query, or into several variations, before searching.</li></ul><h3>How to test your choice</h3><p>Don’t guess. Use the same method as for comparing embedding models.</p><ol><li><strong>Build a test set</strong> of 30 to 50 real questions, each with the chunk that answers it. Make sure it includes both natural-language questions and ones with exact terms.</li><li><strong>Run each method:</strong> keyword only, vector only and hybrid.</li><li><strong>Measure</strong> how often the right chunk appears in the top five, and how high it ranks.</li><li><strong>Look at the failures</strong> for each method. The pattern tells you what’s going wrong.</li><li><strong>Retest</strong> whenever you change documents, chunking or models.</li></ol><p>If hybrid beats both alone on your test set, that’s your answer. It often does, but it isn’t guaranteed, and the improvement may be small if your questions are all one type.</p><h3>Common mistakes</h3><ol><li><strong>Using vector search alone</strong> for content full of identifiers and rare terms.</li><li><strong>Adding scores from different methods directly.</strong></li><li><strong>Skipping filters</strong> and then wondering why results include irrelevant documents.</li><li><strong>Never testing exact-match queries.</strong> They’re where pure vector search fails quietly.</li><li><strong>Assuming hybrid is always better,</strong> without measuring.</li><li><strong>Ignoring how the keyword side is set up.</strong> Things like how words are split and normalised matter.</li></ol><h3>A short checklist</h3><ol><li>Does your content have codes, names or acronyms people will search for exactly?</li><li>Do people phrase questions differently from the documents?</li><li>Have you tested keyword, vector and hybrid on real questions?</li><li>If you use hybrid, how do you merge the two results?</li><li>Do you filter by metadata and permissions?</li><li>Have you looked at what each method gets wrong?</li></ol><h3>The takeaway</h3><p>Keyword search is best at exact terms, and vector search is best at meaning. Each has blind spots the other covers, so for most real collections a hybrid of the two, with filters, gives the most reliable results. Test on your own questions to be sure.</p><h3>Where we go next</h3><p>Next, we’ll look at rerankers: how a second, more careful pass can bring the best passages to the top.</p><p><em>This series is part of the AIBizX publication, where we share research and practical guides on AI and data.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d75dada581ac" width="1" height="1" alt=""><hr><p><a href="https://medium.com/aibizx/vector-keyword-or-hybrid-search-d75dada581ac">Vector, Keyword or Hybrid Search</a> was originally published in <a href="https://medium.com/aibizx">AIbizX</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>