<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Priya Singh on Medium]]></title>
        <description><![CDATA[Stories by Priya Singh on Medium]]></description>
        <link>https://medium.com/@PriyaSingh325?source=rss-bb58ec7be9f0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*227oa-rR9iGuRMZl5owypQ.png</url>
            <title>Stories by Priya Singh on Medium</title>
            <link>https://medium.com/@PriyaSingh325?source=rss-bb58ec7be9f0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 00:42:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@PriyaSingh325/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A RAG Pipeline Is Reproducible Only If You Can Explain What Changed]]></title>
            <link>https://medium.com/@PriyaSingh325/a-rag-pipeline-is-reproducible-only-if-you-can-explain-what-changed-f612ae441ac1?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f612ae441ac1</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[milvus]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Fri, 18 Sep 2026 11:25:59 GMT</pubDate>
            <atom:updated>2026-09-18T11:25:59.240Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xm5WUl7V4Kc1mktp" /></figure><p>Suppose a retrieval experiment improves after a pipeline edit. The YAML changed, but so did the document collection, the embedding model alias, and a reranker’s default candidate limit. Which change produced the improvement? Without a record of those inputs, the successful run is difficult to reproduce and almost impossible to diagnose later.</p><p>Declarative workflows make the sequence easier to inspect. They do not automatically capture the state of the services behind each step. I would treat a pipeline file as one part of an experiment record, then save the remaining information needed to explain the result.</p><p>The <a href="https://milvus.io/blog/how-to-build-a-rag-pipeline-with-ultrarag-v2-and-milvus.md?utm_campaign=mediumkoc">UltraRAG v2 and Milvus tutorial</a> demonstrates a modular approach: MCP tool interfaces expose components, YAML describes workflow control, and Milvus provides retrieval. The article includes integration code and configuration, so it should not be read as a promise that every custom backend works without implementation effort.</p><p>Here is an evaluation workflow I would put around that kind of setup. The code below is an independent experiment-manifest example, not an UltraRAG configuration file or a claim that the full tutorial was deployed.</p><h3>Freeze the question before changing the pipeline</h3><p>Begin with a specific hypothesis. “Does reranking improve retrieval for product identifiers?” is easier to test than “Can we make RAG better?” Select a representative question set and identify the evidence expected for each question before tuning against it.</p><p>Keep a separate evaluation set for the final comparison. Otherwise, repeated changes can adapt to the examples used during development without improving performance on new questions. Preserve questions with no supporting evidence and questions that contain misleading or ambiguous wording.</p><p>Record the acceptance criteria as well as the dataset. A pipeline might improve answer completeness but add unacceptable delay. If latency matters, define the measurement boundary and workload before running the comparison. A smaller candidate pool should not win simply because the test stopped checking difficult questions.</p><h3>Name the contracts between components</h3><p>A retrieval tool needs more than a query string. Its contract should state the allowed scope, expected representation, result identifiers, and failure behavior. A generation step should know which fields contain evidence and which contain diagnostics.</p><p>For each boundary, define the difference between an empty successful result and a dependency error. Returning an empty list for both makes the next step unable to choose between an unsupported answer and a retry. Include a stable error category that the workflow can route deliberately.</p><p>Version those interfaces alongside the pipeline. If a component changes a field from a score where larger is better to a distance where smaller is better, the payload may still parse while the ranking becomes wrong. A schema check catches structural mistakes; a semantic fixture catches this kind of behavioral mismatch.</p><h3>Save a manifest beside every result</h3><p>This complete Python example creates a deterministic fingerprint of a synthetic experiment manifest. Run it locally with Python’s standard library. The invented revision labels illustrate the record format; they are not actual model releases or tested benchmark settings.</p><p>import hashlib import json def fingerprint(manifest): blob = json.dumps(manifest, sort_keys=True, separators=(&quot;,&quot;, &quot;:&quot;)) return hashlib.sha256(blob.encode(&quot;utf-8&quot;)).hexdigest() baseline = { &quot;pipeline_revision&quot;: &quot;workflow-a&quot;, &quot;corpus_digest&quot;: &quot;synthetic-corpus-v1&quot;, &quot;question_set_digest&quot;: &quot;synthetic-questions-v1&quot;, &quot;embedding_revision&quot;: &quot;encoder-a&quot;, &quot;generator_revision&quot;: &quot;generator-a&quot;, &quot;retrieval_limit&quot;: 12, &quot;reranker&quot;: None, } candidate = dict(baseline, reranker=&quot;reranker-a&quot;) changed = {key for key in baseline if baseline[key] != candidate[key]} assert changed == {&quot;reranker&quot;} assert fingerprint(baseline) != fingerprint(candidate) assert fingerprint(baseline) == fingerprint(dict(reversed(list(baseline.items())))) print(json.dumps({&quot;changed&quot;: sorted(changed), &quot;run&quot;: fingerprint(candidate)}, indent=2))</p><p>The fingerprint detects differences in recorded inputs. It cannot account for an unrecorded dependency or make a stochastic model deterministic. Capture package locks, service versions, prompts, and supported sampling settings in the real manifest, and repeat runs where model variation matters.</p><p>Keep credentials out of these records. A manifest should name a credential profile or service environment without copying secrets. Store enough information for an authorized operator to recreate the setup, while preserving the controls around the underlying data.</p><h3>Inspect intermediate outputs on the same questions</h3><p>Save retrieved chunk IDs before reranking, selected IDs after reranking, and source references supplied to generation. This makes it possible to distinguish a retrieval miss from a selection error. An answer score alone cannot locate either failure.</p><p>For a <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a> retrieval stage, include the collection or dataset revision and search configuration in the run record. Rebuilding an index or changing the corpus should create an identifiable experiment change rather than silently altering a supposedly fixed baseline.</p><p>Use small fixtures to test orchestration. A query with one known supporting passage can verify field mapping. A query with no matches can verify the insufficient-evidence branch. A forced timeout can verify the error branch. These checks are especially useful before running a large evaluation that could conceal wiring mistakes in its averages.</p><h3>Put budgets around loops and branches</h3><p>An adaptive workflow can retrieve again when evidence is insufficient. It needs an explicit stopping rule. Set limits on iterations, total retrieved material, elapsed time, and external calls according to the application’s budget. A branch that never resolves uncertainty should end with an understandable limitation.</p><p>Also decide what information carries into the next iteration. Repeating the same query against the same corpus rarely provides new evidence. Preserve the earlier result IDs so the workflow can recognize duplication and distinguish genuinely new support from repeated retrieval.</p><p>Treat retrieved text as evidence, not authority over tool execution. Workflow control belongs to the application and its trusted instructions. A document that contains a command or requests an external action should not change the permissions of the pipeline that retrieved it.</p><h3>Compare the operational cost of the whole request</h3><p>Measure time spent encoding the query, retrieving candidates, reranking, and generating the answer. When a pipeline introduces parallel calls, record both total work and user-visible latency. Lower elapsed time does not necessarily mean lower compute or API cost.</p><p>For deployment, keep the indexing workflow separate from interactive requests. Track when a corpus revision becomes ready, avoid exposing a half-built revision, and retain a rollback route. An MCP boundary helps separate components but does not remove the need for service health checks, resource limits, and version management.</p><p>The <a href="https://github.com/OpenBMB/UltraRAG?utm_campaign=mediumkoc">UltraRAG project</a> can evolve independently of a published v2 tutorial. Pin the revision used for an experiment and check that its examples match that revision. Do not assume commands copied from one version describe every later release.</p><p>The next comparison I would run changes only the reranking stage while retaining the same corpus, questions, and retrieval candidates. If the result improves, the manifest and intermediate traces should explain where the gain appeared and what it cost.</p><p>Editorial disclosure: Prepared for Zilliz’s Medium content program. The framework overview is attributed; the manifest and experiment procedure are editorial examples.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f612ae441ac1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your PDF Answer Needs an Evidence Trail Before It Needs a Better Prompt]]></title>
            <link>https://medium.com/@PriyaSingh325/your-pdf-answer-needs-an-evidence-trail-before-it-needs-a-better-prompt-6e983f1e1013?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e983f1e1013</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[milvus]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Fri, 18 Sep 2026 11:21:58 GMT</pubDate>
            <atom:updated>2026-09-18T11:21:58.396Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HM-4x5R23ND19VzOqJO_sw.png" /></figure><p>Imagine a document assistant that gives a plausible answer and links to the correct PDF, but the quoted number belongs to the next column in a table. The model has a source. The source is relevant. The answer is still wrong because the evidence changed meaning somewhere between the page and the prompt.</p><p>For PDF question answering, I would debug that chain before replacing the model. Start with the page, follow its extracted structure into a chunk, inspect the retrieved passage, and only then review the answer. Each transition should leave enough information to explain what happened.</p><p>The <a href="https://milvus.io/blog/from-pdfs-to-answers-building-a-rag-knowledge-base-with-paddleocr-milvus-and-ernie.md?utm_campaign=mediumkoc">PaddleOCR, Milvus, and ERNIE walkthrough</a> offers an example architecture. It uses layout-aware parsing, stores chunk metadata with vectors, combines dense and keyword retrieval, and generates answers from the selected evidence. That integration is a starting point for the following evidence-focused design, not a guarantee that a particular PDF will parse correctly.</p><h3>Pick a small set of difficult pages</h3><p>Before bulk ingestion, assemble a document sample that reflects the actual workload. Include the table a customer asks about, a scanned page, a two-column report, a revised policy, and a page where the important fact appears only in a figure. A hundred easy paragraphs cannot substitute for those cases.</p><p>Annotate the page and region that contain each answer. Preserve the original file and a content digest so a later upload with the same filename does not silently replace the evaluation source. Keep questions that should produce no answer as well as questions with clear support.</p><p>For each failure, identify the first stage where evidence diverges. If a minus sign disappears during extraction, changing retrieval depth is unlikely to repair it. If the correct chunk exists but never reaches the prompt, work on candidate selection. If the passage reaches the model but the answer changes its units, investigate generation and validation.</p><h3>Preserve structure without pretending extraction is perfect</h3><p>Layout-aware tools such as <a href="https://github.com/PaddlePaddle/PaddleOCR?utm_campaign=mediumkoc">PaddleOCR</a> are useful components, but their output should remain inspectable. Save extraction artifacts separately from the source file and record the parser configuration. A parser upgrade can change reading order or table rendering even when the PDF is unchanged.</p><p>For tables, inspect whether a cell still carries its row label, column heading, and unit. For figures, retain a reference to the image and any caption. If the answer depends on a visual feature that the text extraction omitted, the application should recognize that limitation instead of treating the extracted text as complete.</p><p>Make manual inspection economical. A review view that places the source page next to its extracted text often reveals more than a large aggregate score. Review recurring failure patterns, then decide whether to change parsing, route certain documents differently, or exclude unsupported input types.</p><h3>Give every chunk a stable identity</h3><p>The identity of a chunk should distinguish its source revision and extraction policy. Otherwise, reprocessing a document can leave two different passages under the same identifier or make an earlier citation impossible to resolve.</p><p>Here is a runnable, standard-library example that constructs a chunk record from text already extracted from one page. The strings are synthetic. The code does not perform OCR or decide where a real document should be split.</p><p>import hashlib import json def make_chunk(document_id, revision, page, ordinal, text, parser_version): if page &lt; 1 or ordinal &lt; 0 or not text.strip(): raise ValueError(&quot;Invalid page, ordinal, or empty content&quot;) payload = { &quot;document_id&quot;: document_id, &quot;revision&quot;: revision, &quot;page&quot;: page, &quot;ordinal&quot;: ordinal, &quot;parser_version&quot;: parser_version, &quot;text&quot;: text, } encoded = json.dumps(payload, sort_keys=True, ensure_ascii=False).encode(&quot;utf-8&quot;) payload[&quot;chunk_id&quot;] = hashlib.sha256(encoded).hexdigest() return payload a = make_chunk(&quot;guide&quot;, &quot;rev-1&quot;, 2, 0, &quot;Synthetic policy text.&quot;, &quot;parser-a&quot;) b = make_chunk(&quot;guide&quot;, &quot;rev-2&quot;, 2, 0, &quot;Synthetic policy text.&quot;, &quot;parser-a&quot;) assert a[&quot;chunk_id&quot;] != b[&quot;chunk_id&quot;] assert a[&quot;page&quot;] == 2 print(json.dumps(a, indent=2))</p><p>The digest identifies this particular record; it does not prove that its content is true. Store access controls and source locations alongside it in the application schema. Do not expose internal storage paths as public citation URLs.</p><p>Choose chunk boundaries against the questions you need to answer. A small chunk may isolate the relevant sentence but lose a definition. A larger chunk may preserve context while adding unrelated material. Keep the evaluation corpus fixed while comparing those choices, and report what changed rather than assuming overlap always improves quality.</p><h3>Inspect retrieval before judging the answer</h3><p>For each test question, save the selected chunk IDs, their source revisions, and their ordering. Compare those records against the annotated evidence. An answer can occasionally look correct despite retrieval failure because the model already knows a similar fact; that should not count as successful document grounding.</p><p>If using <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a>, preserve the embedding model and preprocessing version with the collection or ingestion manifest. Query representations must be compatible with indexed representations. Matching dimensions alone do not establish that compatibility.</p><p>When combining dense and keyword results, measure each route separately before adding fusion. Include identifiers, abbreviations, numbers, and paraphrases in the question set. Term-based retrieval is affected by tokenization; it should not be described as guaranteed exact matching for every string.</p><h3>A citation must support the specific claim</h3><p>Require the answer to cite evidence at the level of the claim being made. A document-level link is often too broad for a long report. Page and section references help the reader check whether the cited passage actually supports the answer’s conditions and exceptions.</p><p>Do not turn a relevance score into a percentage probability that an answer is correct. Similarity and reranking scores help order candidates. Calibration would require a defined outcome and evaluation against labeled examples. Until then, describe the score as a retrieval signal, not confidence in the final statement.</p><p>Use an explicit insufficient-evidence path. If the sources disagree, show the conflict and their revisions. If a chart is required but unavailable to the answer model, say what is missing. A short limitation grounded in the evidence is more useful than an unsupported numerical answer.</p><h3>Make ingestion resumable and observable</h3><p>Batch ingestion should track document-level progress independently from the chat request. Record which pages parsed, which chunks were embedded, and which revision became searchable. If an external service returns a rate-limit error, use bounded retries and preserve the failed work for a later attempt.</p><p>Define how a new revision replaces the old one. A partial upload should not leave the assistant citing whichever revision happens to rank higher. Also check that deletion reaches extracted artifacts, indexed chunks, caches, and citation resolution according to the application’s retention policy.</p><p>The next experiment I would run changes one value in a table, reingests that document, and asks the same question again. The answer should cite the new revision, preserve the unit, and let a reviewer locate the exact evidence. That is a more informative acceptance test than asking whether the assistant sounds fluent.</p><p>Editorial disclosure: Prepared for Zilliz’s Medium content program. The linked integration is the source architecture; the evaluation procedure and synthetic code are editorial additions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e983f1e1013" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Give Hermes Better Recall: A Practical Hybrid Retrieval Workflow]]></title>
            <link>https://medium.com/@PriyaSingh325/give-hermes-better-recall-a-practical-hybrid-retrieval-workflow-a16f8eea7cef?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/a16f8eea7cef</guid>
            <category><![CDATA[hermes-agent]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Fri, 11 Sep 2026 07:51:20 GMT</pubDate>
            <atom:updated>2026-09-11T07:52:28.669Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8qQEm6D1qcatBQ7x" /></figure><p>Imagine asking an agent about the same deployment problem a week later, using different words. The useful note is still stored, but the answer never mentions it. Before changing the model or expanding the prompt, I would inspect the retrieval trace: did the relevant note ever reach the agent?</p><p>A <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a> <a href="https://milvus.io/blog/hermes-agent-learning-loop-milvus-hybrid-search.md?utm_campaign=mediumkoc">Hermes integration walkthrough</a> explores this failure mode. Its design places a retrieval script behind an agent Skill and combines dense retrieval with BM25, then merges the results. The article reports a demonstration involving repeated sessions and reusable Skills. I would treat that demonstration as a starting point, not a promise that any second conversation will create a Skill.</p><p>The useful engineering question is more concrete: can we give the agent a stable retrieval tool that handles paraphrases and exact identifiers, exposes its evidence, and remains understandable when it fails? Here is the workflow I would use. The small code example demonstrates rank fusion locally; it is not a claim of a completed Hermes deployment or a measured recall improvement.</p><h3>1. Keep retrieval and reusable procedures separate</h3><p>Hermes’s <a href="https://hermes-agent.nousresearch.com/docs/user-guide/features/skills?utm_campaign=mediumkoc">Skills documentation</a> describes a system for reusable procedures. When designing an integration, I would distinguish the procedure from the information it operates on. A note contains evidence; a procedure explains how to obtain and use evidence.</p><p>I would expose a retrieval tool with a small contract: a query and authorized project scope go in; ranked passages with stable IDs and source references come out. The Skill can explain when to call it, what empty results mean, and how to cite the returned sources.</p><p>The tool should remain useful even when no new Skill is created. Test it directly before asking the agent to learn a workflow around it. Otherwise, an agent-routing problem and a search-quality problem can produce the same symptom, leaving very little evidence about which component needs attention.</p><h3>2. Build a corpus that can be inspected</h3><p>Start with a bounded document set and preserve provenance during ingestion. For each chunk, I would retain its document ID, revision, section, source location, project scope, and ingestion version. Stable chunk IDs make it possible to compare two retrieval configurations without guessing which text each result represents.</p><p>Keep document content separate from any generated summary. A summary can help navigation, but the final answer should still be traceable to the underlying source. Decide how updates replace old chunks, how removed documents disappear from search, and how ingestion failures are reported.</p><p>Milvus’s <a href="https://milvus.io/docs/full-text-search.md?utm_campaign=mediumkoc">full-text search documentation</a> describes a BM25 path in which an analyzer processes raw text and a function generates sparse representations. Its dense counterpart requires a compatible representation for indexed passages and queries. I would record that model and preprocessing configuration with the corpus.</p><p>BM25 is term-based relevance ranking, not guaranteed literal string equality. Analyzer behavior matters for function names, underscores, error codes, and mixed-language content. Test the identifiers your application actually uses. If an exact identifier is a hard requirement, preserve it in a dedicated field and enforce the appropriate lookup or filter semantics.</p><h3>3. Retrieve two candidate lists before choosing the final context</h3><p>Dense retrieval is useful for differently worded requests; term-based retrieval contributes a separate signal for distinctive vocabulary. Neither route should be assumed sufficient without checking representative queries.</p><p>The <a href="https://milvus.io/docs/multi-vector-search.md?utm_campaign=mediumkoc">hybrid-search API</a> accepts multiple search requests, each targeting a vector field, and combines their candidates with a ranking strategy. This lets the two routes search the same corpus while retaining their own retrieval parameters.</p><p>I would begin with a modest candidate pool from each route and a smaller final context limit. Increasing the candidate pool can expose useful passages to fusion, but it also adds work and can introduce more irrelevant material. Tune those limits against a saved query set rather than accepting a multiplier as universally correct.</p><p>Keep project and authorization constraints consistent across routes. Do not let a semantically strong match from an unrelated project enter the final answer just because the other retrieval path was filtered correctly. The application must determine the allowed scope before it constructs the requests.</p><h3>4. Make fusion understandable with a small example</h3><p><a href="https://milvus.io/docs/rrf-ranker.md?utm_campaign=mediumkoc">Reciprocal Rank Fusion</a> combines ranking positions instead of comparing raw similarity scores from different retrieval methods. A document receives a contribution of 1 / (k + rank) from each list where it appears; those contributions are added.</p><p>Here is a complete standard-library example with invented document IDs. It demonstrates the mechanism without requiring a database, credentials, or a model download. It rejects duplicate IDs within a ranked list so that one malformed route cannot give a document extra votes.</p><pre>from collections import defaultdict<br><br>def fuse(rankings, k=60, limit=5):<br>    if k &lt;= 0 or limit &lt; 1:<br>        raise ValueError(&quot;k and limit must be positive&quot;)<br>    scores = defaultdict(float)<br>    for ranking in rankings:<br>        seen = set()<br>        for rank, document_id in enumerate(ranking, start=1):<br>            if document_id in seen:<br>                raise ValueError(f&quot;Repeated ID in one ranking: {document_id}&quot;)<br>            seen.add(document_id)<br>            scores[document_id] += 1.0 / (k + rank)<br>    ordered = sorted(scores, key=lambda doc: (-scores[doc], str(doc)))<br>    return [{&quot;id&quot;: doc, &quot;score&quot;: scores[doc]} for doc in ordered[:limit]]<br><br>if __name__ == &quot;__main__&quot;:<br>    dense = [&quot;async-guide&quot;, &quot;task-notes&quot;, &quot;timeout-fix&quot;]<br>    terms = [&quot;timeout-fix&quot;, &quot;socket-reference&quot;, &quot;task-notes&quot;]<br>    for result in fuse([dense, terms], limit=3):<br>        print(result)</pre><p>The resulting score is a ranking signal, not a probability that a passage is correct. Fusion cannot retrieve a document missing from both candidate lists, and agreement between two routes does not make an outdated document authoritative.</p><p>For a production integration, use the ranker supported by your installed server and client versions, and retain the pre-fusion IDs in a diagnostic trace. The local example is useful for understanding the arithmetic; it does not reproduce every database behavior or tie-breaking rule.</p><h3>5. Return evidence the agent can use</h3><p>I would return a compact object for each selected passage: chunk ID, document revision, source reference, text, and rank. Keep diagnostics separate from the passages so they do not consume the answer’s context budget unnecessarily.</p><p>The Skill should tell the agent how to handle an empty result and when the evidence is insufficient. Retrieved document text should be treated as source material rather than instructions that can change the tool’s behavior. A passage that mentions a shell command does not itself authorize the agent to execute it.</p><p>When a workflow becomes reusable, preserve its assumptions alongside the steps. Name the retrieval interface, required scope, expected output fields, and failure behavior. A future schema change should produce an identifiable compatibility problem instead of a confidently written answer based on missing fields.</p><h3>6. Test recall, latency, and workflow behavior separately</h3><p>Build a small acceptance set containing paraphrases, exact identifiers, ambiguous questions, stale revisions, and questions with no supporting document. Mark relevant chunks before tuning. Compare term-only, dense-only, and fused results using the same corpus and final result limit.</p><p>Then test the agent with the selected passages. Did it cite the correct revision? Did it distinguish missing evidence from an empty tool response? Did it call the tool again unnecessarily? These are different checks from whether the search returned a relevant chunk.</p><p>Measure latency across query representation, retrieval, fusion, and answer generation. For operations, set bounded timeouts and distinguish dependency failures from valid empty results. A fallback to term-only retrieval should be visible in diagnostics rather than reported as a normal hybrid result.</p><p>I would also measure the actual deployment’s memory and storage use. Document count alone is insufficient for capacity planning: chunk size, dimensions, index configuration, and concurrent queries all affect the workload. A quoted inexpensive server configuration is a hypothesis to test, not a production sizing guarantee.</p><p>The next experiment I would run is a fresh-session test with saved paraphrases and a controlled document update. Success would mean the retrieval tool finds the current evidence and the agent uses it correctly, whether or not it decides to create another Skill.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a16f8eea7cef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fix RAG Retrieval Errors with CRAG, LangGraph]]></title>
            <link>https://medium.com/@PriyaSingh325/fix-rag-retrieval-errors-with-crag-langgraph-23fdb0d6c94f?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/23fdb0d6c94f</guid>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Wed, 09 Sep 2026 07:17:13 GMT</pubDate>
            <atom:updated>2026-09-09T07:17:13.035Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aR3UiFsd7DhjJRwnEtIKIA.png" /></figure><p>Last week I found myself staring at a retrieval result that should have answered the question, and it didn’t. A user had asked how to configure an HTTPS certificate in Nginx. The top three chunks were about Apache, TLS in general, and a 2019 walkthrough that still used the old config format. Every chunk was semantically close. The similarity scores were high. The answer that came out was confident and completely useless.</p><p>I expected the failure to be in the prompt or the model. It wasn’t. The problem happened earlier: the retriever handed the generator the wrong kind of relevant. That moment is still a good summary of production <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a>. Similarity measures semantic closeness, but semantic closeness is not the same as answering the question.</p><p>A default RAG pipeline treats retrieved chunks as ground truth. The generator has no reason to question them, so fluent prose and wrong facts get mixed together. The failure is silent because the system produces an answer without any confidence warning. Retrieval metrics look good when the chunks are topically adjacent to the query, but that doesn’t mean they contain a correct, sufficient answer.</p><h3>The three retrieval failure modes</h3><p>Once I stopped blaming the <a href="https://zilliz.com/glossary/large-language-models-(llms)?utm_campaign=mediumkoc">LLM</a>, the bug pattern became clear. Traditional RAG doesn’t fail loudly when the retriever returns an unhelpful chunk. It fails silently. Three versions of that failure showed up in my pipeline.</p><p><strong>Retrieval mismatch.</strong> The document is topically related but doesn’t answer the query. Ask how to configure Nginx and the retriever returns an Apache setup guide. The vector space is close because both are about web servers and certificates, but the instruction content is completely different.</p><p><strong>Missing context.</strong> The chunk contains a fragment of the answer, but not enough context to be useful. For example, a snippet says ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key; but never states where the private key should point, which permissions it needs, or how to handle the key file during renewal. The generator has to fill in the gaps from memory, and memory is not retrieval.</p><p><strong>Outdated or contradictory information.</strong> The chunk is from a version of the software that no longer matches the user’s environment. The <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">vector search</a> finds the old syntax because there are many similar words, but the returned configuration block will break on a current version. Again, the LLM has no way to know the chunk is stale unless another chunk says so.</p><h3>Corrective RAG in practice</h3><p>Corrective RAG, or CRAG, adds a grading step between retrieval and generation. Instead of trusting whatever the retriever returns, the system sends the query plus the retrieved chunks to an LLM grader. The grader decides whether the information is enough, not just whether it’s topically close.</p><p>If the grade is high, generation proceeds with the original chunks. If the grade is low, the graph does a fallback search or calls a tool to retrieve more targeted knowledge. If the grade is mixed, the pipeline can filter out weak chunks and keep only the relevant ones.</p><p>The key difference is that the grader is not answering the original query. It’s reviewing the retrieved material against the query and deciding if that material supports an answer. This is a simpler task, and LLMs are good at it because it requires judging rather than generating facts.</p><h3>Building CRAG with LangGraph</h3><p>I chose LangGraph for this because the corrective flow is naturally a graph. The nodes are retrieve, grade, generate, and web_search. The edges are conditional, so the graph can route to a fallback search when retrieval quality is low.</p><p>First, define the state that moves between nodes.</p><pre>from typing import TypedDict, List<br><br>class RetrievalState(TypedDict):<br>    question: str<br>    docs: List[str]<br>    score: float<br>    action: str</pre><p>The state is simple: the original question, the document chunks, the max relevance score, and the action the grader decides to take.</p><p>Next, write the grader node. I use an LLM with a strict prompt: for each chunk, return a numerical score from 0 to 1 for whether it truly answers the question. The node keeps the highest score and sets the action.</p><pre>from llm import score_document<br><br>def grade_documents(state: RetrievalState) -&gt; RetrievalState:<br>    question = state[&quot;question&quot;]<br>    scores = [score_document(doc, question) for doc in state[&quot;docs&quot;]]<br>    best = max(scores)<br>    state[&quot;score&quot;] = best<br>    state[&quot;action&quot;] = &quot;generate&quot; if best &gt;= 0.7 else &quot;web_search&quot;<br>    return state</pre><p>The grading node doesn’t need to rewrite anything. It only needs to classify whether the retrieved set is good enough. That separation of concerns is what makes CRAG easy to test.</p><p>Now assemble the graph. The generator node is only reached when grading passes. The fallback node is only reached when the retriever missed the answer.</p><pre>from langgraph.graph import StateGraph, START, END<br><br>builder = StateGraph(RetrievalState)<br>builder.add_node(&quot;retrieve&quot;, retrieve)<br>builder.add_node(&quot;grade&quot;, grade_documents)<br>builder.add_node(&quot;generate&quot;, generate_answer)<br>builder.add_node(&quot;web_search&quot;, web_search_fallback)<br><br>builder.add_edge(START, &quot;retrieve&quot;)<br>builder.add_edge(&quot;retrieve&quot;, &quot;grade&quot;)<br>builder.add_conditional_edges(<br>    &quot;grade&quot;,<br>    lambda state: state[&quot;action&quot;],<br>    {&quot;generate&quot;: &quot;generate&quot;, &quot;web_search&quot;: &quot;web_search&quot;},<br>)<br>builder.add_edge(&quot;generate&quot;, END)<br>builder.add_edge(&quot;web_search&quot;, END)<br><br>app = builder.compile()</pre><p>When the graph runs, the retriever pulls chunks as usual. The grader looks at those chunks and decides whether they actually satisfy the query. If the max score is 0.7 or higher, generation continues. If not, the graph calls a web search tied to the original question and then generates from the fresh results.</p><h3>Make the fallback useful</h3><p>A fallback search only works if it knows about the original query and the gaps in the retrieved chunks. I pass both into the web search node. That gives the search tool enough context to avoid repeating the same mistake.</p><p>I also added one more check after the web search: if the fallback returns documents, the pipeline grades them again before generating. This prevents the model from exaggerating partial results. It’s more work, but it’s cheap compared with a confident wrong answer in production.</p><p>The CRAG pattern is not a silver bullet. If the original document source is missing an answer, a web search fallback can still fail. But that’s okay. The point of internal fallback is to fail loudly when there is no retrieval source, and then the system can say to the user: “I couldn’t find a reliable answer from the knowledge base.” In many tasks, that explicit “I don’t know” is more useful than a hallucinated solution.</p><h3>The final piece: observability</h3><p>I also added a log to the grade_documents node. Every time the score falls below 0.7, the log records the question, the top document title, and the score. After a week, that log becomes a dataset of hard failing queries. The same log shows which query groups consistently trigger web search, and I can use that to improve the retriever index.</p><p>This is the main lesson I learned: a RAG application doesn’t fail when the retriever returns bad chunks; it fails when the generator doesn’t know the chunks are bad. Adding a CRAG stage with LangGraph doesn’t make the bad chunks go away. It gives the system the ability to notice them before they reach the prompt and either repair or correct them. That small change converted my running issue from silent confident nonsense to traceable, evidence-based behavior. When the pipeline can’t answer, it knows, and it says so.</p><p>My Nginx question now returns a correct, and the two unrelated chunks are filtered out. If the system doesn’t have enough context, the graph drops the bad chunks and runs a web search. The end result is less hallucinated config and more reliable, because the retriever is no longer the last word on what counts as relevant.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23fdb0d6c94f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Use Boost Ranker for Business-Aware Vector Search]]></title>
            <link>https://medium.com/@PriyaSingh325/how-to-use-boost-ranker-for-business-aware-vector-search-e33ebb303623?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/e33ebb303623</guid>
            <category><![CDATA[filtering]]></category>
            <category><![CDATA[vector-search]]></category>
            <category><![CDATA[reranking]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Sun, 30 Aug 2026 13:34:21 GMT</pubDate>
            <atom:updated>2026-08-30T13:34:21.518Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dsjw3VXOl3EfDJ1_PbPBHw.png" /></figure><p>Last month I spent two days debugging why our document search kept surfacing outdated community posts above official guides. The embeddings were fine. The retrieval worked. The problem was that semantic similarity alone doesn’t understand “official matters more than similar.” A user searching for deployment instructions doesn’t want a three-year-old forum thread just because it mentions the same keywords — they want the current, authoritative guide.</p><p>That’s when I realized we needed a layer between retrieval and display: a way to adjust rankings using metadata without rebuilding indexes or retraining models. <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a> 2.6 introduced Boost Ranker for exactly this — a lightweight, rule-based reranking step that operates on scalar fields like is_official, publish_date, or stock_level. No external API calls, no model fine-tuning, just filter-and-weight logic applied after <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">vector search</a> returns candidates.</p><p>This article walks through how Boost Ranker works, when it solves real problems, and how to configure it with the Function API.</p><h3>What Boost Ranker Actually Does</h3><p>Boost Ranker is a post-processing step inside Milvus. After vector search retrieves a candidate set from each segment, Boost Ranker:</p><ol><li><strong>Filters</strong> matching records using a scalar condition (is_official == true, stock &gt; 10, etc.).</li><li><strong>Multiplies</strong> their scores by a configured weight.</li><li><strong>Optionally shuffles</strong> results slightly with a random factor for diversity.</li><li><strong>Re-sorts</strong> the entire candidate list by updated scores to produce the final Top-K.</li></ol><p>Because it only touches already-retrieved candidates — not the full dataset — the computational cost is negligible. You’re not rescanning the index; you’re adjusting a small list of results.</p><p>The key difference from model-based rerankers like BGE or Cohere: those services re-score results by semantic quality using an external <a href="https://zilliz.com/glossary/large-language-models-(llms)?utm_campaign=mediumkoc">LLM</a> or <a href="https://zilliz.com/blog/choosing-the-right-embedding-model-for-your-data?utm_campaign=mediumkoc">embedding model</a>. Boost Ranker adjusts scores using business rules you define. In production, you often want both — semantic reranking for relevance, then Boost Ranker for business logic.</p><h3>When You Actually Need This</h3><h3>Surfacing High-Value Results First</h3><p>The most common case: you have metadata that signals importance, and you want it reflected in rankings.</p><p><strong>E-commerce:</strong> Boost products from flagship stores or paid placements. Push items with high recent click-through rates higher. The <a href="https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc">embedding</a> model doesn’t know which seller is official; Boost Ranker does.</p><p><strong>Content platforms:</strong> Prioritize recently published posts using a publish_time field, or boost content from verified accounts. Semantic similarity doesn&#39;t understand &quot;newer is better&quot;—you have to tell it.</p><p><strong>Enterprise search:</strong> Give higher weight to documents where doctype == &quot;policy&quot; or is_canonical == true. When someone searches for compliance rules, you want the authoritative policy doc at the top, not a draft email that happens to use similar language.</p><p>All configurable with a filter expression and a weight multiplier. No embedding changes, no index rebuilds.</p><h3>Demoting Without Hard Filtering</h3><p>Boost Ranker can also <em>lower</em> rankings — a softer alternative to removing results entirely.</p><p><strong>Low-stock products:</strong> If stock &lt; 10, reduce their weight slightly. They&#39;re still findable if a user scrolls, but they won&#39;t dominate the top three slots.</p><p><strong>Stale content:</strong> Documents where year &lt; 2020 get ranked lower so newer material surfaces first. Users can still discover older content if they search precisely, but it won&#39;t crowd out fresh results.</p><p><strong>Flagged content:</strong> Lower the weight of posts flagged for review instead of hiding them. Limits exposure without hard censorship.</p><p>This is useful when you want results to remain discoverable but not prominent.</p><h3>Adding Controlled Diversity</h3><p>When many results have similar scores, the Top-K can look identical across queries. Boost Ranker’s random_score parameter introduces slight variation:</p><pre>&quot;random_score&quot;: {<br>    &quot;seed&quot;: 126,<br>    &quot;field&quot;: &quot;id&quot;<br>}</pre><p>The seed controls overall randomness for reproducibility; the field (usually the primary key) ensures the same record gets the same random value each time. This prevents the exact same items from always appearing first, which is useful for recommendation systems and exploratory search.</p><h3>Configuration Mechanics</h3><p>Boost Ranker is configured through the Function API. You create a Function object with function_type=FunctionType.RERANK and params.reranker=&quot;boost&quot;.</p><h3>Required Fields</h3><ul><li>name: any custom identifier</li><li>input_field_names: must be an empty list []</li><li>function_type: set to FunctionType.RERANK</li><li>params.reranker: must be &quot;boost&quot;</li></ul><h3>Key Parameters</h3><p><strong>params.weight</strong> (required): The multiplier applied to matching records&#39; scores. How you set it depends on your metric:</p><ul><li><strong>COSINE or IP</strong> (higher is better): use weight &gt; 1 to boost, weight &lt; 1 to demote.</li><li><strong>L2 / Euclidean</strong> (lower is better): use weight &lt; 1 to boost, weight &gt; 1 to demote.</li></ul><p><strong>params.filter</strong> (optional): A condition selecting which records get adjusted:</p><ul><li>&quot;doctype == &#39;abstract&#39;&quot;</li><li>&quot;is_premium == true&quot;</li><li>&quot;views &gt; 1000 and category == &#39;tech&#39;&quot;</li></ul><p>Only matching records are affected. Everything else keeps its original score.</p><p><strong>params.random_score</strong> (optional): Adds a random value between 0 and 1 for diversity.</p><h3>Single vs. Multiple Rules</h3><p>If you have one business constraint (boost official docs), pass the ranker directly to search(..., ranker=ranker).</p><p>If you need several constraints (prioritize in-stock items + demote low-rated products + add randomness), create multiple Function objects and combine them with FunctionScore. You configure boost_mode (how each rule combines with the original score) and function_mode (how rules combine with each other)—both support multiply or add.</p><h3>Walkthrough: Prioritizing Official Documentation</h3><p>Let’s work through a concrete example. I had a document collection where official guides were getting buried below community posts because the community posts used more colloquial language that matched user queries better. The solution: boost is_official == true by 20%.</p><h3>Schema Setup</h3><p>A collection with these fields:</p><ul><li>id (INT64): primary key</li><li>content (VARCHAR): document text</li><li>embedding (FLOAT_VECTOR, dim=3072): semantic vector</li><li>source (VARCHAR): origin—&quot;official&quot;, &quot;community&quot;, or &quot;ticket&quot;</li><li>is_official (BOOL): true if source == &quot;official&quot;</li></ul><p>The is_official field is what Boost Ranker will use.</p><pre>from pymilvus import (<br>    MilvusClient, DataType, Function, FunctionType<br>)<br><br>client = MilvusClient(uri=&quot;http://localhost:19530&quot;)<br>schema = MilvusClient.create_schema(auto_id=False)<br>schema.add_field(&quot;id&quot;, DataType.INT64, is_primary=True)<br>schema.add_field(&quot;content&quot;, DataType.VARCHAR, max_length=512)<br>schema.add_field(&quot;is_official&quot;, DataType.BOOL)<br>schema.add_field(&quot;embedding&quot;, DataType.FLOAT_VECTOR, dim=3072)<br><br>client.create_collection(&quot;milvus_collection&quot;, schema=schema)</pre><h3>Baseline Search</h3><p>First, run a search without Boost Ranker to see the default ranking:</p><pre>query = &quot;how to deploy milvus&quot;<br>results = client.search(<br>    collection_name=&quot;milvus_collection&quot;,<br>    data=[query],<br>    anns_field=&quot;embedding&quot;,<br>    limit=4,<br>    output_fields=[&quot;content&quot;, &quot;is_official&quot;]<br>)</pre><p>Typical output might show a community post ranked second because it uses casual phrasing that matches the query embedding well, even though an official guide is more authoritative.</p><h3>Adding Boost Ranker</h3><p>Now define a Boost Ranker that multiplies scores by 1.2 for official documents:</p><pre>boost_official = Function(<br>    name=&quot;boost_official&quot;,<br>    input_field_names=[],<br>    function_type=FunctionType.RERANK,<br>    params={<br>        &quot;reranker&quot;: &quot;boost&quot;,<br>        &quot;filter&quot;: &quot;is_official==true&quot;,<br>        &quot;weight&quot;: 1.2<br>    }<br>)<br><br>boosted_results = client.search(<br>    collection_name=&quot;milvus_collection&quot;,<br>    data=[query],<br>    anns_field=&quot;embedding&quot;,<br>    limit=4,<br>    ranker=boost_official,<br>    output_fields=[&quot;content&quot;, &quot;is_official&quot;]<br>)</pre><p>After applying Boost Ranker, official documents move up. A guide that was ranked fourth with a score of 0.64 gets multiplied to 0.77 and jumps to second place. Community posts aren’t removed — they just rank lower. That’s the design: keep <a href="https://zilliz.com/glossary/semantic-search?utm_campaign=mediumkoc">semantic search</a> as the foundation, then layer business rules on top.</p><h3>Production Tradeoffs I’ve Learned</h3><h3>Weight Tuning</h3><p>Start small. For <a href="https://zilliz.com/blog/similarity-metrics-for-vector-search?utm_campaign=mediumkoc">COSINE similarity</a>, a weight of 1.1–1.3 is usually enough to noticeably shift rankings without completely overriding semantic relevance. I’ve seen teams set weights too high (2.0+) and end up with low-relevance official docs dominating results just because they have the right metadata flag. Test with real queries and adjust.</p><h3>Latency</h3><p>Boost Ranker adds minimal overhead — typically sub-millisecond — because it operates on a small candidate set (your Top-K from vector search, not the full collection). The operations are simple filter-and-multiply. In my testing, the bottleneck remained the vector search itself, not the reranking step.</p><h3>Combining with Model Rerankers</h3><p>One design choice that tripped me up: in hybrid (multi-vector) search, Boost Ranker cannot be the top-level ranker. But you <em>can</em> use it inside each individual AnnSearchRequest to adjust results before they&#39;re merged. Common pattern: use RRF to merge multi-modal results, then apply Boost Ranker for metadata-based fine-tuning within each modality.</p><h3>When Boost Ranker Isn’t Enough</h3><p>If your business logic is complex — “boost products from Store A <em>unless</em> they’re out of stock <em>and</em> the user is in Region B” — you’ll need multiple rules combined with FunctionScore, or you&#39;ll need to move that logic into application code. Boost Ranker is designed for straightforward filter-and-weight rules, not branching decision trees.</p><h3>What I Wish I’d Known Earlier</h3><p>The first time I used Boost Ranker, I set the weight to 0.8 to demote low-stock items using COSINE similarity. Results got <em>worse</em>. Took me an hour to realize: for COSINE (higher is better), weights below 1.0 <em>lower</em> the score, which is correct for demotion. But I’d mentally flipped the logic. The rule: match the weight direction to your metric type.</p><p>Another mistake: I initially tried to use Boost Ranker to filter out results entirely by setting weight=0. That&#39;s not what it&#39;s for—use a hard filter in your search parameters instead. Boost Ranker is for <em>adjusting</em> rankings, not removing records.</p><p>Finally, if you’re testing with a small dataset (under 100 docs), the ranking changes can look dramatic because every result is in the Top-K. In production with thousands of candidates, the effect is more subtle — official docs move up a few positions, not from 50th to 1st. Set expectations accordingly.</p><p>Boost Ranker gives you a clean way to inject business logic into vector search without touching embeddings or rebuilding indexes. Boost authoritative content, demote stale results, add controlled diversity — all through simple configuration in the Milvus Function API. Whether you’re building <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a> pipelines, recommendation systems, or enterprise search, it helps bridge the gap between what’s semantically similar and what’s actually useful to your users.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e33ebb303623" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude Context: Reduce Claude Code Token Usage with Code Retrieval]]></title>
            <link>https://pub.aimind.so/claude-context-reduce-claude-code-token-usage-with-code-retrieval-449985a731e7?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/449985a731e7</guid>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Fri, 21 Aug 2026 13:25:38 GMT</pubDate>
            <atom:updated>2026-08-21T13:25:38.171Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UfN8byiJCHrOropR8HiLVA.png" /></figure><p>Last week I was debugging a Django bug report about __iso_year filtering behaving incorrectly. I fired up Claude Code, let it loose on the codebase, and watched it spin for twenty minutes. It kept searching for the wrong symbol, reading irrelevant files, and ultimately editing the wrong registration logic. The agent had consumed over 50,000 tokens before it even found the right class. The frustrating part wasn’t the model’s reasoning—it was the retrieval loop. Every grep call and file read added tokens that didn’t help the task.</p><p>That’s when I realized grep-style retrieval isn’t just slow; it’s expensive in Claude Code workflows. The model can handle large context windows, but the agent still pays for every irrelevant file it reads. I needed a retrieval layer that could return the smallest useful set of code chunks by meaning, not by literal text. That’s how I ended up building a <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a>-backed code retrieval server and calling it Claude Context.</p><p>Here’s what actually happened when I wired it up, what broke along the way, and how the numbers looked once it worked.</p><h3>Why grep-style retrieval burns tokens in AI coding agents</h3><p>Code retrieval for AI agents isn’t about finding lines that contain a keyword. It’s about finding the implementation and surrounding context that explains behavior. Grep-style retrieval fails in three predictable ways:</p><ul><li><strong>Information overload</strong>: large repositories produce many literal matches, and most are not useful for the current task.</li><li><strong>Semantic blindness</strong>: grep matches strings, not intent, behavior, or equivalent implementation patterns.</li><li><strong>Context loss</strong>: line-level matches don’t automatically include the surrounding class, dependencies, tests, or call graph.</li></ul><p>I saw this pattern repeatedly in our team’s debugging runs. The agent would search for swap_dims, find twenty scattered matches across files, read a few of them, search again, and still miss the actual implementation. Every file read added tokens, and the model’s context window filled with noise.</p><p>The alternative is <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a>-style retrieval: index the codebase in advance, then retrieve relevant chunks with semantic, lexical, or hybrid search. That requires chunking, embeddings, indexing, and update logic — exactly the kind of infrastructure most coding tools don’t want to own directly. But if you’re building an agent system that needs to stay fast and cost-effective, it’s the only approach that scales.</p><h3>What is Claude Context?</h3><p>Claude Context is an open-source Model Context Protocol (MCP) server that gives coding agents a better way to find relevant code. It parses the repository, generates embeddings, stores code chunks in a <a href="https://zilliz.com/learn/what-is-vector-database?utm_campaign=mediumkoc">vector database</a>, and exposes retrieval through MCP-compatible tools. The goal is simple: when the agent asks for context, return the smallest useful set of code chunks.</p><p>I built it to solve the retrieval loop I kept hitting with Claude Code. Instead of searching and reading files repeatedly, the agent can query a Milvus-backed index first, then read or edit only the files that matter. The same retrieval layer can support multiple agent interfaces because it’s exposed through MCP.</p><h3>How Claude Context works under the hood</h3><p>Claude Context has two main layers: a reusable core module and integration modules. The core handles parsing, chunking, indexing, search, and incremental sync. The upper layer exposes those capabilities through MCP and editor integrations.</p><p>MCP provides the interface between the <a href="https://zilliz.com/glossary/large-language-models-(llms)?utm_campaign=mediumkoc">LLM</a> host and external tools. By exposing Claude Context as an MCP server, the retrieval layer stays independent from any one IDE or coding assistant. The agent calls a search tool; Claude Context handles the code index and returns relevant chunks.</p><p>I chose Milvus for the vector database because it’s designed for high-performance <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">vector search</a> and can support dense vectors, sparse vectors, and reranking workflows. For teams building retrieval-heavy agent systems, the multi-vector hybrid search pattern is exactly what you need in production.</p><h3>Which embedding providers does Claude Context support?</h3><p>Claude Context supports multiple <a href="https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc">embedding</a> options:</p><ul><li><strong>OpenAI embeddings</strong>: general-purpose hosted embeddings with broad ecosystem support.</li><li><strong>Voyage AI embeddings</strong>: code-oriented retrieval, especially when search quality matters.</li><li><strong>Ollama</strong>: local embedding workflows for privacy-sensitive environments.</li></ul><p>The core module abstracts the vector database and embedding provider into a composable Context object:</p><pre>import { Context, MilvusVectorDatabase, OpenAIEmbedding } from &#39;@zilliz/claude-context-core&#39;;<br><br>// Initialize embedding provider<br>const embedding = new OpenAIEmbedding(process.env.OPENAI_API_KEY);<br>// Initialize vector database<br>const vectorDatabase = new MilvusVectorDatabase({<br>  uri: process.env.MILVUS_URI,<br>  token: process.env.MILVUS_TOKEN,<br>  collectionName: &#39;code_retrieval&#39;,<br>});<br>// Create context instance<br>const context = new Context({ embedding, vectorDatabase });<br>// Index your codebase with progress tracking<br>const stats = await context.indexCodebase(&#39;./your-project&#39;);<br>// Perform semantic search<br>const results = await context.semanticSearch(<br>  &#39;./your-project&#39;,<br>  &#39;vector database operations&#39;<br>);</pre><h3>How Claude Context chunks code and keeps indexes fresh</h3><p>Chunking and incremental updates determine whether a code retrieval system is usable in practice. If chunks are too small, the model loses context. If chunks are too large, the retrieval system returns noise. If indexing is too slow, developers stop using it.</p><p>Claude Context handles this with AST-based chunking, a fallback text splitter, and Merkle tree-based change detection.</p><h3>AST-based code chunking preserves context</h3><p>AST chunking is the primary strategy. Instead of splitting files by line count or character count, Claude Context parses code structure and chunks around semantic units such as functions, classes, and methods.</p><p>That gives each chunk three useful properties:</p><ul><li><strong>Syntactic completeness</strong>: functions and classes are not split in the middle.</li><li><strong>Logical coherence</strong>: related logic stays together, so retrieved chunks are easier for the model to use.</li><li><strong>Multi-language support</strong>: different tree-sitter parsers can handle JavaScript, Python, Java, Go, and other languages.</li></ul><p>For languages or files that AST parsing cannot handle, Claude Context falls back to LangChain’s RecursiveCharacterTextSplitter. It’s less precise than AST chunking, but it prevents indexing from failing on unsupported input.</p><pre>import { RecursiveCharacterTextSplitter } from &#39;langchain/text_splitter&#39;;<br>const splitter = RecursiveCharacterTextSplitter.fromLanguage(language, {<br>  chunkSize: 1000,<br>  chunkOverlap: 200,<br>});</pre><h3>Avoiding re-indexing the whole repository</h3><p>Re-indexing an entire repository after every change is too expensive. Claude Context uses a Merkle tree to detect exactly what changed.</p><p>A Merkle tree assigns each file a hash, derives each directory hash from its children, and rolls the whole repository into a root hash. If the root hash is unchanged, Claude Context can skip indexing. If the root changes, it walks down the tree to find the changed files and re-embeds only those files.</p><p>Sync runs in three stages:</p><ol><li><strong>Quick check</strong>: compare the current Merkle root with the last snapshot. If nothing changed, the check finishes quickly.</li><li><strong>Precise diff</strong>: walk the tree to identify added, deleted, and modified files. Only changed paths move forward.</li><li><strong>Incremental update</strong>: recompute embeddings for changed files and update Milvus.</li></ol><p>Local sync state is stored under ~/.context/merkle/, so Claude Context can restore the file hash table and serialized Merkle tree after a restart.</p><h3>What happens when Claude Code uses Claude Context?</h3><p>Setup is a single command before launching Claude Code:</p><pre>claude mcp add claude-context \<br>  -e OPENAI_API_KEY=your-openai-api-key \<br>  -e MILVUS_TOKEN=your-zilliz-cloud-api-key \<br>  -- npx @zilliz/claude-context-mcp@latest</pre><p>After indexing the repository, Claude Code can call Claude Context when it needs codebase context. In the same bug-finding scenario that previously burned time on grep and file reads, Claude Context found the exact file and line number with a full explanation.</p><p>The tool isn’t limited to bug hunting. It also helps with refactoring, duplicate code detection, issue resolution, test generation, and any task where the agent needs accurate repository context.</p><h3>How does Claude Context compare with grep on real bugs?</h3><p>The benchmark compares pure text search with Milvus-backed code retrieval on real debugging tasks. The difference isn’t just fewer tokens — Claude Context changes the agent’s search path so it starts closer to the implementation that needs to change.</p><h3>Case 1: Django YearLookup bug</h3><p><strong>Problem</strong>: In the Django framework, the YearLookup query optimization breaks __iso_year filtering. When using the __iso_year filter, the YearLookup class incorrectly applies the standard BETWEEN optimization—valid for calendar years, but not for ISO week-numbering years.</p><p><strong>Baseline (grep)</strong>:</p><ul><li>Retrieved 3,000+ lines of directory structure (~50k tokens)</li><li>Problem: massive information overload, no direct relevance</li><li>Found 21 matches across multiple files, most unrelated imports and registrations</li><li>Edited registration statements, but in the wrong direction</li></ul><p><strong>Claude Context</strong>:</p><ul><li>Searched for YearLookup</li><li>Found 10 results, including the YearLookup base class and year_lookup_bounds method</li><li>Directly located the core optimization logic, added ISO year handling</li><li>Token reduction: 93% fewer tokens</li></ul><h3>Case 2: Xarray swap_dims bug</h3><p><strong>Problem</strong>: The Xarray library’s .swap_dims() method unexpectedly mutates the original object, violating the expectation of immutability.</p><p><strong>Baseline (grep)</strong>:</p><ul><li>Retrieved directory structure across multiple subdirectories</li><li>Found 21 matches scattered across different contexts</li><li>Spent time navigating directories and reading nearby code before locating the actual implementation path</li></ul><p><strong>Claude Context</strong>:</p><ul><li>Searched for .swap_dims()</li><li>Found 10 results, including the implementation in xarray/core/dataset.py and related tests</li><li>Located the relevant swap_dims() implementation and related context faster</li><li>Token reduction: 62% fewer tokens</li></ul><p>At equivalent recall, Claude Context reduced token consumption by 39.4% and reduced tool calls by 36.1% in the benchmark. That matters because tool calls and irrelevant file reads often dominate the cost of coding-agent workflows.</p><h3>Production considerations</h3><p>The retrieval system needs to stay fast and reliable in day-to-day use. Here’s what I learned while running it in production:</p><ul><li><a href="https://zilliz.com/blog/choosing-the-right-embedding-model-for-your-data?utm_campaign=mediumkoc"><strong>Embedding model</strong></a><strong> versioning</strong>: embedding your chunks at indexing time and at query time with different models — even slightly different versions — tanks your retrieval recall. Always version-lock your embedding model and document which model your index was built with.</li><li><strong>Hybrid search tuning</strong>: dense vectors capture semantic similarity, but lexical matching helps with exact identifiers. Tune the balance between vector and bm25 weights based on your codebase’s naming conventions.</li><li><strong>Incremental sync overhead</strong>: Merkle tree diffs are fast, but re-embedding changed files can still be expensive for large repositories. Consider batching small changes and throttling sync during active development sessions.</li><li><strong>Memory usage</strong>: storing AST-based chunks and embeddings for large repos can exceed local RAM. If you’re running this on a laptop, monitor memory usage and consider offloading the vector database to a managed service.</li></ul><h3>Next steps</h3><p>If you want to try the exact tool from this post, start with the <a href="https://github.com/zilliz-io/claude-context">Claude Context GitHub repository</a> and the <a href="https://www.npmjs.com/package/@zilliz/claude-context-mcp">Claude Context MCP package</a>. The repository includes setup instructions, benchmarks, and the core TypeScript packages.</p><p>If you want to understand or customize the retrieval layer, these resources are useful next steps:</p><ul><li>Learn the vector database basics with the <a href="https://milvus.io/docs/install_standalone-docker.md">Milvus Quickstart</a>.</li><li>Explore <a href="https://milvus.io/docs/text_search.md">Milvus full text search</a> and the <a href="https://python.langchain.com/docs/modules/data_connection/retrievers/integrations/milvus/">LangChain full-text search tutorial</a> if you want to combine BM25-style search with dense vectors.</li><li>Review <a href="https://milvus.io/docs/alternatives.md">open-source vector search engines</a> if you are comparing infrastructure options.</li></ul><p>For help with Milvus or code retrieval architecture, join the <a href="https://milvus.io/community">Milvus community</a> or book <a href="https://milvus.io/office-hours">Milvus Office Hours</a> for one-on-one guidance.</p><h4>A Message from AI Mind</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*5Wm7sOfTpe5DEbhg.gif" /></figure><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://pub.aimind.so/">AI Mind Publication</a></li><li>🧠 Improve your <a href="https://www.aimind.so/prompt-generator?utm_source=pub&amp;utm_medium=message">AI prompts effortlessly and FREE</a></li><li><strong>🧰 Discover </strong><a href="https://www.aimind.so/?utm_source=pub&amp;utm_medium=message"><strong>Intuitive AI Tools</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=449985a731e7" width="1" height="1" alt=""><hr><p><a href="https://pub.aimind.so/claude-context-reduce-claude-code-token-usage-with-code-retrieval-449985a731e7">Claude Context: Reduce Claude Code Token Usage with Code Retrieval</a> was originally published in <a href="https://pub.aimind.so/">AI Mind</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Choose the Best Embedding Model for RAG in 2026: 10 Models Benchmarked]]></title>
            <link>https://medium.com/@PriyaSingh325/how-to-choose-the-best-embedding-model-for-rag-in-2026-10-models-benchmarked-c28975572053?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/c28975572053</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[embedding]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 09:43:15 GMT</pubDate>
            <atom:updated>2026-08-17T09:43:15.830Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*iBbVg5q9M4k3ZhragK8Bxw.png" /></figure><p>Last week, I was wrestling with a particularly stubborn <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a> pipeline. We were scaling up a knowledge base for a new internal project, and it wasn’t just text anymore. Suddenly, our engineering team was asking for images to be searchable, then PDFs, and next thing I knew, someone mentioned video. My trusty text-[embedding](https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc)-3-small was starting to look a lot like a toy when faced with real-world multimodal data.</p><p>The initial thought was, “Okay, let’s just swap it out for something better.” But “better” is a moving target, especially when you’re trying to integrate different data types, handle multiple languages, and deal with documents that are longer than a short story. The MTEB leaderboard is great for English text, but it doesn’t tell you anything about how a model handles a Chinese query for an English document, or how well it finds a specific fact in a 30,000-character report. And don’t even get me started on multimodal.</p><p>So, I decided to get serious. We needed a systematic way to evaluate embedding models for what we actually build: production RAG systems. I wasn’t looking for theoretical perfection; I needed something that worked across the messy realities of diverse data, long documents, and the ever-present pressure to keep storage costs down.</p><h3>Beyond MTEB: What Real-World RAG Needs</h3><p>Most of us start with OpenAI’s text-embedding-3-small because it&#39;s convenient. And for simple English text retrieval, it&#39;s fine. But as soon as your RAG system grows beyond a prototype, you hit the wall. Images, PDFs, multilingual content – these are standard requirements in production, and a text-only model just won&#39;t cut it.</p><p>The MTEB leaderboard is a good starting point, but it focuses almost exclusively on single-language text retrieval. It completely misses critical scenarios like:</p><ul><li><strong>Cross-modal retrieval:</strong> Asking a text query to find the right image.</li><li><strong>Cross-lingual search:</strong> A query in one language finding a document in another.</li><li><strong>Key </strong><a href="https://zilliz.com/learn/what-is-information-retrieval?utm_campaign=mediumkoc"><strong>information retrieval</strong></a><strong>:</strong> Finding a specific “needle” in a very long document “haystack.”</li><li><strong>Dimension compression (MRL):</strong> How much quality you lose when you truncate embeddings to save space in your <a href="https://zilliz.com/learn/what-is-vector-database?utm_campaign=mediumkoc">vector database</a>.</li></ul><p>These are the capabilities that make or break a RAG pipeline in the wild. I needed to see how models performed on these dimensions, not just on English sentence similarity. So, I set up a benchmark to test exactly these things.</p><h3>The Models I Put Through Their Paces</h3><p>I gathered 10 models, a mix of API services and open-source options, plus CLIP ViT-L-14 as a baseline from 2021 to see how far things have come.</p><p>Here’s a quick overview of what I tested:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TtgNcNinIdGhT0uGNDLYBQ.png" /></figure><p>My goal was to see how these models stacked up against real-world RAG challenges.</p><h3>Cross-Modal Retrieval: Text to Image, and Back</h3><p>If your RAG pipeline is going to handle images, your <a href="https://zilliz.com/blog/choosing-the-right-embedding-model-for-your-data?utm_campaign=mediumkoc">embedding model</a> needs to put both text and images into the same vector space. This means a text query should be able to find the right image, and an image should be able to find its description.</p><p>I took 200 image-text pairs and, crucially, generated 3 <em>hard negatives</em> for each image. These were descriptions that differed by only one or two details. The model had to find the correct image among 200 images and 600 distractors. This wasn’t about finding <em>any</em> image; it was about finding the <em>right</em> image when the wrong ones looked very similar.</p><p>Here’s a simplified version of how I generated the hard negatives for evaluation:</p><pre>from openai import OpenAI<br><br>client = OpenAI()<br>def generate_hard_negative(correct_description: str) -&gt; str:<br>    prompt = f&quot;&quot;&quot;Given the correct image description:<br>    &quot;{correct_description}&quot;<br>    Generate a hard negative description that is very similar but changes one<br>    or two key visual details. The new description must be plausible for a<br>    different, but similar, image.<br>    Example:<br>    Correct: &quot;A red car is parked on a busy street.&quot;<br>    Hard Negative: &quot;A blue car is parked on a quiet street.&quot;<br>    Hard Negative:&quot;&quot;&quot;<br>    <br>    response = client.chat.completions.create(<br>        model=&quot;gpt-4o-mini&quot;,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt}],<br>        max_tokens=100<br>    )<br>    return response.choices[0].message.content.strip()</pre><p>The results were interesting. Qwen3-VL-2B, an open-source model, actually came out on top, ahead of closed-source APIs like Gemini Embedding 2. The key factor here was the “modality gap” — the L2 distance between the text and image embedding clusters. Qwen had a significantly smaller gap, meaning its text and image embeddings were more tightly integrated in the vector space. This is critical for storing both modalities in the same collection in a vector database and expecting reliable cross-modal <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">similarity search</a>.</p><h3>Cross-Lingual Retrieval: Bridging Language Barriers</h3><p>Many of our internal knowledge bases are multilingual. A user in one region might ask a question in their native language, but the answer could be in an English document. The embedding model needs to understand the semantic meaning across languages, not just within a single one.</p><p>I created 166 parallel sentence pairs in Chinese and English, with three difficulty tiers. The “Hard” tier included Chinese idioms that required genuine semantic understanding to align with their English equivalents, not just literal translation.</p><pre>import numpy as np<br>from sklearn.metrics.pairwise import cosine_similarity<br><br>def evaluate_cross_lingual(model_embeddings_zh, model_embeddings_en):<br>    zh_queries = model_embeddings_zh[&#39;correct&#39;]<br>    en_targets = np.vstack([model_embeddings_en[&#39;correct&#39;], <br>                            model_embeddings_en[&#39;hard_negatives&#39;]])<br>    hits = 0<br>    for i, query_embed in enumerate(zh_queries):<br>        similarities = cosine_similarity(query_embed.reshape(1, -1), en_targets)<br>        top_match_idx = np.argmax(similarities)<br>        <br>        if top_match_idx == i: # Assuming correct_pairs are aligned by index<br>            hits += 1<br>            <br>    accuracy = hits / len(zh_queries)<br>    return accuracy</pre><p>Gemini Embedding 2 was the clear winner here, scoring almost perfectly across all difficulty levels, including the tricky idioms. This tells me that if you’re building a truly global RAG system, Gemini’s cross-lingual capabilities are top-tier. Lightweight, English-focused models completely fall apart on cross-lingual tasks. They might be cheap and fast, but if your data isn’t 100% English, they’re a non-starter.</p><h3>Key Information Retrieval: Finding the Needle in the Haystack</h3><p>Production RAG systems often deal with very long documents — legal contracts, research papers, technical manuals. The challenge isn’t just indexing these documents, but ensuring the embedding model can still pinpoint a specific fact buried deep within thousands of characters.</p><p>I took Wikipedia articles, ranging from 4K to 32K characters, and inserted a fabricated “needle” fact at different positions. The model’s job was to determine which version of the document (with the needle or without) was more similar to a query about that needle.</p><pre>from transformers import AutoTokenizer, AutoModel<br>import torch<br>from sklearn.metrics.pairwise import cosine_similarity<br><br>def get_document_embedding(text: str, tokenizer, model):<br>    inputs = tokenizer(text, return_tensors=&quot;pt&quot;, truncation=True, <br>                      max_length=tokenizer.model_max_length)<br>    with torch.no_grad():<br>        outputs = model(**inputs)<br>    return outputs.last_hidden_state.mean(dim=1).squeeze().numpy()<br>def evaluate_needle_in_haystack(query_text, doc_with_needle, <br>                                doc_without_needle, tokenizer, model):<br>    query_embed = get_document_embedding(query_text, tokenizer, model)<br>    doc_with_embed = get_document_embedding(doc_with_needle, tokenizer, model)<br>    doc_without_embed = get_document_embedding(doc_without_needle, tokenizer, model)<br>    sim_with = cosine_similarity(query_embed.reshape(1, -1), <br>                                 doc_with_embed.reshape(1, -1))[0][0]<br>    sim_without = cosine_similarity(query_embed.reshape(1, -1), <br>                                    doc_without_embed.reshape(1, -1))[0][0]<br>    return sim_with &gt; sim_without</pre><p>Gemini Embedding 2 was the only model that scored perfectly across the full 4K–32K range. Many other top models performed perfectly within their context windows (up to 8K), but the lightweight models degraded sharply at just 4K characters. This is a critical factor for any RAG system dealing with lengthy documents. If your documents are routinely long, you need a model with a robust context window and the ability to maintain semantic understanding throughout.</p><h3>MRL Dimension Compression: Saving Space Without Losing Quality</h3><p>Storage costs for <a href="https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc">vector embeddings</a> can add up quickly, especially with large datasets and high-dimensional vectors. Matryoshka Representation Learning (MRL) is a technique where the first N dimensions of a vector are meaningful on their own, allowing you to truncate the vector to a smaller size without a massive drop in quality. Going from 3072 to 256 dimensions is a 12x storage reduction — that’s huge for production systems.</p><p>I tested how well models preserved quality when truncated to 1024, 512, and 256 dimensions using the STS-B benchmark.</p><pre>import numpy as np<br>from scipy.stats import spearmanr<br>from sklearn.metrics.pairwise import cosine_similarity<br><br>def evaluate_mrl_truncation(full_dim_embeddings, truncated_embeddings, human_scores):<br>    full_dim_similarities = []<br>    for i in range(0, len(full_dim_embeddings), 2):<br>        sim = cosine_similarity(full_dim_embeddings[i].reshape(1, -1), <br>                                full_dim_embeddings[i+1].reshape(1, -1))[0][0]<br>        full_dim_similarities.append(sim)<br>    rho_full, _ = spearmanr(human_scores, full_dim_similarities)<br>    truncated_similarities = []<br>    for i in range(0, len(truncated_embeddings), 2):<br>        sim = cosine_similarity(truncated_embeddings[i].reshape(1, -1), <br>                                truncated_embeddings[i+1].reshape(1, -1))[0][0]<br>        truncated_similarities.append(sim)<br>    rho_truncated, _ = spearmanr(human_scores, truncated_similarities)<br>    <br>    decay = ((rho_full - rho_truncated) / rho_full) * 100 if rho_full != 0 else 0<br>    return rho_full, rho_truncated, decay</pre><p>Voyage Multimodal 3.5 and Jina Embeddings v4 led the pack, losing less than 1% quality at 256 dimensions. This is because they were explicitly trained with MRL in mind. Gemini Embedding 2, while strong in other areas, showed more decay. This is a crucial design tradeoff: if storage and memory footprint are paramount for your vector database, then MRL-optimized models are a must. If not, a model like Gemini might still be the right choice.</p><h3>The Full Scorecard and My Takeaways</h3><p>Here’s the consolidated view of how the models performed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CDDNk0tpvfExLHuPV-macQ.png" /></figure><p>“ — “ means the model doesn’t support that modality or capability.</p><p>No single model wins everything. That’s the pragmatic truth of building real systems.</p><ul><li><strong>For cross-modal:</strong> Qwen3-VL-2B surprised me by leading, even over closed-source APIs. Its small modality gap is a huge advantage. If you need to self-host and have multimodal data, it’s a strong contender.</li><li><strong>For cross-lingual and long documents:</strong> Gemini Embedding 2 is the clear all-rounder. Its performance on idioms and 32K-character documents is unmatched. If your RAG needs to handle diverse languages and very long content, this is your model.</li><li><strong>For dimension compression:</strong> Voyage Multimodal 3.5 and Jina Embeddings v4 are excellent choices. If you’re managing a massive vector database and every byte counts, these models will save you significant infrastructure costs without a major hit to retrieval quality.</li></ul><p>The fix was simpler than I expected: understanding my actual use case requirements dictated the model choice. It wasn’t about finding “the best” model in a vacuum, but the best model for <em>my</em> specific constraints. Always evaluate new models against your own data and your own specific use cases. Public benchmarks are a starting point, but they rarely capture the nuances of your production environment. The embedding landscape changes so rapidly that what’s “best” today might be old news in six months. Building an internal evaluation pipeline, even a simple one, is the most durable investment you can make.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c28975572053" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build Multimodal Search for 3D Assets with Tripo and Zilliz Cloud]]></title>
            <link>https://medium.com/@PriyaSingh325/build-multimodal-search-for-3d-assets-with-tripo-and-zilliz-cloud-f64612067ca2?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/f64612067ca2</guid>
            <category><![CDATA[multimodal-rag]]></category>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Mon, 03 Aug 2026 08:37:25 GMT</pubDate>
            <atom:updated>2026-08-03T08:37:25.381Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zu1ECm-gQUaeYlrg5L0ESA.png" /></figure><p>Last week, I was looking at how one of our internal creative teams was managing their 3D assets. We’ve been using AI generation tools like Tripo to rapidly prototype new models for various projects — everything from game assets to marketing visuals. The problem wasn’t generating them; it was finding them again. They had thousands of models, each with multiple variations, scattered across network drives and cloud storage, organized into a folder structure that only made sense to the person who created it. Designers were recreating assets they knew existed but couldn’t locate, and project managers were spending more time hunting than actually using the models.</p><p>It was a classic retrieval problem, but with a twist: these weren’t just text documents. They were complex 3D models, often with associated images and metadata. I knew a traditional keyword search wouldn’t cut it. We needed something that understood visual similarity, not just text. That’s when I started thinking about multimodal embeddings and how we could wire them into a searchable catalog.</p><p>The core idea was simple: instead of just indexing filenames or basic tags, we’d embed a visual representation of each 3D asset into a vector space. Then, we could query that space using text, an image, or even a combination of both. The goal was to move from a chaotic pile of files to a truly queryable asset library.</p><h3>The Asset Problem: More Than Just Files</h3><p>A 3D model isn’t just a single file. In a real production pipeline, it’s usually a mesh, textures, materials, and a bunch of associated metadata. For our purposes, the most important piece for search was a high-quality render preview image. Why a render preview? Because it captures the visual essence of the 3D model — its shape, color, material properties, and overall style — in a format that a multimodal <a href="https://zilliz.com/blog/choosing-the-right-embedding-model-for-your-data?utm_campaign=mediumkoc">embedding model</a> can easily process.</p><p>Here’s what an asset record typically looked like for us:</p><ul><li><strong>Render Preview Image:</strong> A .webp or .png image showing the model. This is what we&#39;d embed.</li><li><strong>Input Image (Optional):</strong> If the 3D model was generated from an image, we’d keep that reference. Useful for debugging or refining future searches.</li><li><strong>Structured Metadata:</strong> This was crucial for filtering. Things like:</li><li>caption: A text description or the prompt used to generate it.</li><li>category: e.g., &quot;weapon&quot;, &quot;character&quot;, &quot;environment&quot;</li><li>style: e.g., &quot;fantasy&quot;, &quot;realistic&quot;, &quot;sci-fi&quot;</li><li>color: Primary colors present.</li><li>use_case: e.g., &quot;game asset&quot;, &quot;marketing&quot;, &quot;e-commerce&quot;</li><li>project_id: Which project it belonged to.</li><li>owner_id: Who generated it.</li><li>url: A link back to the original asset&#39;s storage location.</li></ul><p>This combination of a visual <a href="https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc">embedding</a> and rich metadata is what transforms a simple search into a powerful catalog. You can find visually similar items <em>and</em> filter them down to exactly what you need for a specific project or use case.</p><h3>Wiring Up the Retrieval Layer</h3><p>For the retrieval layer, I decided to use <a href="https://zilliz.com/cloud?utm_campaign=mediumkoc">Zilliz Cloud</a>. It’s a managed <a href="https://zilliz.com/learn/what-is-vector-database?utm_campaign=mediumkoc">vector database</a> service that’s built on <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a>, which I’ve used before for <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a> pipelines. It handles the vector storage, <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">similarity search</a>, and, critically for this project, allows combining <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">vector search</a> with structured filters.</p><p>The workflow looked like this:</p><ol><li><strong>Generate 3D Asset (Tripo):</strong> The creative team generates models.</li><li><strong>Render Preview + Metadata:</strong> We extract a render image and all relevant metadata.</li><li><strong>Multimodal Embedding:</strong> The render image (and potentially the caption) gets embedded into a vector.</li><li><strong>Index in </strong><a href="https://zilliz.com/?utm_campaign=mediumkoc"><strong>Zilliz</strong></a><strong> Cloud:</strong> The embedding and metadata are stored as an entity in a Zilliz Cloud collection.</li><li><strong>Search:</strong> Users query using text, an image, or both, combined with metadata filters.</li><li><strong>Return Results:</strong> Matched render images and their metadata are returned.</li></ol><p>This setup supports common creative search patterns. A game artist could search for “female character” and filter by use_case == &quot;game asset&quot;. A marketing person could upload a reference image of a &quot;leather bag&quot; and filter by project_id == &quot;summer_campaign&quot;. It&#39;s about giving them control beyond just keywords.</p><h3>Building the Collection Schema</h3><p>The first step was to define the schema for our Zilliz Cloud collection. Each record would represent one 3D asset. The multimodal vector would live alongside all the structured metadata. I went with a 3072-dimensional vector field, matching the output of the embedding model I planned to use, and COSINE similarity for vector comparisons.</p><p>Here’s a simplified version of the schema definition:</p><pre>from pymilvus import MilvusClient, DataType, FieldSchema, CollectionSchema<br><br>VECTOR_FIELD = &quot;multimodal_vector&quot;<br>VECTOR_DIM = 3072 # Dimension of the embedding model output<br><br>schema = MilvusClient.create_schema(<br>    auto_id=False,<br>    enable_dynamic_field=False # Keeping schema strict for production<br>)<br><br>schema.add_field(FieldSchema(&quot;id&quot;, DataType.INT64, is_primary=True))<br>schema.add_field(FieldSchema(&quot;project_id&quot;, DataType.VARCHAR, max_length=64))<br>schema.add_field(FieldSchema(&quot;operator_id&quot;, DataType.VARCHAR, max_length=64))<br>schema.add_field(FieldSchema(&quot;caption&quot;, DataType.VARCHAR, max_length=2048))<br>schema.add_field(FieldSchema(&quot;llm_object&quot;, DataType.VARCHAR, max_length=512))<br>schema.add_field(FieldSchema(&quot;llm_category&quot;, DataType.VARCHAR, max_length=128))<br>schema.add_field(FieldSchema(&quot;llm_style&quot;, DataType.VARCHAR, max_length=128))<br>schema.add_field(FieldSchema(&quot;llm_color&quot;, DataType.VARCHAR, max_length=256))<br>schema.add_field(FieldSchema(&quot;llm_use_case&quot;, DataType.VARCHAR, max_length=128))<br>schema.add_field(FieldSchema(&quot;generation_mode&quot;, DataType.VARCHAR, max_length=32))<br>schema.add_field(FieldSchema(&quot;url&quot;, DataType.VARCHAR, max_length=512))<br>schema.add_field(FieldSchema(&quot;render_image_file&quot;, DataType.VARCHAR, max_length=256))<br>schema.add_field(FieldSchema(VECTOR_FIELD, DataType.FLOAT_VECTOR, dim=VECTOR_DIM))<br><br># Then create the index<br>index_params = MilvusClient.prepare_index_params()<br>index_params.add_index(<br>    field_name=VECTOR_FIELD,<br>    index_type=&quot;AUTOINDEX&quot;, # AUTOINDEX is fine for initial setup<br>    metric_type=&quot;COSINE&quot;,<br>)<br><br># Assuming &#39;client&#39; is an initialized MilvusClient instance<br># client.create_collection(collection_name=&quot;tripo_assets&quot;, schema=schema, index_params=index_params)</pre><p>One thing I learned the hard way: always align your vector dimension with your embedding model’s output <em>before</em> creating the collection. Changing it later is a pain. Also, while AUTOINDEX is convenient for getting started, for production-scale systems, you&#39;ll want to dive into specific index types like IVF_FLAT or [HNSW](https://zilliz.com/learn/hierarchical-navigable-small-worlds-HNSW?utm_campaign=mediumkoc) and tune their parameters for optimal performance and recall.</p><h3>Generating Multimodal Embeddings</h3><p>For embedding, I used a multimodal model via OpenRouter. The process involves taking the render preview image, converting it to a base64 data URL, sending it to the API, and then taking the returned vector and L2-normalizing it.</p><p>Here’s the core logic for embedding an image:</p><pre>import base64<br>import mimetypes<br>import requests<br>from pathlib import Path<br>import numpy as np<br><br># Helper for L2 normalization<br>def l2_normalize(vector):<br>    norm = np.linalg.norm(vector)<br>    if norm == 0:<br>        return vector<br>    return (vector / norm).tolist()<br><br>def data_url_for_image(path: Path) -&gt; str:<br>    mime_type = mimetypes.guess_type(path.name)[0] or &quot;image/webp&quot;<br>    data = base64.b64encode(path.read_bytes()).decode(&quot;ascii&quot;)<br>    return f&quot;data:{mime_type};base64,{data}&quot;<br><br>class OpenRouterEmbeddingClient:<br>    def __init__(self, api_key: str):<br>        self._headers = {<br>            &quot;Authorization&quot;: f&quot;Bearer {api_key}&quot;,<br>            &quot;Content-Type&quot;: &quot;application/json&quot;<br>        }<br><br>    def embed_content(self, content: list[dict]) -&gt; list[float]:<br>        body = {<br>            &quot;model&quot;: &quot;google/gemini-embedding-2-preview&quot;, # Or another suitable multimodal model<br>            &quot;input&quot;: [{&quot;content&quot;: content}],<br>            &quot;encoding_format&quot;: &quot;float&quot;,<br>        }<br>        response = requests.post(<br>            &quot;https://openrouter.ai/api/v1/embeddings&quot;,<br>            headers=self._headers,<br>            json=body,<br>            timeout=120,<br>        )<br>        response.raise_for_status() # Raise an exception for bad status codes<br>        vector = response.json()[&quot;data&quot;][0][&quot;embedding&quot;]<br>        return l2_normalize(vector)<br><br>    def embed_image(self, image_path: Path) -&gt; list[float]:<br>        return self.embed_content(<br>            [<br>                {<br>                    &quot;type&quot;: &quot;image_url&quot;,<br>                    &quot;image_url&quot;: {&quot;url&quot;: data_url_for_image(image_path)},<br>                }<br>            ]<br>        )<br><br>    def embed_text(self, text: str) -&gt; list[float]:<br>        return self.embed_content(<br>            [<br>                {<br>                    &quot;type&quot;: &quot;text&quot;,<br>                    &quot;text&quot;: text<br>                }<br>            ]<br>        )<br><br>    def embed_text_image(self, text: str, image_path: Path) -&gt; list[float]:<br>        return self.embed_content(<br>            [<br>                {<br>                    &quot;type&quot;: &quot;text&quot;,<br>                    &quot;text&quot;: text<br>                },<br>                {<br>                    &quot;type&quot;: &quot;image_url&quot;,<br>                    &quot;image_url&quot;: {&quot;url&quot;: data_url_for_image(image_path)}<br>                }<br>            ]<br>        )<br><br># Example usage:<br># api_key = os.getenv(&quot;OPENROUTER_API_KEY&quot;)<br># embedding_client = OpenRouterEmbeddingClient(api_key)<br># image_vector = embedding_client.embed_image(Path(&quot;./milvus_render_images/fantasy_sword.webp&quot;))<br># print(image_vector[:5]) # Print first 5 dimensions</pre><p>For a real ingestion pipeline, especially with thousands of assets, caching these embeddings is non-negotiable. Calling an external API for every single asset during ingestion would be incredibly slow and expensive if you’re constantly rebuilding your index. The fix was simpler than I expected: a local cache (e.g., a simple JSON file or a SQLite database) mapping asset IDs to their embeddings. This makes imports deterministic and allows for quick schema changes or re-indexing without re-embedding everything.</p><h3>Importing and Searching</h3><p>Once the embeddings were generated and cached, importing the data into Zilliz Cloud was straightforward. Each row from our metadata CSV, combined with its corresponding cached vector, became an entity.</p><pre>def row_to_entity(row_index: int, row: dict, cached_item: dict) -&gt; dict:<br>    # This maps CSV row data to the collection schema fields<br>    return {<br>        &quot;id&quot;: row_index,<br>        &quot;project_id&quot;: row.get(&quot;project_id&quot;, &quot;&quot;),<br>        &quot;operator_id&quot;: row.get(&quot;operator_id&quot;, &quot;&quot;),<br>        &quot;caption&quot;: row.get(&quot;caption&quot;, &quot;&quot;),<br>        &quot;llm_object&quot;: row.get(&quot;llm_object&quot;, &quot;&quot;),<br>        &quot;llm_category&quot;: row.get(&quot;llm_category&quot;, &quot;&quot;),<br>        &quot;llm_style&quot;: row.get(&quot;llm_style&quot;, &quot;&quot;),<br>        &quot;llm_color&quot;: row.get(&quot;llm_color&quot;, &quot;&quot;),<br>        &quot;llm_use_case&quot;: row.get(&quot;llm_use_case&quot;, &quot;&quot;),<br>        &quot;generation_mode&quot;: row.get(&quot;generation_mode&quot;, &quot;&quot;), # Assuming a helper for this<br>        &quot;url&quot;: row.get(&quot;url&quot;, &quot;&quot;),<br>        &quot;render_image_file&quot;: row.get(&quot;render_image_file&quot;, &quot;&quot;),<br>        VECTOR_FIELD: cached_item[VECTOR_FIELD],<br>    }<br><br># Example of how you might import:<br># entities = []<br># for i, row in enumerate(csv_data): # Assume csv_data is loaded from milvus_dataset.csv<br>#     cached_embedding = load_from_cache(row[&quot;asset_id&quot;]) # Load pre-computed embedding<br>#     entities.append(row_to_entity(i, row, {&quot;multimodal_vector&quot;: cached_embedding}))<br># client.insert(collection_name=&quot;tripo_assets&quot;, data=entities)</pre><p>The real magic happens during search. We can query by text, image, or both. The query itself is embedded using the <em>same</em> multimodal model used for indexing, ensuring the query vector is in the same semantic space. Then, Zilliz Cloud performs a similarity search.</p><p>Here’s how a flexible search function might look:</p><pre>import json<br>import os<br><br>def connect_client():<br>    # Placeholder for connecting to Zilliz Cloud<br>    # In a real scenario, this would use MilvusClient with your Zilliz Cloud endpoint and token<br>    return MilvusClient(<br>        uri=os.getenv(&quot;ZILLIZ_CLOUD_URI&quot;),<br>        token=os.getenv(&quot;ZILLIZ_CLOUD_TOKEN&quot;)<br>    )<br><br>def filter_expr(args):<br>    # Builds a filter expression based on command-line arguments<br>    filters = []<br>    if args.use_case:<br>        filters.append(f&#39;llm_use_case == &quot;{args.use_case}&quot;&#39;)<br>    if args.category:<br>        filters.append(f&#39;llm_category == &quot;{args.category}&quot;&#39;)<br>    # Add more filters as needed<br>    return &quot; and &quot;.join(filters) if filters else &quot;&quot;<br><br>def perform_search(args):<br>    if not args.text and not args.image:<br>        raise SystemExit(&quot;Provide --text, --image, or both.&quot;)<br><br>    embedding_client = OpenRouterEmbeddingClient(os.getenv(&quot;OPENROUTER_API_KEY&quot;))<br>    client = connect_client()<br><br>    vector = None<br>    if args.text and args.image:<br>        vector = embedding_client.embed_text_image(args.text, Path(args.image))<br>    elif args.image:<br>        vector = embedding_client.embed_image(Path(args.image))<br>    else:<br>        vector = embedding_client.embed_text(args.text)<br><br>    results = client.search(<br>        collection_name=&quot;tripo_assets&quot;,<br>        data=[vector],<br>        anns_field=VECTOR_FIELD,<br>        filter=filter_expr(args),<br>        limit=args.top_k,<br>        output_fields=[<br>            &quot;project_id&quot;, &quot;caption&quot;, &quot;llm_object&quot;, &quot;llm_category&quot;,<br>            &quot;llm_style&quot;, &quot;llm_color&quot;, &quot;llm_use_case&quot;, &quot;generation_mode&quot;,<br>            &quot;render_image_file&quot;, &quot;url&quot;<br>        ],<br>        search_params={&quot;metric_type&quot;: &quot;COSINE&quot;},<br>    )<br><br>    for rank, hit in enumerate(results[0], start=1):<br>        entity = hit[&quot;entity&quot;]<br>        print(<br>            json.dumps(<br>                {<br>                    &quot;rank&quot;: rank,<br>                    &quot;score&quot;: hit[&quot;distance&quot;],<br>                    **entity,<br>                },<br>                ensure_ascii=False,<br>                indent=2,<br>            )<br>        )<br><br># Example command line calls (assuming args are parsed)<br># perform_search(arg_parser.parse_args([&#39;--text&#39;, &#39;female&#39;, &#39;--use-case&#39;, &#39;game asset&#39;, &#39;--top-k&#39;, &#39;12&#39;]))<br># perform_search(arg_parser.parse_args([&#39;--text&#39;, &#39;fantasy sword with blue gemstone&#39;, &#39;--image&#39;, &#39;./examples/sword_reference.png&#39;, &#39;--top-k&#39;, &#39;12&#39;]))</pre><p>This is where the power of combining vector search with structured filters really shines. Searching for “female” might return all sorts of female figures, but adding [llm](https://zilliz.com/glossary/large-language-models-(llms)?utm_campaign=mediumkoc)_use_case == &quot;game asset&quot; immediately narrows it down to relevant characters, ignoring marketing mascots or e-commerce models. This is what makes it a usable catalog, not just a pile of vectors.</p><h3>Production Considerations</h3><p>When you’re building something like this, you always have to think about how it will run in production.</p><ul><li><strong>Latency:</strong> For interactive search, query latency is critical. The embedding model API call and the vector database search both contribute. Caching embeddings helps for ingestion, but query-time embeddings need to be fast. Choosing a low-latency embedding model and optimizing your <a href="https://zilliz.com/learn/vector-index?utm_campaign=mediumkoc">vector index</a> (e.g., HNSW with appropriate parameters) are key.</li><li><strong>Scalability:</strong> As the asset library grows (and with AI generation, it grows fast), your vector database needs to scale. Zilliz Cloud handles this automatically, but understanding your expected query load and data volume helps in choosing the right tier.</li><li><strong>Data Freshness:</strong> How quickly do new assets need to be searchable? Our current setup is near real-time, as soon as an asset is generated and its embedding is created. For some pipelines, a batch ingestion might be acceptable.</li><li><strong>Image Format:</strong> I mentioned using .webp for space efficiency. However, some embedding models prefer .png or .jpeg. If you see embedding errors, converting image formats before sending them to the API is a simple fix. This adds a small processing step but ensures reliability.</li><li><strong>Embedding Model Versioning:</strong> This is critical. Always version-lock your embedding model. If you re-index your entire dataset with a slightly different model version, your query vectors (from the old model) will no longer align perfectly, and your retrieval quality will tank. Document which model version was used to build each index.</li></ul><p>This project transformed how our creative team interacts with their generated assets. It moved them from a manual, folder-based nightmare to a dynamic, intelligent search system. The important shift is simple: generation creates the asset, but search makes the asset reusable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f64612067ca2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is MCP Dead? What We Learned Building with MCP, CLI, and Agent Skills]]></title>
            <link>https://medium.com/@PriyaSingh325/is-mcp-dead-what-we-learned-building-with-mcp-cli-and-agent-skills-d1120611938f?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1120611938f</guid>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Thu, 30 Jul 2026 10:40:11 GMT</pubDate>
            <atom:updated>2026-07-30T10:40:11.785Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bZtUnKJu48KBok9MYIO6Lg.png" /></figure><p>Last week I was reviewing a prototype AI agent integration using Model Context Protocol (MCP) and hit a wall I hadn’t anticipated. The agent’s context window was almost maxed out before it even made a move, choking on tool definitions that felt like dead weight. At the same time, I realized the agent couldn’t leverage its own <a href="https://zilliz.com/glossary/large-language-models-(llms)?utm_campaign=mediumkoc">LLM</a> to filter noisy search results inside the MCP server, forcing awkward workarounds. After digging into these limitations, I ended up rethinking the whole approach — shifting toward a combination of CLI tools and Agent Skills instead. Here’s what actually happened, what broke, and how we wired up a more practical architecture.</p><h3>MCP’s Context Window Problem: Why 72% Is Too Much</h3><p>MCP was supposed to be the universal standard for AI agents to discover and call external tools. The idea was simple: define tools as JSON schemas, load them all upfront, and let the agent call whichever it needs. But the reality in production quickly diverges from theory.</p><p>In one setup, I connected three MCP servers — GitHub, Playwright, and an IDE integration — to a 200K-token context window model. The tool definitions alone consumed roughly 143K tokens, or about 72% of the entire context window. The agent hadn’t even started reasoning yet. This upfront bloat isn’t just an inefficiency; it actively harms the agent’s focus.</p><p>The problem is what researchers call <strong>context rot</strong> — when an agent’s reasoning quality degrades because it’s overwhelmed by irrelevant context. Imagine the agent wading through a hundred tool schemas every time it needs to decide what to do next. In measured tests, tool selection accuracy dropped from 43% to below 14% as tool count increased. More tools paradoxically mean worse tool use.</p><p>This is baked into MCP’s architecture: it loads all tool descriptions fully at session start, no matter if they’ll be used or not. It’s a protocol-level design choice, not a bug. So every additional tool adds to the upfront token cost.</p><p>Contrast this with <strong>Agent Skills</strong>, which use <em>progressive disclosure</em>. At session start, the agent only loads lightweight metadata for each Skill — name, one-line description, trigger condition — a few dozen tokens total. The full Skill content only loads when relevant. Think of it as showing an index first, then fetching full chapters on demand.</p><p>CLI tools follow the same pattern. When you run git --help or docker --help, you discover commands and options on demand, not by preloading every flag upfront. Context cost becomes pay-as-you-go, not front-loaded.</p><p>At small scale, this difference is subtle. But at production scale, it’s the difference between an agent that works and one that drowns in its own tool definitions.</p><h3>MCP’s Passive Architecture and Workflow Limitations</h3><p>MCP’s design is clean but constrained. It’s a tool-calling protocol — how to discover tools, call them, and get results. But this simplicity comes with tradeoffs.</p><h3>Flat Tool Space, No Workflow Awareness</h3><p>MCP tools are flat function signatures with no hierarchy or session lifecycle awareness. They wait passively to be called. There’s no concept of subcommands, no multi-step workflows, no error handling baked in.</p><p>Compare that to CLI tools. Commands like git commit, git push, and git log are distinct execution paths under one interface. An agent can explore commands incrementally, expanding only what it needs without dumping all parameters into context.</p><h3>Skills Encode Workflow Logic</h3><p>Agent Skills are Markdown files encoding full standard operating procedures (SOPs): what to do first, next steps, failure handling, when to escalate to the user. Skills actively shape agent behavior during conversations, encoding triggers, preparation, and recovery.</p><p>MCP tools can’t do this. They’re passive and stateless — just waiting for calls. This limits the complexity of agent workflows you can build on MCP alone.</p><h3>The Killer Limitation: MCP Can’t Use the Agent’s Own LLM</h3><p>This was the dealbreaker for us.</p><p>We built an MCP plugin called claude-context to add <a href="https://zilliz.com/glossary/semantic-search?utm_campaign=mediumkoc">semantic search</a> to Claude Code and other AI coding agents. The goal: retrieve relevant historical conversation snippets from <a href="https://milvus.io/?utm_campaign=mediumkoc">Milvus</a> and surface them as context.</p><p>The <a href="https://zilliz.com/learn/vector-similarity-search?utm_campaign=mediumkoc">vector search</a> retrieval worked fine. But the problem was filtering results before passing them to the agent. We’d retrieve the top 10 results, but only 3 were actually useful; the other 7 were noise. Feeding all 10 to the outer agent distracted it, degrading answer quality.</p><p>We tried reranking inside the MCP server with a small model — accuracy was poor, and thresholds needed tuning per use case. Using a large model was technically sound but introduced a new problem: MCP servers run as separate processes with no access to the outer agent’s LLM. That meant managing a separate LLM client, a separate API key, and a separate call path.</p><p>What we really wanted was simple: let the agent’s own LLM participate directly in filtering. Retrieve the top 10, let the agent judge what’s relevant, and pass only those up. No second model, no extra API keys.</p><p>MCP couldn’t do this. The server-agent boundary is also an intelligence boundary. The server can’t use the agent’s LLM; the agent can’t govern what happens inside the server. This isolation is fine for simple CRUD tools but a real constraint when the tool needs judgment.</p><p>Agent Skills solve this elegantly. A retrieval Skill can call vector search for the top 10, then have the agent’s LLM assess relevance and return only the filtered results. The agent does the filtering itself, no extra models or keys needed.</p><h3>What We Built Instead: CLI and Skills as a Pattern</h3><p>After hitting these walls, we shifted toward a CLI + Skills architecture for agent-tool interaction. This isn’t just about memory retrieval — it’s a pattern for the whole stack.</p><h3>memsearch: Skills-Based Memory Layer</h3><p>We built <strong>memsearch</strong>, an open-source memory layer for Claude Code and other agents. It runs as a Skill inside a subagent with three stages:</p><ol><li><strong>Milvus vector search</strong> for broad initial retrieval.</li><li>The agent’s own LLM evaluates relevance and expands context for promising hits.</li><li>A final drill-down accesses original conversations only when needed.</li></ol><p>At each stage, noise is discarded so irrelevant data never clogs the primary context window.</p><p>The key insight: the agent’s intelligence is part of the tool execution. The LLM already in the loop filters results — no second model, no extra API keys, no brittle threshold tuning.</p><p>While memsearch focuses on conversation-context retrieval for coding agents, this architecture generalizes to any tool needing judgment, not just execution.</p><h3>Zilliz CLI, Skills, and Plugin for Vector DB Operations</h3><p>We’ve layered this pattern into our developer tools for Milvus and <a href="https://zilliz.com/cloud?utm_campaign=mediumkoc">Zilliz Cloud</a>:</p><ul><li><a href="https://zilliz.com/?utm_campaign=mediumkoc"><strong>Zilliz</strong></a><strong> CLI</strong> is the infrastructure layer for managing clusters, collections, vector search, RBAC, backups, billing — everything you’d do in the cloud console, but from the terminal. Both humans and agents use the same commands. This CLI also underpins Skills.</li><li><strong>Milvus Skill</strong> teaches AI coding agents (Claude Code, Codex, etc.) to operate Milvus deployments through Python code (pymilvus): connection, schema design, CRUD, hybrid and full-text search, <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a> pipelines.</li><li><strong>Zilliz Skill</strong> does the same for Zilliz Cloud, teaching agents to manage cloud infrastructure through the CLI.</li><li><strong>Zilliz Plugin</strong> wraps CLI + Skill into a guided experience inside Claude Code, with slash commands like /zilliz:quickstart and /zilliz:status.</li></ul><p>Here, CLI handles execution, Skills encode knowledge and workflow logic, and the plugin delivers the UX. Notably, no MCP server is involved.</p><h3>Is MCP Really Dead?</h3><p>There’s a lot of chatter about MCP being “dead,” but it’s more nuanced.</p><p>MCP has been donated to the Linux Foundation. There are over 10,000 active servers and 97 million monthly SDK downloads. An ecosystem that size doesn’t disappear overnight because of a conference comment.</p><p>What’s changed is the scope and use cases.</p><ul><li><strong>MCP over stdio</strong> — the local, process-isolated version — suffers from unstable IPC, environment isolation issues, and high token overhead. In that context, CLI + Skills is a better fit in almost every production scenario.</li><li><strong>MCP over HTTP</strong> still makes sense for enterprise internal tooling platforms needing centralized OAuth, role-based access control, unified telemetry, and audit logging. Fragmented CLI tools struggle to provide these consistently. MCP’s centralized architecture adds real value here.</li></ul><p>When Perplexity’s CTO said they were deprioritizing MCP internally, they primarily meant MCP over stdio. That nuance got lost in transmission, leading to broad anti-MCP sentiment.</p><p>MCP solved a real problem: before it, every AI app rolled its own tool-calling logic. MCP created a unified interface that accelerated ecosystem growth. Production experience has since surfaced limitations — that’s a natural lifecycle for infrastructure tooling, not a death sentence.</p><h3>Choosing Between MCP, CLI, and Skills</h3><p>Here’s a practical framework based on what we’ve learned:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KxXeKO0P2OROMhRzBUiQ4g.png" /></figure><ul><li><strong>Local development and agent tooling:</strong> CLI + Skills usually outperform MCP over stdio in latency, context efficiency, and workflow flexibility.</li><li><strong>Enterprise internal tooling:</strong> MCP over HTTP remains a strong choice for centralized security and telemetry.</li></ul><h3>A Code Snippet: Progressive Disclosure in an Agent Skill</h3><p>To illustrate how progressive disclosure works, here’s a simplified example of an Agent Skill metadata loader that fetches full content on demand:</p><pre>class AgentSkill:<br>    def __init__(self, metadata_url):<br>        self.metadata_url = metadata_url<br>        self.metadata = None<br>        self.full_content = None<br><br>    def load_metadata(self):<br>        # Load lightweight metadata only<br>        response = fetch(self.metadata_url + &quot;/metadata.json&quot;)<br>        self.metadata = response.json()<br><br>    def load_full_content(self):<br>        if self.full_content is None:<br>            response = fetch(self.metadata_url + &quot;/full_skill.md&quot;)<br>            self.full_content = response.text<br><br>    def get_trigger_condition(self):<br>        if self.metadata is None:<br>            self.load_metadata()<br>        return self.metadata.get(&quot;trigger_condition&quot;, &quot;&quot;)<br><br># Usage<br>skill = AgentSkill(&quot;https://skills.example.com/memsearch&quot;)<br>print(skill.get_trigger_condition())  # Loads only metadata<br># Later, when triggered<br>skill.load_full_content()  # Loads full Skill content on demand</pre><p>This pattern keeps the agent’s context light at startup and fetches detailed logic only when needed, avoiding MCP’s upfront token bloat.</p><h3>Production Considerations</h3><p>In production, these architectural choices impact:</p><ul><li><strong>Latency:</strong> MCP’s upfront loading adds delay before the agent can act, especially with many tools. CLI + Skills defer loading, improving responsiveness.</li><li><strong>Memory and Token Usage:</strong> Large upfront schemas consume precious context tokens, limiting downstream reasoning. Progressive disclosure preserves tokens for relevant info.</li><li><strong>Reliability:</strong> MCP over stdio suffers from unstable IPC and environment isolation issues, increasing failure rates. CLI + Skills run inside the agent’s process or call stable subprocesses.</li><li><strong>Security and Auditing:</strong> MCP over HTTP provides centralized OAuth and telemetry, valuable for enterprise compliance. CLI tools are more fragmented but can be wrapped with orchestration layers.</li></ul><h3>Conclusion</h3><p>Last week’s exploration made it clear: MCP isn’t dead, but its role is evolving. The upfront token cost, passive tool model, and intelligence boundary limit its usefulness in local agent tooling. CLI and Agent Skills offer a more scalable, flexible, and context-efficient alternative that leverages the agent’s own LLM for judgment calls.</p><p>For production RAG pipelines and AI coding assistants, this layered approach — CLI for execution, Skills for workflow logic, and plugins for UX — is what actually works in practice.</p><p>MCP over HTTP still holds value for centralized enterprise tooling platforms, but for local development and agent workflows, CLI + Skills is where I’d place my bets today.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1120611938f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When Basic RAG Is Not Enough: My Practical GraphRAG Checklist]]></title>
            <link>https://pub.aimind.so/when-basic-rag-is-not-enough-my-practical-graphrag-checklist-06b0b976dc48?source=rss-bb58ec7be9f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/06b0b976dc48</guid>
            <dc:creator><![CDATA[Priya Singh]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 15:32:00 GMT</pubDate>
            <atom:updated>2026-07-29T15:32:00.815Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CaW3w-Xi1Om_gI8-I5jqZw.png" /></figure><p>Last week I hit the same old RAG problem in a new wrapper.</p><p>The user asked a question that required connecting three facts across different documents. Each fact was present. None of the individual chunks looked wrong. But the answer still came back shallow because the retriever kept returning locally similar text instead of the pieces needed for the whole chain of reasoning.</p><p>That is the point where baseline <a href="https://zilliz.com/learn/Retrieval-Augmented-Generation?utm_campaign=mediumkoc">RAG</a> starts to feel less like a retrieval system and more like a very confident search box.</p><p>GraphRAG is interesting because it does not throw away retrieval. It adds a graph layer on top of it. Instead of only asking “which chunks are semantically close to this query?”, it also asks “which entities, relationships, communities, and summaries help explain this question?”</p><p>That difference matters when the question is not local.</p><h3>Why normal chunk retrieval breaks down</h3><p>A basic RAG pipeline usually looks like this:</p><p>1. Split documents into chunks.</p><p>2. Embed the chunks.</p><p>3. Store them in a <a href="https://zilliz.com/learn/what-is-vector-database?utm_campaign=mediumkoc">vector database</a>.</p><p>4. Retrieve the top-k closest chunks for a user question.</p><p>5. Ask the model to answer from those chunks.</p><p>That works well for questions where the answer sits inside one or two passages. It is much weaker for questions like:</p><p>• What are the main themes across this whole corpus?</p><p>• Which organizations are connected through the same events?</p><p>• What changed over time across multiple reports?</p><p>• Which person is related to the person who did X?</p><p>The issue is not always the embedding model. The issue is that semantic similarity is local. The nearest chunk may mention the right words without containing the missing relationship.</p><p>GraphRAG tries to make those relationships explicit.</p><h3>The index is doing more work</h3><p>The GraphRAG indexing flow has four pieces that I care about:</p><p>• Text unit segmentation</p><p>• Entity, relationship, and claim extraction</p><p>• Hierarchical clustering</p><p>• Community summary generation</p><p>The first step still looks familiar. You split the corpus into text units. The next step is where the pipeline changes. An LLM extracts entities, relationships, and claims from those text units. Then a community detection algorithm such as Leiden clusters the resulting graph. Finally, the system summarizes each community from the bottom up.</p><p>That gives the query side more than raw chunks. It can use entity descriptions, graph relationships, source text, and community summaries together.</p><p>One thing I learned the hard way: this is not free. Indexing can be expensive because the graph construction uses LLM calls over the corpus. For a first pass, I would always trim the input, cache the artifacts, and make the index build reproducible.</p><pre>from pathlib import Path<br><br>def prepare_graphrag_input(source: Path, target: Path, max_lines: int = 934) -&gt; None:<br>    &quot;&quot;&quot;Create a smaller GraphRAG input file for an initial indexing run.&quot;&quot;&quot;<br>    target.parent.mkdir(parents=True, exist_ok=True)<br>    lines = source.read_text(encoding=&quot;utf-8&quot;).splitlines()<br>    target.write_text(&quot;\n&quot;.join(lines[:max_lines]), encoding=&quot;utf-8&quot;)<br><br>prepare_graphrag_input(<br>    Path(&quot;raw/davinci.txt&quot;),<br>    Path(&quot;graphrag_index/input/davinci.txt&quot;),<br>)</pre><p>That tiny helper is not the GraphRAG algorithm. It is the production habit around it: control the input before you let an indexing job spend money.</p><h3>Local search and global search are different tools</h3><p>The source article explains two query paths: global search and local search.</p><p>Global search uses community summaries. I reach for this when the question asks for a high-level synthesis across the dataset. For example: “What are the top themes in these reports?” A normal top-k retriever might grab scattered chunks. Community summaries give the model a map of the corpus.</p><p>Local search starts from specific entities. It expands around nearby entities, relationships, text units, and reports. I reach for this when the user question has a named person, product, organization, or concept and needs grounded detail.</p><p>That distinction helped me frame GraphRAG better. It is not “RAG but with a graph” as a vague feature. It is two retrieval strategies:</p><p>• Use global search when the answer is about the corpus.</p><p>• Use local search when the answer is about connected entities.</p><h3>What I would store in Milvus</h3><p>In the implementation flow, the GraphRAG index produces parquet artifacts: entities, entity embeddings, relationships, community reports, text units, and related metadata. During query time, entity descriptions can be embedded and stored in Milvus for local search.</p><p>The important part is the key. If the vector store uses entity IDs, query code should use entity IDs. If it uses entity titles, query code should use titles. Mixing those up creates retrieval bugs that look like “the model is bad” but are actually key mismatches.</p><p>I would store at least this metadata with each entity <a href="https://zilliz.com/glossary/vector-embeddings?utm_campaign=mediumkoc">embedding</a>:</p><p>• Entity ID</p><p>• Entity title</p><p>• Community level</p><p>• Source text unit IDs</p><p>• Build version</p><p>That build version matters. If I rebuild the graph and the entity IDs shift, I want to know which query results came from which index.</p><h3>The biggest tradeoff</h3><p>GraphRAG gives you better structure for multi-hop reasoning and corpus-level synthesis. The cost is complexity.</p><p>You now have two systems to reason about:</p><p>• The graph index and its extracted relationships</p><p>• The vector retrieval layer used during search</p><p>If either side is stale, the answer can drift. If entity extraction is noisy, the graph can encode noise very confidently. If community summaries are too broad, global answers can sound polished while missing important minority details.</p><p>So I would not start every chatbot with GraphRAG. I would start with baseline RAG and add GraphRAG when evals show a specific failure mode:</p><p>• Multi-hop questions fail.</p><p>• Summary questions miss themes.</p><p>• Entity-heavy questions need relationship traversal.</p><p>• Users ask “how are these things connected?” more often than “find this paragraph.”</p><h3>My practical checklist</h3><p>Before shipping a GraphRAG-backed feature, I would check:</p><p>1. Can I rebuild the index from scratch?</p><p>2. Are the parquet artifacts stored with a version?</p><p>3. Do I have separate eval sets for local and global questions?</p><p>4. Is indexing cost capped for large corpora?</p><p>5. Can I inspect the entity context used for an answer?</p><p>The last point is the one I care about most. GraphRAG can make answers better, but it can also make debugging harder if the context builder becomes invisible.</p><p>The fix was simpler than I expected: treat GraphRAG as a retrieval architecture, not a magic reasoning upgrade. The graph gives the model better context. It does not remove the need for evals, cost controls, versioned indexes, or boring debugging tools.</p><h4>A Message from AI Mind</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*5Wm7sOfTpe5DEbhg.gif" /></figure><p>Thanks for being a part of our community! Before you go:</p><ul><li>👏 Clap for the story and follow the author 👉</li><li>📰 View more content in the <a href="https://pub.aimind.so/">AI Mind Publication</a></li><li>🧠 Improve your <a href="https://www.aimind.so/prompt-generator?utm_source=pub&amp;utm_medium=message">AI prompts effortlessly and FREE</a></li><li><strong>🧰 Discover </strong><a href="https://www.aimind.so/?utm_source=pub&amp;utm_medium=message"><strong>Intuitive AI Tools</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06b0b976dc48" width="1" height="1" alt=""><hr><p><a href="https://pub.aimind.so/when-basic-rag-is-not-enough-my-practical-graphrag-checklist-06b0b976dc48">When Basic RAG Is Not Enough: My Practical GraphRAG Checklist</a> was originally published in <a href="https://pub.aimind.so/">AI Mind</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>