<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Niranjan Akella on Medium]]></title>
        <description><![CDATA[Stories by Niranjan Akella on Medium]]></description>
        <link>https://medium.com/@niranjanakella?source=rss-4d80f7255174------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QZGlZIxoWqg7CpeKmwbIvQ.png</url>
            <title>Stories by Niranjan Akella on Medium</title>
            <link>https://medium.com/@niranjanakella?source=rss-4d80f7255174------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 02:37:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@niranjanakella/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Enabled Cognition | Agentic Autonomous Humanoids| Embedded Edge Qdrant VectorDB]]></title>
            <link>https://pub.towardsai.net/enabled-cognition-agentic-autonomous-humanoids-embedded-edge-qdrant-vectordb-6d4bce397b62?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/6d4bce397b62</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Tue, 01 Sep 2026 14:01:04 GMT</pubDate>
            <atom:updated>2026-09-01T14:01:04.540Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dbM4zCz587zLub4u7d46Tw.png" /></figure><h3>Robots Are Only as Smart as Their Memory</h3><p>Okay so here’s what’s actually happening in robotics right now and it’s kind of wild. We’ve got humanoid robots that are walking, arm-swinging, task-doing robots, that are cloud dependent for their basic cognition. The robot sees a red cup, sends a JPEG to AWS, waits 800ms for a response, then decides to pick it up. By which point… the cup moved. Or the WiFi dropped. Or the API ratelimited. Game over.</p><p>This is genuinely broken. And here’s why it can’t work on a physics level, not just an engineering preference level.</p><p>“Reactive obstacle avoidance” the kind where the robot doesn’t walk into a wall or a toddler that needs to happen in under 50ms. That’s the hard budget. At a walking speed of 1 ms, 50ms is 5cm of travel without correction. A cloud round trip to us-east-1 costs you 20–50ms before any compute even starts. Add model inference (a GPT4 class VLM takes 500ms+ to first token), and you’re looking at 800ms to 2 seconds per decision. That’s literally impossible to build safe reactive behavior on. The math doesn’t work.</p><p>The latency problem is severe. The privacy problem is… honestly even more severe, just less obvious at first.</p><p>Think about what a humanoid robot in your home actually sees. It sees your medication on the counter. It sees your kid’s face. It sees your daily routine, your security vulnerabilities, what you bought, who you called, when you left for work. If all of that raw sensor data streams to a cloud server for processing every frame, every audio clip, every interaction you have built the most comprehensive domestic surveillance system in history. Inside your own house. That you paid for.</p><p>GDPR hates it. CCPA hates it. Your users will hate it the moment they find out. And honestly? They should.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/200/1*6oellcmr9XTsOf5e1qSIFw.gif" /></figure><p>So the question becomes: what if the robot just… carried its own brain? Full vector memory, local LLM reasoning, vision embeddings, speech processing and all that…… all running on a small computer strapped to its chassis. No WiFi needed. No privacy leak. No 800ms decision latency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*4nSlUWMKEW7dlDUhErfBKg.gif" /></figure><p>That’s what we’re building today. And it’s not theoretical. The hardware exists, the models are small enough, and <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is the vector memory layer that ties it all together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1O8Xo94PDBehl5GKgIhWDA.png" /><figcaption>Feat Image Diffusion</figcaption></figure><h3>LLM as Head, Qdrant as Hippocampus</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/412/0*91aKtIvtpmllzwvZ.gif" /></figure><p>Turns out the neuroscience metaphor here is actually useful, not just cute.</p><p>The human hippocampus handles two kinds of memory. Episodic memory is autobiographical and specific events with full context.</p><blockquote>“On Tuesday at 2pm I picked up a coffee mug from the kitchen counter, it was heavier than expected, and I set it on the wrong shelf.”</blockquote><p>Semantic memory is general knowledge and decontextualized facts. “Mugs are cylindrical containers. They’re usually in kitchens. Handle is on the right side.”</p><p>Different systems, different update rates, different query patterns.</p><p>That’s exactly what we need for a robot. And <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> maps to this structure naturally.</p><p>The cognitive loop looks like this:</p><p>Each memory point in Qdrant carries up to three named vectors with one per modality. The same event, captured from every sensory angle:</p><pre>from qdrant_client import QdrantClient, models<br>client = QdrantClient(&quot;:memory:&quot;)  # or path=&quot;/opt/robot/memory&quot; for persistence<br>client.create_collection(<br>    collection_name=&quot;robot_memory&quot;,<br>    vectors_config={<br>        &quot;vision&quot;: models.VectorParams(size=512,  distance=models.Distance.COSINE),<br>        &quot;audio&quot;:  models.VectorParams(size=384,  distance=models.Distance.COSINE),<br>        &quot;text&quot;:   models.VectorParams(size=768,  distance=models.Distance.COSINE),<br>    }<br>)</pre><p>That’s it. One collection. Three vector spaces. The vision vector is a CLIP ViT-B/32 embedding of the camera frame (512D). The audio vector is an embedding of whatever Whisper heard (384D). The text vector is a sentence-transformer encoding of a natural language description of the event (768-dim). Same point_id, same payload with timestamp and location and task context, three different ways of encoding the same memory.</p><p>Now here’s where it gets interesting. The grounding loop is the thing that connects language to physical action and it works like this:</p><p>Robot hears “pick up the red cup.”</p><p>That text gets embedded. Qdrant searches the text vector space for similar past experiences. It finds 3 memories where the robot previously encountered cups, objects on counters, pick-and-place tasks. Those memories, including what worked and what didn&#39;t, what the visual context looked like - get handed to the local LLM as context. The LLM reasons with actual experience, not just training data. And then it acts.</p><p>This is RAG for robots. but the “documents” are the robot’s own lived experience and the “generation” is a motor plan.</p><h3>Now come the SENSES! Wiring Vision, Audio, and Speech Locally 😎</h3><p>Let’s get concrete about what each pipeline looks like on the hardware.</p><p><strong>Vision:</strong> Camera frame hits OpenCV. You run it through CLIP ViT-B/32 in PyTorch with torch.no_grad() and L2-normalize the output. You get a 512-dimensional vector in about 18ms on a Jetson Orin GPU. That vector goes straight into Qdrant&#39;s vision space.</p><pre>import clip, torch, cv2<br>from PIL import Image<br>device = &quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;<br>model, preprocess = clip.load(&quot;ViT-B/32&quot;, device=device)<br>model.eval()<br>def embed_frame(frame):  # OpenCV BGR frame<br>    image = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))<br>    inp = preprocess(image).unsqueeze(0).to(device)<br>    with torch.no_grad():<br>        emb = model.encode_image(inp)<br>        emb = emb / emb.norm(dim=-1, keepdim=True)<br>    return emb.cpu().numpy().squeeze().tolist()  # 512 floats</pre><p>18ms per frame. At 30Hz you obviously don’t store every frame, sample at 1Hz for background memory accumulation, or trigger writes on detected state changes.</p><p><strong>Audio:</strong> Microphone stream feeds into Silero VAD (Voice Activity Detection) first then a tiny 1.8MB model that runs in under 1ms per 32ms chunk and tells you exactly when speech starts and stops. Once VAD says “speech detected,” you buffer audio and send it through faster-whisper. The whole thing takes about 80ms for a 3second voice clip. Then embed the transcript with a sentence-transformer and write to the audio vector space.</p><p><strong>Fuzzy command matching</strong> is where voice interfaces usually fall apart. If you do exact string matching on ASR output you’re going to have a bad time. ASR is noisy. People have accents. They paraphrase. “Kill the lights” and “turn off the lights” mean exactly the same thing but won’t match on string comparison.</p><p>Semantic search doesn’t have this problem:</p><pre># Instead of: if command == &quot;turn off the lights&quot; (fragile, breaks constantly)<br># Do this:<br>results = client.query_points(<br>    collection_name=&quot;robot_memory&quot;,<br>    query=embed_text(spoken_command),<br>    using=&quot;text&quot;,<br>    limit=1,<br>    score_threshold=0.75<br>)<br>best_match = results.points[0]  # &quot;kill the lights&quot; still matches &quot;turn off lights&quot;</pre><p>The embedding of “kill the lights” and “turn off the lights” are close in semantic space. Score over 0.85: execute confidently. Score 0.75–0.85: ask for confirmation. Score below 0.75: “sorry, can you repeat that?” Clean, robust, handles accents and paraphrases automatically.</p><blockquote><strong>The real power of this semantics kick in when the agent running locally needs to pull right parts of history &amp; stats from online into context for better function calling or script coding/execution</strong></blockquote><p><strong>Scene understanding with LLaVA:</strong> Sometimes the robot needs richer reasoning than CLIP gives you. When you need it to describe what it’s seeing rather than just embed it, route the frame through LLaVA via Ollama:</p><pre>import ollama, base64, cv2<br>def describe_scene(frame, question=&quot;What objects do you see? Are any relevant to the current task?&quot;):<br>    _, buf = cv2.imencode(&quot;.jpg&quot;, frame, [cv2.IMWRITE_JPEG_QUALITY, 85])<br>    img_b64 = base64.b64encode(buf.tobytes()).decode()<br>    response = ollama.chat(<br>        model=&quot;llava:7b&quot;,<br>        messages=[{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: question, &quot;images&quot;: [img_b64]}]<br>    )<br>    return response[&quot;message&quot;][&quot;content&quot;]</pre><p>Use that natural language description to build a richer text vector for the memory point. Now future queries can match on meaning, not just visual pixel similarity.</p><p><strong>Speech output</strong> closes the loop. Piper TTS takes the LLM’s response text and synthesizes audio in about 30ms for a short sentence. No API. No network. The robot just… talks. Locally.</p><h3>Memory That Grows With Experience</h3><p>Every time something interesting happens, it gets written to Qdrant with full multimodal context and a rich payload:</p><pre>import uuid<br>from datetime import datetime, timezone<br>client.upsert(<br>    collection_name=&quot;robot_memory&quot;,<br>    points=[<br>        models.PointStruct(<br>            id=str(uuid.uuid4()),<br>            vector={<br>                &quot;vision&quot;: clip_embedding,   # what it looked like<br>                &quot;audio&quot;:  audio_embedding,  # what it sounded like<br>                &quot;text&quot;:   text_embedding,   # natural language description<br>            },<br>            payload={<br>                &quot;timestamp&quot;:       datetime.now(timezone.utc).isoformat(),<br>                &quot;location&quot;:        {&quot;room&quot;: &quot;kitchen&quot;, &quot;x&quot;: 1.2, &quot;y&quot;: 0.5},<br>                &quot;task&quot;:            &quot;pick_cup&quot;,<br>                &quot;outcome&quot;:         &quot;success&quot;,<br>                &quot;confidence&quot;:      0.92,<br>                &quot;scene_description&quot;: &quot;Red ceramic mug on wooden counter, handle facing right&quot;,<br>            }<br>        )<br>    ]<br>)</pre><p>Before the robot attempts any task, it queries Qdrant for similar past situations. “I’m about to reach for something on a shelf what do I know about shelf interactions?” The retrieved memories become context for the LLM’s reasoning step. The robot isn’t starting from scratch every time it powers on. It’s actually learning from experience.</p><p>The forgetting problem is real though. A robot storing memories at 1Hz accumulates 86,400 points per day. After a year that’s 31 million points. You need a retention strategy. Importance-weighted retention tracks how often each memory gets retrieved and points that come up frequently are important, keep them. Points that were never retrieved in 30 days are probably routine, delete them:</p><pre># Delete memories from 30+ days ago that were never retrieved<br>client.delete(<br>    collection_name=&quot;robot_memory&quot;,<br>    points_selector=models.FilterSelector(<br>        filter=models.Filter(must=[<br>            models.FieldCondition(key=&quot;last_retrieved&quot;, range=models.DatetimeRange(lt=thirty_days_ago)),<br>            models.FieldCondition(key=&quot;retrieval_count&quot;, range=models.Range(lt=2)),<br>        ])<br>    )<br>)</pre><p><strong>Cross-modal recall</strong> is where it gets genuinely cool. The robot hears a sound it doesn’t recognize. It embeds that audio and queries the audio vector space. Finds past memories where it heard similar sounds. Then checks what the vision embeddings looked like at those moments - what was the robot seeing the last time it heard this?</p><pre># Heard something unfamiliar - what was I seeing in similar past moments?<br>audio_matches = client.query_points(<br>    collection_name=&quot;robot_memory&quot;,<br>    query=current_audio_embedding,<br>    using=&quot;audio&quot;,<br>    limit=5,<br>    with_payload=True,<br>)<br>for match in audio_matches.points:<br>    print(f&quot;Similarity: {match.score:.2f} | Visual context: {match.payload.get(&#39;scene_description&#39;)}&quot;)<br>    print(f&quot;Location: {match.payload.get(&#39;location&#39;)} | Outcome: {match.payload.get(&#39;outcome&#39;)}&quot;)</pre><p>Multi-modal memory association. The robot connects what it hears to what it’s seen in similar situations. That’s not a lookup table. That’s something closer to actual memory.</p><h3>The Full Stack on a Jetson Orin (FUN PART ! 🤩)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/0*sh3y6t_PJefTwfaZ.gif" /></figure><p>The hardware: NVIDIA Jetson Orin NX 16GB. About $499 for the module, ~$750 for the dev kit with carrier board. Critically the unified memory architecture, meaning CPU and GPU share the same 16GB pool. No PCIe bandwidth tax when moving data between CPU and GPU memory. This matters a lot for a stack that constantly shuffles tensors around.</p><p>All you would need is a Gemma kinda model that’s QAT (Quantization Aware Trained) to match better function calling + scription + context awareness with say something like Circular Attention or Linear Scaled Attention for large contexts like 1M tokens.</p><p>The latency comparison is pretty stark:</p><p><strong>Cloud-dependent robot pipeline:</strong> hear command (10ms) + network (30ms) + cloud STT (400ms) + network (30ms) + cloud LLM (800ms) + network (30ms) + cloud TTS (300ms) = <strong>~1.6 to 2.5 seconds</strong></p><p><strong>Local stack:</strong> hear command (10ms) + Whisper (80ms) + embed + Qdrant search (5ms) + Ollama first token (120ms) + Piper TTS (30ms) = <strong>~250ms</strong></p><p>That’s about 6–10x faster. But the numbers actually undersell the real win. The cloud pipeline has variance with sometimes 800ms, sometimes 2.5 seconds depending on API load. The local pipeline is predictable. Every time. No cold starts, no rate limits, no “degraded service” alerts at 3pm on a Tuesday.</p><p>For reactive behavior &amp; obstacle avoidance, balance correction, collision response, the cloud isn’t even in the conversation. Sub100ms is the budget.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*hdomNE-ONDxyAeGyBBh0_g.gif" /></figure><p>A CLIP embedding + Qdrant lookup + rule check takes 25–60ms on Jetson. A cloud round-trip alone is 50–200ms minimum. The local stack wins by physics.</p><p>On the vector DB options at the edge: Chroma runs in-process which is convenient, but has no quantization and no multi-vector support per point. Milvus Lite works but runs heavier. <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is the only one with Turbo4 4bit vector compression…..which means you can fit 500K vectors in ~1GB instead of ~4GB. That compression is the difference between this working on 16GB hardware and not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*1yCjTTgAWaZRVRXr.gif" /></figure><h3>Run It Yourself Guys! A Local Robot Brain in ~60 Lines</h3><p>Here’s how to spin up the infrastructure:</p><pre># Qdrant (ARM64 compatible - runs natively on Jetson and M-series Mac)<br>docker run -d -p 6333:6333 \<br>  -v $(pwd)/qdrant_data:/qdrant/storage \<br>  qdrant/qdrant<br><br># Ollama<br>docker run -d --gpus all -p 11434:11434 \<br>  -v ollama_models:/root/.ollama \<br>  ollama/ollama<br><br># Pull the model (one-time, ~2GB download)<br>docker exec -it $(docker ps -qf &quot;ancestor=ollama/ollama&quot;) ollama pull llama3.2:3b</pre><p>Now the full demo. Runs completely locally, no API keys, no cloud:</p><pre>&quot;&quot;&quot;<br>Local robot brain demo: multimodal memory + LLM reasoning.<br>Requirements: pip install qdrant-client ollama sentence-transformers numpy<br>&quot;&quot;&quot;<br>import uuid<br>import numpy as np<br>import ollama<br>from qdrant_client import QdrantClient, models<br>from sentence_transformers import SentenceTransformer<br># --- Setup ---<br>client = QdrantClient(&quot;:memory:&quot;)  # swap to path=&quot;/opt/robot/memory&quot; for persistence<br>text_model = SentenceTransformer(&quot;all-MiniLM-L6-v2&quot;)  # 384-dim, fast, small (~90MB)<br>client.create_collection(<br>    collection_name=&quot;robot_memory&quot;,<br>    vectors_config={<br>        &quot;vision&quot;: models.VectorParams(size=512, distance=models.Distance.COSINE),<br>        &quot;text&quot;:   models.VectorParams(size=384, distance=models.Distance.COSINE),<br>    }<br>)<br>def embed_text(s):<br>    return text_model.encode(s).tolist()<br>def fake_vision_embed(description):<br>    # Simulates a CLIP embedding - replace with real clip.encode_image() on hardware<br>    rng = np.random.default_rng(hash(description) % (2**32))<br>    vec = rng.standard_normal(512).astype(np.float32)<br>    return (vec / np.linalg.norm(vec)).tolist()<br># --- Seed 5 past experiences ---<br>memories = [<br>    (&quot;coffee mug on kitchen counter, ceramic, red, handle right&quot;,    &quot;kitchen&quot;, &quot;pick_object&quot;, &quot;success&quot;),<br>    (&quot;wooden chair blocking hallway, brown, four legs&quot;,               &quot;hallway&quot;, &quot;navigate&quot;,    &quot;failure&quot;),<br>    (&quot;blue water bottle on desk, plastic, cap on&quot;,                    &quot;office&quot;,  &quot;pick_object&quot;, &quot;success&quot;),<br>    (&quot;person standing near doorway, arms crossed, casual clothes&quot;,    &quot;hallway&quot;, &quot;avoid_human&quot;, &quot;success&quot;),<br>    (&quot;small white bowl on kitchen table, ceramic, empty&quot;,             &quot;kitchen&quot;, &quot;pick_object&quot;, &quot;success&quot;),<br>]<br>print(&quot;Seeding robot memory...&quot;)<br>for desc, room, task, outcome in memories:<br>    client.upsert(<br>        collection_name=&quot;robot_memory&quot;,<br>        points=[models.PointStruct(<br>            id=str(uuid.uuid4()),<br>            vector={&quot;vision&quot;: fake_vision_embed(desc), &quot;text&quot;: embed_text(desc)},<br>            payload={&quot;description&quot;: desc, &quot;room&quot;: room, &quot;task&quot;: task, &quot;outcome&quot;: outcome},<br>        )]<br>    )<br># --- Simulate incoming voice command ---<br>voice_command = &quot;grab the mug from the counter&quot;  # swap with Whisper output on real hardware<br>print(f&quot;\nVoice command: &#39;{voice_command}&#39;&quot;)<br># --- Retrieve relevant past experiences ---<br>results = client.query_points(<br>    collection_name=&quot;robot_memory&quot;,<br>    query=embed_text(voice_command),<br>    using=&quot;text&quot;,<br>    limit=3,<br>    with_payload=True,<br>    score_threshold=0.3,<br>)<br>context_lines = [<br>    f&quot;- {r.payload[&#39;description&#39;]} | task={r.payload[&#39;task&#39;]} | outcome={r.payload[&#39;outcome&#39;]}&quot;<br>    for r in results.points<br>]<br>context = &quot;\n&quot;.join(context_lines) if context_lines else &quot;No relevant past experience found.&quot;<br>print(f&quot;\nRetrieved memories:\n{context}&quot;)<br># --- Ask the local LLM to reason with that context ---<br>prompt = f&quot;&quot;&quot;You are a robot assistant. Use past experience to plan your next action.<br>Past relevant experience:<br>{context}<br>Current command: &quot;{voice_command}&quot;<br>Respond with ONE short action plan (2 sentences max).&quot;&quot;&quot;<br>print(&quot;\nThinking (local LLM, no cloud)...&quot;)<br>response = ollama.generate(model=&quot;llama3.2:3b&quot;, prompt=prompt, options={&quot;temperature&quot;: 0.1})<br>robot_speech = response[&quot;response&quot;].strip()<br>print(f&quot;\nRobot says: {robot_speech}&quot;)</pre><p>Run it and you’ll see something like (Hopefully 🤣):</p><pre>Seeding robot memory...<br>Voice command: &#39;grab the mug from the counter&#39;<br>Retrieved memories:<br>- coffee mug on kitchen counter, ceramic, red, handle right | task=pick_object | outcome=success<br>- small white bowl on kitchen table, ceramic, empty | task=pick_object | outcome=success<br>Thinking (local LLM, no cloud)...<br>Robot says: Based on past experience, approach the kitchen counter and locate the red ceramic<br>mug. Grip the handle from the right side for a successful pick.</pre><p>Yup! The robot used its own memories to inform the action plan. No API key. No latency spike. No data leaving the machine.</p><p><strong>Want to try this on a Raspberry Pi 5 (8GB)?</strong> The Qdrant + sentence-transformer text-only version runs fine on Pi. Just skip the vision embedding and run text-only queries. Ollama will be slow (~3–5 tokens/sec on Pi 5 CPU) but it works offline. For realtime vision and reasonable LLM speeds you need the Jetson.</p><p>The <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant docs</a> cover quantization options for when you push this to production — INT8 scalar quantization cuts vector memory by 4x, letting you store 2 million memories in the same 1GB budget.</p><p><strong>Where does this go next?</strong></p><p>The Unitree G1 starts at $13.5K now a full humanoid, Jetson Orin onboard, 4-mic array, depth cameras, full SDK access. Figure’s Helix VLA model runs at 200Hz on embedded hardware. HuggingFace’s LeRobot just added G1 as a native hardware target. The pieces are all here.</p><p>The gap isn’t hardware and it isn’t models. It’s the memory layer and the thing that makes a robot accumulate experience instead of starting from scratch every power cycle. Episodic memory. Semantic memory. Cross-modal recall. Forgetting curves. All of it maps directly onto what Qdrant already does.</p><p>We’re building robot brains from off the shelf components. A $500 compute module, open-source models, a vector database that fits in 1GB of RAM. The question isn’t “can we do this” but it’s “how fast can we wire it up.”</p><p>Pretty wild right?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*mLSDYAEyF6L5aBEvqADUAg.gif" /><figcaption>Smoother Right?</figcaption></figure><p><em>Want to stay up to date on what’s happening at the intersection of vector search and AI? Subscribe to the </em><a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella"><em>Qdrant newsletter</em></a><em> and get practical tutorials and project deep-dives. And if you want to run Qdrant at scale without managing infrastructure, </em><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella"><em>Qdrant Cloud</em></a><em> has a generous free tier to start.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6d4bce397b62" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/enabled-cognition-agentic-autonomous-humanoids-embedded-edge-qdrant-vectordb-6d4bce397b62">Enabled Cognition | Agentic Autonomous Humanoids| Embedded Edge Qdrant VectorDB</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why You Probably Shouldn’t Self-Host Qdrant (An Honest Look)]]></title>
            <link>https://pub.towardsai.net/why-you-probably-shouldnt-self-host-qdrant-an-honest-look-e2f2bdce44ca?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/e2f2bdce44ca</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Sat, 29 Aug 2026 14:31:01 GMT</pubDate>
            <atom:updated>2026-09-01T04:58:03.659Z</atom:updated>
            <content:encoded><![CDATA[<p>So you’re building something with vector search. You found <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a>, it’s fast, it’s open source, and you’re thinking “I’ll just self-host this, how hard can it be?” well….</p><p>So I’m gonna show you waht it actually takes to handle a successful self-hosted Qdrant DB at scale ===&gt; And you shall decide where you wish to take that route &amp; whether your team can handle it. Simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BvXEVbnd8uUr14EnuAR2pA.png" /></figure><p>Hmm..? Yeah. I’ve been there.</p><p>This post isn’t here to scare you off self-hosting entirely. Sometimes it genuinely is the right call. But there’s a gap between “runs fine in Docker on my laptop” and “production-grade on Monday,” and that gap is filled with mmap limits, WAL management, 2am alerts about yellow collections, and a Prometheus stack you now have to maintain forever. Let’s talk about all of it honestly.</p><h3>Section 1: The Honest Case for Self-Hosting (and Where It Breaks Down)</h3><p>The appeal is real. Self-hosting Qdrant means no recurring cloud bill, full control over your data, no vendor dependency, and complete flexibility on where and how your vectors live. For teams with strict data sovereignty requirements or regulated industries, it might not even be a choice… it’s a hard mandate.</p><p>And honestly? For the right team, self-hosting works great. If you have a platform engineering crew that already operates Kubernetes at scale, the marginal cost of running a Qdrant StatefulSet is pretty low. If you’re at a company with massive compute already committed, running Qdrant on spare capacity makes economic sense. If you’re in healthcare or finance and your compliance team won’t let production data touch any SaaS control plane, self-hosting is your starting point.</p><p>But here’s the thing… most teams aren’t in those situations.</p><p>What actually happens more often: a developer gets Qdrant running locally, it works beautifully, they push it to a VPS or a Kubernetes cluster before launch, and everything is fine for a few weeks. Then the first incident happens. Maybe disk fills up during a large batch insert. Maybe a node goes yellow and nobody knows why. Maybe the team lead who set it up leaves, and the “Qdrant owner” role quietly lands on whoever happened to be watching the on-call rotation that week.</p><p>The hidden costs don’t show up until month 2 or 3:</p><ul><li>Someone has to track Qdrant releases (they ship fast | every 4–6 weeks) and plan upgrades</li><li>Someone has to build and maintain the Prometheus/Grafana monitoring stack</li><li>Someone has to write and test the backup automation, verify it works, and handle retention</li><li>Someone is the person who gets paged at 2am when a collection goes yellow or WAL fills disk</li></ul><p>That “someone” cost is real, and it compounds. So before you commit, be honest about who that someone is on your team, and whether they have the bandwidth.</p><p><strong>Self-hosting makes sense if you are:</strong></p><ul><li>A team with existing infra engineers who already run K8s at scale</li><li>A regulated industry that genuinely cannot use any external data plane</li><li>Operating at a scale where cloud pricing is material and you have the infra maturity to manage it</li></ul><p><strong>Self-hosting probably doesn’t make sense if you are:</strong></p><ul><li>A startup or small team without dedicated infra capacity</li><li>Moving fast and treating the database as infrastructure that you don’t want to actually think about right?</li><li>Anyone who would rather ship features than debug mmap exhaustion at 2am</li></ul><p>Fair enough? Okay,but let’s say you’re going self-hosted anyway. Here’s what you actually need to sort out.</p><h3>Section 2: The Ops Checklist Nobody Tells You About</h3><p>Right, so you’ve got Qdrant running. Cool. But “running” and “production-ready” are different things. Here’s what you now own.</p><p><strong>OS-level tuning (this is not optional)</strong></p><p>Qdrant opens one mmap file handle per segment per vector store. A collection with 50 segments and 3 vector fields holds 150+ file descriptors. With multiple collections, you’ll blow past Linux defaults (1024 soft limit) without any warning. You just start getting Too many open files errors. Fix it:</p><pre># /etc/security/limits.conf<br>qdrant soft nofile 65535<br>qdrant hard nofile 65535<br><br># Or in your systemd unit:<br>LimitNOFILE=65535<br>LimitMEMLOCK=infinity</pre><p>And the mmap limit. Qdrant uses mmap() for all vector storage. Linux defaults to vm.max_map_count=65530. A production cluster with hundreds of segments exhausts this silently - new mmap calls fail with ENOMEM even when physical memory is available. Same requirement as Elasticsearch, by the way:</p><pre>sysctl -w vm.max_map_count=262144<br>echo &quot;vm.max_map_count=262144&quot; &gt;&gt; /etc/sysctl.conf</pre><p>Also worth setting:</p><pre># Prevent swap from wrecking latency<br>vm.swappiness = 10<br><br># Tune dirty page writeback to reduce write stalls during batch ingestion<br>vm.dirty_ratio = 15<br>vm.dirty_background_ratio = 5</pre><p><strong>Memory sizing is not obvious</strong></p><p>Qdrant has three memory placement modes: pinned (locked in RAM), cached (default - pre-loads vectors into OS page cache on startup), and cold (on-demand from disk). The default cached mode means if you have 10GB of vectors and 8GB of RAM, Qdrant appears to start fine, then degrades under memory pressure as the OS starts evicting pages. No warning. Just latency that gets worse over time.</p><p>The formula for a rough estimate: (vector_dim × 4 bytes × num_vectors) + HNSW graph (~vector_count × m × 8 bytes) + payload indexes + WAL + process overhead. For 1M OpenAI 1536-dim vectors: roughly 6-7.5GB minimum. Add 25% headroom because Qdrant&#39;s own metrics underreport memory by 10-15% due to allocator overhead. Size accordingly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*dffwK7KBMW8xgzbf.gif" /></figure><p><strong>The “collection goes yellow” incident</strong></p><p>So yeah, this one’s a rite of passage. A collection’s status goes yellow when one or more replicas are in a non-active state. grey means active optimization. green is what you want. At 2am, you&#39;ll be staring at yellow asking what happened.</p><p>Common causes: a node went down, a shard transfer failed, a replica diverged because one node ran out of disk during a write, or WAL replay after a restart failed. You debug it like this:</p><pre># Check cluster state<br>GET /cluster<br><br># Check which shards are where and their states<br>GET /collections/{name}/cluster</pre><p>If a node is permanently gone, you manually remove it from consensus and trigger shard replication yourself:</p><pre>DELETE /cluster/peer/{peer_id}?force=true<br><br>POST /collections/{name}/cluster<br>{<br>  &quot;replicate_shard&quot;: {<br>    &quot;shard_id&quot;: 0,<br>    &quot;from_peer_id&quot;: &lt;live_peer&gt;,<br>    &quot;to_peer_id&quot;: &lt;new_peer&gt;<br>  }<br>}</pre><p>Note: Qdrant in self-hosted mode does <strong>not</strong> automatically rebalance shards when a new node comes online. You do this manually. Every time.</p><p><strong>Monitoring: the stack you’ll build</strong></p><p>You need Prometheus scraping each node individually and not through a load balancer, because /metrics only reports a single peer&#39;s state. Key alerts to set up:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cy2cGp-oqOjG5OCYgzsRhg.png" /></figure><p>The /healthz endpoint returns 200 as soon as the process is alive. Use /readyz for Kubernetes readiness - it only returns 200 when fully loaded. This matters a lot: after a restart on a large collection, WAL replay can take 10-30 minutes, and if your readiness probe has a short failure threshold, Kubernetes will restart the pod before recovery completes. Crash loop. Fun times.</p><h3>Section 3: Backup, Retention, and “Oh No” Recovery</h3><p>Okay so let’s talk about what happens when things go wrong and you don’t have a backup strategy. Spoiler: you lose data. Not in a “probably fine” way. In a “the collection is gone” way.</p><p>Qdrant’s backup mechanism is snapshots. A snapshot is a point-in-time tar archive of a collection.. all segments, indexes, and config. The API is simple:</p><pre>POST /collections/{name}/snapshots</pre><p>The gotcha is that snapshots don’t cover in-flight WAL entries. If you’re under heavy write load, writes buffered in the WAL but not yet flushed to segments (up to flush_interval_sec=5 seconds worth) aren&#39;t captured. Under normal conditions this is seconds of data. Under heavy batch ingestion it could be more. That&#39;s your RPO floor.</p><p><strong>The complete backup script</strong></p><p>This handles snapshot creation, download, S3 upload, server-side cleanup, and local retention cleanup. Drop it in cron and forget about it:</p><pre>#!/usr/bin/env python3<br>&quot;&quot;&quot;qdrant_backup.py - Automated snapshot backup with S3 upload and retention.&quot;&quot;&quot;<br><br>import os, sys, time, logging, datetime, pathlib, subprocess, urllib.request<br>logging.basicConfig(<br>    level=logging.INFO,<br>    format=&quot;%(asctime)s %(levelname)s %(message)s&quot;,<br>    handlers=[logging.StreamHandler(), logging.FileHandler(&quot;/var/log/qdrant-backup.log&quot;)]<br>)<br>log = logging.getLogger(__name__)<br>try:<br>    from qdrant_client import QdrantClient<br>except ImportError:<br>    sys.exit(&quot;pip install qdrant-client required&quot;)<br>QDRANT_URL  = os.environ.get(&quot;QDRANT_URL&quot;, &quot;http://localhost:6333&quot;)<br>QDRANT_KEY  = os.environ.get(&quot;QDRANT_API_KEY&quot;, &quot;&quot;)<br>BACKUP_DIR  = pathlib.Path(os.environ.get(&quot;BACKUP_DIR&quot;, &quot;/backups/qdrant&quot;))<br>RETAIN_DAYS = int(os.environ.get(&quot;RETAIN_DAYS&quot;, &quot;7&quot;))<br>S3_BUCKET   = os.environ.get(&quot;S3_BUCKET&quot;, &quot;&quot;)<br>def get_client():<br>    kwargs = {&quot;url&quot;: QDRANT_URL, &quot;timeout&quot;: 600}<br>    if QDRANT_KEY:<br>        kwargs[&quot;api_key&quot;] = QDRANT_KEY<br>    return QdrantClient(**kwargs)<br>def backup_all(client, timestamp):<br>    for col in client.get_collections().collections:<br>        name = col.name<br>        dest = BACKUP_DIR / name<br>        dest.mkdir(parents=True, exist_ok=True)<br>        log.info(f&quot;Snapshotting: {name}&quot;)<br>        try:<br>            snap = client.create_snapshot(collection_name=name)<br>        except Exception as e:<br>            log.error(f&quot;Snapshot failed for {name}: {e}&quot;); continue<br>        out_file = dest / f&quot;{timestamp}_{snap.name}&quot;<br>        url = f&quot;{QDRANT_URL}/collections/{name}/snapshots/{snap.name}&quot;<br>        req = urllib.request.Request(url)<br>        if QDRANT_KEY:<br>            req.add_header(&quot;api-key&quot;, QDRANT_KEY)<br>        try:<br>            with urllib.request.urlopen(req, timeout=3600) as r, open(out_file, &quot;wb&quot;) as f:<br>                while chunk := r.read(8 * 1024 * 1024):<br>                    f.write(chunk)<br>        except Exception as e:<br>            log.error(f&quot;Download failed: {e}&quot;); continue<br>        log.info(f&quot;Saved {out_file.stat().st_size / 1e6:.1f} MB to {out_file}&quot;)<br>        # Clean up server-side snapshot to free disk space<br>        try:<br>            client.delete_snapshot(collection_name=name, snapshot_name=snap.name)<br>        except Exception as e:<br>            log.warning(f&quot;Could not delete server snapshot: {e}&quot;)<br>        # Upload to S3 if configured<br>        if S3_BUCKET:<br>            key = f&quot;qdrant-snapshots/{name}/{timestamp}_{snap.name}&quot;<br>            subprocess.run(<br>                [&quot;aws&quot;, &quot;s3&quot;, &quot;cp&quot;, str(out_file), f&quot;s3://{S3_BUCKET}/{key}&quot;,<br>                 &quot;--storage-class&quot;, &quot;STANDARD_IA&quot;],<br>                check=True, timeout=7200<br>            )<br>            log.info(f&quot;Uploaded to s3://{S3_BUCKET}/{key}&quot;)<br>def apply_retention(days):<br>    cutoff = datetime.datetime.now() - datetime.timedelta(days=days)<br>    deleted = sum(<br>        1 for f in BACKUP_DIR.rglob(&quot;*.snapshot&quot;)<br>        if datetime.datetime.fromtimestamp(f.stat().st_mtime) &lt; cutoff<br>        and not f.unlink()<br>    )<br>    log.info(f&quot;Retention: deleted {deleted} snapshots older than {days}d&quot;)<br>if __name__ == &quot;__main__&quot;:<br>    ts = datetime.datetime.utcnow().strftime(&quot;%Y%m%dT%H%M%SZ&quot;)<br>    log.info(f&quot;=== Qdrant backup start {ts} ===&quot;)<br>    backup_all(get_client(), ts)<br>    apply_retention(RETAIN_DAYS)<br>    log.info(&quot;=== Backup complete ===&quot;)</pre><p>Add an S3 lifecycle rule to move old snapshots to Glacier after 14 days and expire after 30. Cheap long-term storage with basically zero management.</p><p><strong>Restoring from a snapshot</strong></p><p>If you need to recover from zero on a single-node setup:</p><pre># Option A: restore at startup (simplest)<br>docker run -d --name qdrant \<br>  -p 6333:6333 \<br>  -v /data/qdrant:/qdrant/storage \<br>  -v /backups:/snapshots \<br>  qdrant/qdrant:v1.18.1 \<br>  ./qdrant --snapshot /snapshots/my_collection-2024-01-15.snapshot:my_collection<br><br># Option B: restore via API while running<br>curl -X PUT http://localhost:6333/collections/my_collection/snapshots/recover \<br>  -H &#39;Content-Type: application/json&#39; \<br>  -d &#39;{&quot;location&quot;: &quot;file:///tmp/my_collection.snapshot&quot;, &quot;priority&quot;: &quot;snapshot&quot;}&#39;</pre><p>Then verify the restore actually worked:</p><pre>info = client.get_collection(&quot;my_collection&quot;)<br>print(info.points_count)  # Does this match what you expected?<br>print(info.status)        # Should be &quot;green&quot;</pre><p>If the point count is suspiciously low, you may have hit the WAL gap is points written after the snapshot was taken aren’t there.</p><p><strong>RPO/RTO reality check</strong></p><p>With hourly snapshots, you can lose up to 1 hour of writes plus the WAL gap (a few seconds to a minute under normal load). If that’s okay for your use case, great. If you’re storing anything where data loss is measured in dollars or compliance violations, hourly snapshots combined with replication (not just one or the other) is what you need.</p><p>Qdrant Cloud handles this automatically with incremental backups, configurable retention, and one-click restore via the console. Not magic , but something you don’t have to build and test yourself. Gotcha.</p><h3>Section 4: Replication, Failover, and the 3am Phone Call</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*Ji02NVJycNPooKQj.gif" /></figure><p>Okay so you want HA. You want Qdrant to survive a node going down. Totally reasonable. Here’s what that actually involves.</p><p><strong>The Kubernetes setup</strong></p><p>You need a StatefulSet with pod anti-affinity (so pods don’t land on the same physical node), readiness probes, a PodDisruptionBudget, and a startup probe with enough budget for large-collection WAL replay. The Helm chart handles most of this:</p><pre># values-prod.yaml<br># Fetched from OSS implementations and tons of AI research and Vibe code this<br>replicaCount: 3<br>image:<br>  repository: docker.io/qdrant/qdrant<br>  tag: &quot;v1.18.1&quot;  # pin the version, never use latest in prod<br><br>config:<br>  cluster:<br>    enabled: true<br>    p2p:<br>      port: 6335<br>    consensus:<br>      tick_period_ms: 100<br>  log_level: INFO<br>resources:<br>  requests:<br>    cpu: &quot;2&quot;<br>    memory: &quot;16Gi&quot;<br>  limits:<br>    cpu: &quot;4&quot;<br>    memory: &quot;32Gi&quot;<br>persistence:<br>  size: 100Gi<br>  storageClassName: &quot;fast-ssd&quot;<br>snapshotPersistence:<br>  enabled: true<br>  size: 100Gi<br>readinessProbe:<br>  enabled: true<br>  initialDelaySeconds: 20<br>  periodSeconds: 5<br>  failureThreshold: 6<br>startupProbe:<br>  enabled: true<br>  initialDelaySeconds: 10<br>  periodSeconds: 5<br>  failureThreshold: 60  # 5 minute startup budget for large datasets<br>affinity:<br>  podAntiAffinity:<br>    requiredDuringSchedulingIgnoredDuringExecution:<br>    - labelSelector:<br>        matchExpressions:<br>          - key: app.kubernetes.io/name<br>            operator: In<br>            values: [qdrant]<br>      topologyKey: &quot;kubernetes.io/hostname&quot;<br>podDisruptionBudget:<br>  enabled: true<br>  maxUnavailable: 1<br>podManagementPolicy: Parallel  # Critical: OrderedReady causes deadlock on fresh cluster</pre><pre>helm repo add qdrant https://qdrant.github.io/qdrant-helm<br>helm upgrade -i qdrant qdrant/qdrant -f values-prod.yaml</pre><p><strong>What happens when a node dies</strong></p><p>Raft detects the dead peer via heartbeat. With tick_period_ms: 100, detection takes a few hundred milliseconds to a few seconds. The dead node&#39;s shard replicas are marked dead. Surviving active replicas keep serving reads and writes.</p><p>Yup! So queries continue. But here’s the catch: Qdrant does <strong>not</strong> automatically re-replicate shards to maintain your desired replication factor in self-hosted mode. That’s on you. Manual intervention every time a node fails permanently:</p><pre># Remove dead peer from consensus<br>DELETE /cluster/peer/{dead_peer_id}?force=true<br><br># Start new node and join cluster<br>./qdrant --bootstrap http://any_existing_node:6335<br># Manually replicate missing shards (one call per shard)<br>POST /collections/{name}/cluster<br>{<br>  &quot;replicate_shard&quot;: {<br>    &quot;shard_id&quot;: 0,<br>    &quot;from_peer_id&quot;: &lt;live_peer&gt;,<br>    &quot;to_peer_id&quot;: &lt;new_peer&gt;<br>  }<br>}</pre><p>You have to do this for every shard. If you have 12 shards distributed across 3 nodes, a node failure means you’re manually triggering multiple shard replications and monitoring each one until they complete.</p><p><strong>Resharding: the fun nobody talks about</strong></p><p>Turn up 6 months in and need to add more nodes to handle growth? Add the nodes. But your shards don’t automatically redistribute to use the new capacity. And if you want to change the shard count itself… you can’t, without recreating the collection from scratch and migrating all the data:</p><pre>docker run --net=host --rm -it registry.cloud.qdrant.io/library/qdrant-migration qdrant \<br>    --source.url &#39;http://localhost:6334&#39; \<br>    --source.collection &#39;my-collection&#39; \<br>    --target.url &#39;http://new-cluster:6334&#39; \<br>    --target.collection &#39;my-collection-v2&#39; \<br>    --migration.batch-size 64</pre><p>For large collections this migration runs for hours. During that time you’re managing two copies of your data. Fun.</p><p>Qdrant Cloud has native resharding via API since <a href="https://qdrant.tech/blog/qdrant-1.13.x/">v1.13.0</a>. Self-hosted does not. That’s just the reality.</p><p><strong>The honest cost</strong></p><p>Running a distributed Qdrant cluster means you now own a distributed system, Raft consensus, shard placement, replica health, manual failover procedures, upgrade sequencing. The ops burden for a 3-node cluster at steady state is roughly 8–15 hours per week when you account for monitoring, incident response, maintenance windows, and upgrades. That’s real engineering time that isn’t going into product.</p><h3>Section 5: Upgrades: The Thing Everyone Avoids Until It’s Urgent</h3><p>Qdrant ships a new minor version every 4–6 weeks. Patch releases come more frequently. There have been minor versions with 5–6 patches in quick succession.</p><p>The “3 versions behind” scenario happens quietly. You launch on v1.16, you get busy, four months pass, and you’re suddenly staring at v1.19 in the changelog wondering how you got here. Wait what? Three minor versions behind, and now you have to chain through v1.17, v1.18, v1.19 in sequence. You can’t skip.</p><p><strong>Why you can’t skip versions</strong></p><p>Qdrant’s storage format has changed multiple times across the v1.x series: RocksDB was removed, Gridstore became the default, shard key formats changed. Each minor release includes migration code for the previous format only. Skip two minors and the migration code in your target version doesn’t know how to handle your older storage layout. Startup failures or, worse, data read errors.</p><p>Also: Raft consensus only guarantees compatibility between adjacent minor versions. A v1.19 node cannot talk to a v1.17 node in the same cluster. So rolling upgrades are always limited to one step at a time.</p><p><strong>Single-node upgrade (step by step)</strong></p><pre># 1. Check current version<br>curl -s http://localhost:6333/ | jq &#39;.version&#39;<br><br># 2. Take a snapshot of each collection - this is your rollback point<br>curl -s -X POST http://localhost:6333/collections/my_collection/snapshots<br># 3. Stop the container<br>docker stop qdrant &amp;&amp; docker rm qdrant<br># 4. Pull the NEXT minor version (not latest - the specific next step)<br>docker pull qdrant/qdrant:v1.17.1<br># 5. Start with the same storage volume<br>docker run -d --name qdrant \<br>  -p 6333:6333 -p 6334:6334 \<br>  -v $(pwd)/qdrant_storage:/qdrant/storage:z \<br>  qdrant/qdrant:v1.17.1<br># 6. Watch logs for startup migrations<br>docker logs -f qdrant<br># 7. Verify<br>curl -s http://localhost:6333/readyz<br>curl -s http://localhost:6333/ | jq &#39;.version&#39;</pre><p>Repeat for each minor version in the chain. For large collections, startup migrations (like the RocksDB-to-Gridstore migration introduced in v1.16) can take minutes to tens of minutes before the node is ready.</p><p><strong>Cluster rolling upgrade</strong></p><pre># For each node in the cluster, in order:<br>#   1. Verify all collections are green<br>#   2. Drain the node (kubectl drain for K8s)<br>#   3. Update the image tag in values-prod.yaml<br>#   4. helm upgrade qdrant qdrant/qdrant -f values-prod.yaml<br>#   5. Wait for node to rejoin and all replicas to go active<br>#   6. Verify: GET /cluster and GET /collections/{name} for all collections<br>#   7. Repeat for next node<br><br># Monitor cluster state during the process<br>watch -n 5 &#39;curl -s http://localhost:6333/cluster | jq .result.raft_info&#39;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*2TttuhKrJCIHCl0-.gif" /></figure><p><strong>Can you roll back?</strong></p><p>Honestly? Not easily. Once Qdrant writes data in a newer storage format, an older binary can’t read it. You can’t downgrade in-place. If something goes wrong post-upgrade, your rollback path is:</p><ol><li>Stop the new version cluster</li><li>Provision a fresh cluster running the OLD version</li><li>Restore from the pre-upgrade snapshots you took before starting</li></ol><p>Any writes that happened between the snapshot and the rollback decision are lost. This is why taking snapshots before every upgrade isn’t optional — it’s your only recovery path.</p><p><strong>The v1.18 to v1.19 gotcha</strong></p><p>If you’re running anything below v1.19 and calling the legacy /search, /recommend, or /discover REST endpoints directly, stop. These were removed in v1.19. Not deprecated with a grace period. Removed. If your code calls these endpoints and you upgrade to v1.19, your application breaks immediately on startup. Audit your API call sites before any upgrade chain that crosses this boundary.</p><p>On Qdrant Cloud, upgrades are fully automated for HA clusters. You wake up on a new version. The cloud handles chaining through intermediate versions. You don’t think about it until the changelog email shows up in your inbox. That’s… actually pretty nice.</p><h3>Section 6: When Qdrant Cloud (or Hybrid Cloud as you wish) Just Makes Sense</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*FC4ZHu4WuZ4qf48V.gif" /></figure><p>Okay so after all of that, let me make the positive case for not self-hosting.</p><p><strong>The free tier is genuinely useful</strong></p><p><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud’s free tier</a> is permanent (no time limit), 0.5 vCPU, 1GB RAM, 4GB disk, and fits roughly 1 million 768-dimension vectors. For most prototypes and small production workloads, that’s real capacity. No credit card required. No infrastructure to manage.</p><p>It does auto-suspend after a week of inactivity, so it’s not for always-on workloads — but for development and small-scale production? Yup, it covers a lot of ground.</p><p><strong>What you get that you’d otherwise have to build</strong></p><p>With <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a>, the following things just… work:</p><ul><li>Automated incremental backups with configurable retention</li><li>Built-in monitoring with capacity alerts at 80% RAM and 80% disk</li><li>Managed rolling upgrades, zero-downtime for HA clusters</li><li>Automatic shard rebalancing when you add or remove nodes</li><li>Load balancer included, send everything to one endpoint</li><li>TLS managed, certificate rotation handled</li><li>Multi-AZ distribution at Premium tier with up to 99.95% uptime SLA</li></ul><p>All the things in sections 2–5 above? Not your problem. The engineering time that goes into operating self-hosted Qdrant that’s roughly 4–8 hours per week for a small team once you account for monitoring, incidents, upgrades, and maintenance for that time goes back to building product.</p><p><strong>Qdrant Hybrid Cloud: the middle path</strong></p><p>Here’s where it gets interesting for teams with real compliance requirements. Hybrid Cloud runs Qdrant on your own Kubernetes cluster (AWS, GCP, Azure, on-prem, anywhere) but the operations are managed by Qdrant’s control plane.</p><p>The architecture: a Kubernetes Operator and a Cloud Agent are deployed into your namespace. The Cloud Agent makes a single outbound HTTPS connection to cloud.qdrant.io:443. No inbound firewall rules required. Your vector data, payloads, and query contents never leave your infrastructure. What goes to Qdrant&#39;s control plane is only operational telemetry: resource usage, operation counts, cluster health. Not your data.</p><p>From your side: manage the Kubernetes cluster and its compute and storage. From Qdrant’s side: zero-downtime upgrades, shard rebalancing, scaling operations, backups, and monitoring is all through the same Cloud Console UI as the fully managed product.</p><p>This is not just “Qdrant on your own K8s.” That’s self-hosting. Hybrid Cloud is self-hosting with a managed operator and it’s the difference between running your own restaurant versus having a professional management company run a restaurant in your building. You provide the space; they handle operations.</p><p><strong>Who Hybrid Cloud is for:</strong></p><ul><li>Healthcare, financial services, government: regulated industries where data must stay in your infrastructure but you still want managed operations</li><li>Teams with existing cloud committed spend who want to maximize utilization of already-paid resources</li><li>Multi-cloud architectures where you need Qdrant co-located with different services across providers</li><li>Organizations that need air-gap-adjacent security posture without giving up operational quality</li></ul><p><strong>The migration path from self-hosted to Qdrant Cloud</strong></p><p>If you’re already self-hosting and want to migrate, it’s snapshot-based:</p><pre>from qdrant_client import QdrantClient<br>import urllib.request<br><br># Step 1: Create snapshot on your self-hosted instance<br>self_hosted = QdrantClient(url=&quot;http://your-server:6333&quot;, api_key=&quot;your-key&quot;)<br>snap = self_hosted.create_snapshot(collection_name=&quot;my_collection&quot;)<br>print(f&quot;Created: {snap.name}&quot;)<br># Step 2: Download it<br>url = f&quot;http://your-server:6333/collections/my_collection/snapshots/{snap.name}&quot;<br>req = urllib.request.Request(url)<br>req.add_header(&quot;api-key&quot;, &quot;your-key&quot;)<br>with urllib.request.urlopen(req) as r, open(f&quot;/tmp/{snap.name}&quot;, &quot;wb&quot;) as f:<br>    while chunk := r.read(8 * 1024 * 1024):<br>        f.write(chunk)<br># Step 3: Upload to Qdrant Cloud via multipart<br># curl -X POST \<br>#   &#39;https://your-cluster.cloud.qdrant.io/collections/my_collection/snapshots/upload?priority=snapshot&#39; \<br>#   -H &#39;api-key: your-cloud-api-key&#39; \<br>#   -H &#39;Content-Type: multipart/form-data&#39; \<br>#   -F &quot;snapshot=@/tmp/my_collection.snapshot&quot;<br># Step 4: Verify<br>cloud = QdrantClient(<br>    url=&quot;https://your-cluster.cloud.qdrant.io&quot;,<br>    api_key=&quot;your-cloud-api-key&quot;<br>)<br>info = cloud.get_collection(&quot;my_collection&quot;)<br>print(f&quot;Points: {info.points_count}&quot;)<br>print(f&quot;Status: {info.status}&quot;)</pre><p>Update your connection string. Done.</p><p><strong>The honest recommendation</strong></p><p>If you have an existing Kubernetes platform team that already operates stateful services at scale, and you have specific reasons (compliance, cost at scale, air-gapped requirements), self-hosting Qdrant is a reasonable choice. Go in with eyes open about the ops surface: the monitoring stack you’ll build, the upgrade cadence you’ll need to keep up with, the manual failover procedures, the backup automation.</p><p>If you don’t have that team, or if you have that team but they’re already stretched, Qdrant Cloud removes a significant ongoing maintenance burden. The free tier proves the product; the standard tier scales to production without infrastructure work.</p><p>If you’re in a regulated industry where data residency matters but you still want managed operations, Hybrid Cloud is worth a serious look. It’s the only path that gives you both data sovereignty and zero-touch operations.</p><p>The <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant documentation</a> is solid for whichever path you choose — it covers the self-hosted ops details thoroughly and the cloud setup is well-documented too.</p><p>So yeah. Self-hosting Qdrant isn’t impossible. It’s actually quite manageable if you know what you’re getting into. But “knowing what you’re getting into” is the whole point of this post — and now you do.</p><p><em>Found this useful? Subscribe to </em><a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella"><em>the Qdrant newsletter</em></a><em> for more honest takes on building with vector search in production.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e2f2bdce44ca" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/why-you-probably-shouldnt-self-host-qdrant-an-honest-look-e2f2bdce44ca">Why You Probably Shouldn’t Self-Host Qdrant (An Honest Look)</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns]]></title>
            <link>https://pub.towardsai.net/migrate-pinecone-to-qdrant-complete-migration-guide-zero-heart-burns-b8ba3cbbfd12?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8ba3cbbfd12</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Fri, 28 Aug 2026 17:01:03 GMT</pubDate>
            <atom:updated>2026-08-28T17:01:03.619Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7QB8kUACydIlNBkrRAjDxA.png" /></figure><p>Okay so… you opened a bill from Pinecone and did a double-take. Or you tried to self-host it and discovered that’s just not a thing Pinecone lets you do. Or maybe you hit the 40KB metadata limit mid-sprint and your whole architecture needed a rethink. Whatever the reason might be you’re here, you want to move your vectors, and you want to do it without a disaster.</p><p>I’ve been through this migration. Here’s everything I wish someone had told me before I started.</p><h3>Why You’re Even Reading This</h3><p>Here’s the thing about Pinecone… it’s genuinely easy to get started with. The docs are everywhere in AI tutorials, the API is clean, and you can have vectors in the cloud in like 20 minutes. That’s why so many teams start there. But then production hits…</p><p><strong>The cost curve surprises you.</strong> Pinecone’s serverless model charges per read unit, so every query costs money. At moderate QPS (say 500 queries per second), that meter is spinning fast. One team I talked to went from a comfortable $200/month in early testing to over $2k/month once their app got traction. The data doesn’t really get bigger, but the bills do.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/0*mDX8-6uZYLo-gHaO.jpg" /></figure><p><strong>Vendor lock-in is real.</strong> Here’s a thing that quietly terrifies me: Pinecone has no export API. Like, none. You cannot say “give me all my vectors as a file.” If you want your data back, you have to iterate through IDs and fetch them back one batch at a time, and that only works on serverless indexes. Pod-based indexes? No list API at all. Pretty wild right?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*qEjhMX2yDq84QiNKE6jPeA.jpeg" /></figure><p><strong>Namespace limits sneak up on you.</strong> Pinecone namespaces are a nice partitioning primitive but you’re capped at 100,000 namespaces on Standard. If you’re building a multi-tenant SaaS where each user has their own vector space… yeah, that can get uncomfortable fast.</p><p><strong>Metadata is flat and limited.</strong> 40KB per record, no nested JSON, no geo-coordinates, no full-text search on metadata fields. Need to store a document’s full text alongside its embedding? You’ll be doing a lot of creative truncation.</p><p><strong>No self-hosting, ever.</strong> This one is fundamental. Pinecone is SaaS-only. There is no Docker image, no Kubernetes operator, no “run it on your own machine” option. For teams with data residency requirements, regulated industries, or just a preference for not being dependent on external infrastructure… this is a hard blocker.</p><p>The lightbulb moment for most developers is when they run <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> locally with docker run -p 6333:6333 qdrant/qdrant and realize it&#39;s the same thing - same API, same capabilities, same performance - but it&#39;s just... there, on your machine, completely under your control.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/0*wCAakO-ZPW_z6bJH.gif" /></figure><p>Here’s the shape of this migration so you know what you’re getting into:</p><ol><li>Export your Pinecone data to a local JSONL file</li><li>Spin up Qdrant locally or on Qdrant Cloud</li><li>Load the data in and verify it</li><li>If you’re in production: run dual-write while you migrate, then cut over with zero downtime</li></ol><p>Nothing here is magic. It’s just a bit of Python and some patience.</p><h3>The Mental Model Shift</h3><p>Before touching any code, let’s map the concepts. This is reference material to skim it now, come back when something breaks.</p><h3>Core Concept Mapping</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*elLeciTO-lVQLARTNdAVJA.png" /></figure><p><strong>The namespace decision is actually important.</strong> You have two options:</p><ul><li><strong>Separate collection per namespace: </strong>great if you have a small number of large namespaces and want clean isolation</li><li><strong>Single collection with a </strong><strong>namespace payload field</strong> - better if you have many small namespaces, or want cross-namespace queries</li></ul><p>For most Pinecone migrations I’d go with option B (payload field), because collections have a recommended cap around 1,000 in Qdrant, and many Pinecone users have way more namespaces than that.</p><h3>Distance Metric Mapping</h3><pre># Pinecone metric values -&gt; Qdrant Distance enum<br>METRIC_MAP = {<br>    &quot;cosine&quot;:     Distance.COSINE,<br>    &quot;euclidean&quot;:  Distance.EUCLID,<br>    &quot;dotproduct&quot;: Distance.DOT,<br>}<br># Qdrant also has Distance.MANHATTAN - no Pinecone equivalent</pre><p>One gotcha here: Pinecone rescales cosine similarity to [0, 1]. Qdrant returns [-1, 1]. The rankings are identical, but any code that thresholds on raw score values will behave differently. Keep this in mind when you port your search logic.</p><h3>Filter Syntax from Before and After</h3><pre># === PINECONE ===<br>results = index.query(<br>    vector=[0.1] * 1536,<br>    filter={<br>        &quot;$and&quot;: [<br>            {&quot;genre&quot;: {&quot;$eq&quot;: &quot;sci-fi&quot;}},<br>            {&quot;year&quot;: {&quot;$gte&quot;: 2020}},<br>            {&quot;tags&quot;: {&quot;$in&quot;: [&quot;ai&quot;, &quot;robots&quot;]}}<br>        ]<br>    },<br>    top_k=10,<br>    namespace=&quot;my-namespace&quot;,<br>    include_metadata=True<br>)<br>matches = results[&quot;matches&quot;]  # list of {id, score, metadata}<br><br># === QDRANT ===<br>from qdrant_client.models import Filter, FieldCondition, MatchValue, MatchAny, Range<br>results = client.search(<br>    collection_name=&quot;my_collection&quot;,<br>    query_vector=[0.1] * 1536,<br>    query_filter=Filter(<br>        must=[<br>            FieldCondition(key=&quot;genre&quot;,  match=MatchValue(value=&quot;sci-fi&quot;)),<br>            FieldCondition(key=&quot;year&quot;,   range=Range(gte=2020)),<br>            FieldCondition(key=&quot;tags&quot;,   match=MatchAny(any=[&quot;ai&quot;, &quot;robots&quot;])),<br>            # If you&#39;re using the payload-field namespace strategy:<br>            FieldCondition(key=&quot;_namespace&quot;, match=MatchValue(value=&quot;my-namespace&quot;)),<br>        ]<br>    ),<br>    limit=10,<br>    with_payload=True<br>)<br># results is a list of ScoredPoint objects with .id, .score, .payload</pre><p>The filter mapping rule of thumb: $and maps to must, $or maps to should, $ne maps to must_not. Numeric ranges map directly. $in maps to MatchAny. Honestly pretty intuitive once you see it side by side.</p><h3>Sparse + Dense Hybrid</h3><p>Pinecone hybrid search uses a single alpha parameter (0 = pure sparse, 1 = pure dense, linear blend). Qdrant uses explicit prefetch legs and rank fusion - either RRF (Reciprocal Rank Fusion) or DBSF. More code to write, but dramatically more control over how the fusion behaves. You can weight each leg independently, which ends up being pretty important for production tuning.</p><h3>Getting Your Data Out of Pinecone</h3><p>Here’s the honest truth: <strong>Pinecone has no export feature.</strong> No “Download All Data” button, no export endpoint, no snapshot you can take away. The only way out is to iterate through vector IDs and fetch them back in batches.</p><p>The approach:</p><ol><li>Call list() to page through all vector IDs in a namespace (returns 100 IDs per page)</li><li>For each page of IDs, call fetch() to get the actual vectors + metadata (up to 1,000 IDs per call)</li><li>Write everything to a JSONL file</li></ol><p>One more thing before the code: this <strong>only works on serverless indexes</strong>. Pod-based indexes have no list API at all. If you&#39;re on a pod-based index... the only real option is to re-embed from your original document source, because there&#39;s no supported way to bulk-export from a pod index. Yikes.</p><p>Rate limits to know: list is 200 req/s, fetch is 100 req/s. With 1,000 IDs per fetch call, that&#39;s theoretically 100,000 vectors per second. In practice, you&#39;ll be slower because of network and parsing overhead - but you won&#39;t hit limits if you stay around 10 concurrent fetch calls.</p><p>Here’s a complete, resumable Pinecone exporter:</p><pre>#!/usr/bin/env python3<br># VIBE CODED for smooth and quick setups<br>&quot;&quot;&quot;<br>pinecone_dumper.py<br>Exports all vectors from a Pinecone serverless index to a JSONL file.<br>Resumable: if interrupted, just rerun - it will pick up from where it left off.<br>Requirements:<br>    pip install pinecone&gt;=3.0.0 tqdm<br>Usage:<br>    python pinecone_dumper.py \<br>        --api-key pcsk_XXX \<br>        --index-name my-index \<br>        --output vectors.jsonl<br>&quot;&quot;&quot;<br>import argparse<br>import json<br>import os<br>import time<br>import math<br>import pickle<br>from pathlib import Path<br>from pinecone import Pinecone<br>from tqdm import tqdm<br><br>LIST_BATCH_SIZE  = 100     # Pinecone max IDs per list page<br>FETCH_BATCH_SIZE = 1000    # Pinecone max IDs per fetch call<br>FETCH_RATE_SLEEP = 0.011   # ~90 req/s, safely under 100 req/s limit<br><br>def load_checkpoint(ckpt_path: str) -&gt; dict:<br>    if Path(ckpt_path).exists():<br>        with open(ckpt_path, &quot;rb&quot;) as f:<br>            return pickle.load(f)<br>    return {}<br><br>def save_checkpoint(ckpt_path: str, data: dict):<br>    tmp = ckpt_path + &quot;.tmp&quot;<br>    with open(tmp, &quot;wb&quot;) as f:<br>        pickle.dump(data, f)<br>    os.replace(tmp, ckpt_path)<br><br>def list_all_ids(index, namespace: str, checkpoint: dict, ckpt_path: str) -&gt; list:<br>    ckpt_key = f&quot;ids:{namespace}&quot;<br>    if ckpt_key in checkpoint:<br>        print(f&quot;  Resuming ID listing for namespace=&#39;{namespace}&#39; from checkpoint...&quot;)<br>        return checkpoint[ckpt_key]<br>    all_ids = []<br>    cursor = None<br>    page_num = 0<br>    print(f&quot;  Listing IDs for namespace=&#39;{namespace}&#39;...&quot;)<br>    while True:<br>        kwargs = {&quot;limit&quot;: LIST_BATCH_SIZE, &quot;namespace&quot;: namespace}<br>        if cursor:<br>            kwargs[&quot;pagination_token&quot;] = cursor<br>        try:<br>            resp = index.list_paginated(**kwargs)<br>        except Exception as e:<br>            print(f&quot;  ERROR listing page {page_num}: {e}. Retrying in 5s...&quot;)<br>            time.sleep(5)<br>            continue<br>        page_ids = [v.id for v in (resp.vectors or [])]<br>        all_ids.extend(page_ids)<br>        page_num += 1<br>        if page_num % 100 == 0:<br>            print(f&quot;  ... {len(all_ids):,} IDs listed so far&quot;)<br>        cursor = resp.pagination.next if resp.pagination else None<br>        if not cursor:<br>            break<br>    print(f&quot;  Found {len(all_ids):,} IDs in namespace=&#39;{namespace}&#39;&quot;)<br>    checkpoint[ckpt_key] = all_ids<br>    save_checkpoint(ckpt_path, checkpoint)<br>    return all_ids<br><br>def fetch_and_write(index, all_ids, namespace, output_file, checkpoint, ckpt_path):<br>    ckpt_key = f&quot;fetched_batches:{namespace}&quot;<br>    completed_batches = checkpoint.get(ckpt_key, set())<br>    total_batches = math.ceil(len(all_ids) / FETCH_BATCH_SIZE)<br>    new_writes = 0<br>    with tqdm(total=len(all_ids), desc=f&quot;  Fetching vectors (ns=&#39;{namespace}&#39;)&quot;) as pbar:<br>        for batch_idx in range(total_batches):<br>            start = batch_idx * FETCH_BATCH_SIZE<br>            batch_ids = all_ids[start:start + FETCH_BATCH_SIZE]<br>            if batch_idx in completed_batches:<br>                pbar.update(len(batch_ids))<br>                continue<br>            # Fetch with exponential backoff retry<br>            for attempt in range(5):<br>                try:<br>                    resp = index.fetch(ids=batch_ids, namespace=namespace)<br>                    break<br>                except Exception as e:<br>                    wait = 2 ** attempt<br>                    print(f&quot;\n  Fetch attempt {attempt+1} failed: {e}. Waiting {wait}s...&quot;)<br>                    time.sleep(wait)<br>            else:<br>                print(f&quot;\n  ERROR: Could not fetch batch {batch_idx}. Skipping.&quot;)<br>                pbar.update(len(batch_ids))<br>                continue<br>            vectors_map = resp.get(&quot;vectors&quot;, {})<br>            for pid, pdata in vectors_map.items():<br>                record = {<br>                    &quot;id&quot;:         pid,<br>                    &quot;values&quot;:     pdata.get(&quot;values&quot;, []),<br>                    &quot;metadata&quot;:   pdata.get(&quot;metadata&quot;, {}),<br>                    &quot;_namespace&quot;: namespace,<br>                }<br>                sv = pdata.get(&quot;sparse_values&quot;) or pdata.get(&quot;sparseValues&quot;)<br>                if sv:<br>                    record[&quot;sparse_values&quot;] = {<br>                        &quot;indices&quot;: sv.get(&quot;indices&quot;, []),<br>                        &quot;values&quot;:  sv.get(&quot;values&quot;, []),<br>                    }<br>                output_file.write(json.dumps(record) + &quot;\n&quot;)<br>                new_writes += 1<br>            completed_batches.add(batch_idx)<br>            if batch_idx % 10 == 0:<br>                checkpoint[ckpt_key] = completed_batches<br>                save_checkpoint(ckpt_path, checkpoint)<br>            pbar.update(len(batch_ids))<br>            time.sleep(FETCH_RATE_SLEEP)<br>    checkpoint[ckpt_key] = completed_batches<br>    save_checkpoint(ckpt_path, checkpoint)<br>    return new_writes<br><br>def main():<br>    parser = argparse.ArgumentParser(description=&quot;Export Pinecone index to JSONL&quot;)<br>    parser.add_argument(&quot;--api-key&quot;,    required=True)<br>    parser.add_argument(&quot;--index-name&quot;, required=True)<br>    parser.add_argument(&quot;--index-host&quot;, default=None,   help=&quot;Index host URL (optional, faster)&quot;)<br>    parser.add_argument(&quot;--namespace&quot;,  default=None,   help=&quot;Export one namespace only&quot;)<br>    parser.add_argument(&quot;--output&quot;,     default=&quot;pinecone_export.jsonl&quot;)<br>    parser.add_argument(&quot;--checkpoint&quot;, default=&quot;pinecone_dump_checkpoint.pkl&quot;)<br>    parser.add_argument(&quot;--fresh&quot;,      action=&quot;store_true&quot;, help=&quot;Ignore existing checkpoint&quot;)<br>    args = parser.parse_args()<br>    pc = Pinecone(api_key=args.api_key)<br>    index = pc.Index(host=args.index_host) if args.index_host else pc.Index(args.index_name)<br>    stats = index.describe_index_stats()<br>    namespaces = list(stats.get(&quot;namespaces&quot;, {}).keys())<br>    if not namespaces:<br>        namespaces = [&quot;&quot;]   # default namespace<br>    if args.namespace:<br>        namespaces = [args.namespace]<br>    total_vectors = stats.get(&quot;total_vector_count&quot;, 0)<br>    dimension     = stats.get(&quot;dimension&quot;, &quot;unknown&quot;)<br>    print(f&quot;\nIndex: {args.index_name}&quot;)<br>    print(f&quot;  Dimension:     {dimension}&quot;)<br>    print(f&quot;  Total vectors: {total_vectors:,}&quot;)<br>    print(f&quot;  Namespaces:    {namespaces}&quot;)<br>    print(f&quot;  Output:        {args.output}\n&quot;)<br>    checkpoint = {} if args.fresh else load_checkpoint(args.checkpoint)<br>    total_written = 0<br>    with open(args.output, &quot;a&quot;, encoding=&quot;utf-8&quot;) as out_f:<br>        for ns in namespaces:<br>            print(f&quot;Exporting namespace=&#39;{ns}&#39;...&quot;)<br>            ids     = list_all_ids(index, ns, checkpoint, args.checkpoint)<br>            written = fetch_and_write(index, ids, ns, out_f, checkpoint, args.checkpoint)<br>            total_written += written<br>            print(f&quot;  Done. {written:,} vectors written.\n&quot;)<br>    print(f&quot;Export complete! Total: {total_written:,} vectors -&gt; {args.output}&quot;)<br>    print(f&quot;\nNext step: python qdrant_uploader.py --input {args.output}&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()<br></pre><p>Run it like this:</p><pre>pip install &quot;pinecone&gt;=3.0.0&quot; tqdm<br><br>python pinecone_dumper.py \<br>    --api-key pcsk_YOUR_KEY \<br>    --index-name my-index \<br>    --output my_vectors.jsonl</pre><p>If it gets interrupted mid-run, just run it again …. it’ll skip everything it already fetched. The checkpoint file tracks progress at the batch level so you don’t re-fetch things you already have.</p><p>For very large indexes (100M+ vectors), this will take a while. The math: at 1,000 vectors per fetch and ~90 requests per second, you’re looking at roughly 90,000 vectors per second throughput. 100M vectors = ~18 minutes of pure fetch time, plus listing time on top. Run it overnight.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*GJuJL9zMyhEipydm.gif" /></figure><h3>Loading Into Qdrant (Local + Cloud)</h3><p>Okay, you’ve got your my_vectors.jsonl file. Now let&#39;s get it into Qdrant.</p><p><strong>Step 1: Start Qdrant locally</strong></p><pre>docker run -d --name qdrant \<br>    -p 6333:6333 -p 6334:6334 \<br>    -v $(pwd)/qdrant_storage:/qdrant/storage \<br>    qdrant/qdrant</pre><p>The web UI is at http://localhost:6333/dashboard. It&#39;s actually quite nice - you can browse collections, run test queries, inspect points. Better than nothing!</p><p><strong>Step 2: Recreate your collection</strong></p><p>You need the dimension and metric from your Pinecone index. The describe_index_stats() call doesn&#39;t return the metric (annoyingly) - check your index creation code or the Pinecone console.</p><pre>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    Distance, VectorParams, SparseVectorParams,<br>    HnswConfigDiff, OptimizersConfigDiff<br>)<br><br>client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br># For Qdrant Cloud:<br># client = QdrantClient(<br>#     url=&quot;https://YOUR-CLUSTER.cloud.qdrant.io&quot;,<br>#     api_key=&quot;YOUR_QDRANT_API_KEY&quot;<br># )<br>COLLECTION = &quot;my_collection&quot;<br>DIMENSION  = 1536   # match your Pinecone index dimension<br>METRIC     = &quot;cosine&quot;<br>METRIC_MAP = {<br>    &quot;cosine&quot;:     Distance.COSINE,<br>    &quot;euclidean&quot;:  Distance.EUCLID,<br>    &quot;dotproduct&quot;: Distance.DOT,<br>}<br># Disable HNSW during bulk load - re-enable after (3-5x faster ingest)<br>client.create_collection(<br>    collection_name=COLLECTION,<br>    vectors_config=VectorParams(size=DIMENSION, distance=METRIC_MAP[METRIC]),<br>    optimizers_config=OptimizersConfigDiff(indexing_threshold=0),<br>)<br>print(f&quot;Created collection &#39;{COLLECTION}&#39;&quot;)</pre><p>Notice indexing_threshold=0 - that disables HNSW index building during the bulk load. We&#39;ll re-enable it after. This makes loading 3-5x faster because Qdrant isn&#39;t trying to build the graph while you&#39;re flooding it with vectors.</p><p><strong>Step 3: The uploader script</strong></p><pre>#!/usr/bin/env python3<br><br># Once again, have vibe coded this part to perfection!<br><br>&quot;&quot;&quot;<br>qdrant_uploader.py<br>Reads a JSONL file from pinecone_dumper.py and bulk-upserts into Qdrant.<br>Handles ID translation, batching, retries, and progress tracking.<br>Requirements:<br>    pip install &quot;qdrant-client&gt;=1.9.0&quot; tqdm<br>Usage:<br>    python qdrant_uploader.py \<br>        --input vectors.jsonl \<br>        --collection my_collection \<br>        --qdrant-url http://localhost:6333<br>&quot;&quot;&quot;<br>import argparse<br>import json<br>import hashlib<br>import time<br>from pathlib import Path<br>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    PointStruct, SparseVector, Distance, VectorParams,<br>    OptimizersConfigDiff, UpdateStatus<br>)<br>from tqdm import tqdm<br><br>UPSERT_BATCH_SIZE = 512    # 256-512 is the sweet spot<br>MAX_RETRIES       = 5<br><br>def pinecone_id_to_qdrant(pinecone_id: str) -&gt; str:<br>    &quot;&quot;&quot;<br>    Convert a Pinecone string ID to a deterministic UUID for Qdrant.<br>    We store the original ID in payload as _pinecone_id for reverse lookup.<br>    &quot;&quot;&quot;<br>    h = hashlib.sha256(pinecone_id.encode()).hexdigest()<br>    return f&quot;{h[:8]}-{h[8:12]}-{h[12:16]}-{h[16:20]}-{h[20:32]}&quot;<br><br>def count_lines(filepath: str) -&gt; int:<br>    count = 0<br>    with open(filepath, &quot;rb&quot;) as f:<br>        for _ in f:<br>            count += 1<br>    return count<br><br>def upsert_batch_with_retry(client, collection, points, max_retries=MAX_RETRIES):<br>    for attempt in range(max_retries):<br>        try:<br>            result = client.upsert(<br>                collection_name=collection,<br>                points=points,<br>                wait=True,<br>            )<br>            return result.status == UpdateStatus.COMPLETED<br>        except Exception as e:<br>            wait = 2 ** attempt<br>            print(f&quot;\n  Upsert attempt {attempt+1} failed: {e}. Waiting {wait}s...&quot;)<br>            time.sleep(wait)<br>    print(f&quot;\n  ERROR: Batch failed after {max_retries} retries. Skipping {len(points)} points.&quot;)<br>    return False<br><br>def build_point(record: dict, has_sparse: bool) -&gt; PointStruct:<br>    pinecone_id = record[&quot;id&quot;]<br>    qdrant_id   = pinecone_id_to_qdrant(pinecone_id)<br>    values      = record.get(&quot;values&quot;, [])<br>    payload = dict(record.get(&quot;metadata&quot;, {}))<br>    payload[&quot;_pinecone_id&quot;] = pinecone_id<br>    payload[&quot;_namespace&quot;]   = record.get(&quot;_namespace&quot;, &quot;&quot;)<br>    if has_sparse and &quot;sparse_values&quot; in record:<br>        sv = record[&quot;sparse_values&quot;]<br>        vector = {<br>            &quot;dense&quot;:  values,<br>            &quot;sparse&quot;: SparseVector(<br>                indices=sv.get(&quot;indices&quot;, []),<br>                values=sv.get(&quot;values&quot;, [])<br>            )<br>        }<br>    else:<br>        vector = values<br>    return PointStruct(id=qdrant_id, vector=vector, payload=payload)<br><br>def main():<br>    parser = argparse.ArgumentParser(description=&quot;Upload JSONL to Qdrant&quot;)<br>    parser.add_argument(&quot;--input&quot;,       required=True)<br>    parser.add_argument(&quot;--collection&quot;,  required=True)<br>    parser.add_argument(&quot;--qdrant-url&quot;,  default=&quot;http://localhost:6333&quot;)<br>    parser.add_argument(&quot;--api-key&quot;,     default=None, help=&quot;Qdrant API key (for cloud)&quot;)<br>    parser.add_argument(&quot;--has-sparse&quot;,  action=&quot;store_true&quot;)<br>    parser.add_argument(&quot;--batch-size&quot;,  type=int, default=UPSERT_BATCH_SIZE)<br>    args = parser.parse_args()<br>    client = QdrantClient(url=args.qdrant_url, api_key=args.api_key)<br>    try:<br>        info = client.get_collection(args.collection)<br>        print(f&quot;Collection &#39;{args.collection}&#39; found - {info.vectors_count or 0:,} existing vectors&quot;)<br>    except Exception:<br>        print(f&quot;ERROR: Collection &#39;{args.collection}&#39; not found. Create it first.&quot;)<br>        return<br>    total_lines = count_lines(args.input)<br>    print(f&quot;\nInput file: {args.input} ({total_lines:,} records)&quot;)<br>    print(f&quot;Batch size: {args.batch_size}\n&quot;)<br>    batch = []<br>    total_uploaded = 0<br>    total_failed   = 0<br>    with open(args.input, &quot;r&quot;, encoding=&quot;utf-8&quot;) as f, \<br>         tqdm(total=total_lines, desc=&quot;Uploading&quot;, unit=&quot;vec&quot;) as pbar:<br>        for line in f:<br>            line = line.strip()<br>            if not line:<br>                continue<br>            try:<br>                record = json.loads(line)<br>            except json.JSONDecodeError as e:<br>                print(f&quot;\n  Bad JSON line: {e}. Skipping.&quot;)<br>                pbar.update(1)<br>                continue<br>            batch.append(build_point(record, args.has_sparse))<br>            if len(batch) &gt;= args.batch_size:<br>                ok = upsert_batch_with_retry(client, args.collection, batch)<br>                if ok:<br>                    total_uploaded += len(batch)<br>                else:<br>                    total_failed += len(batch)<br>                pbar.update(len(batch))<br>                batch = []<br>        if batch:<br>            ok = upsert_batch_with_retry(client, args.collection, batch)<br>            total_uploaded += len(batch) if ok else 0<br>            total_failed   += len(batch) if not ok else 0<br>            pbar.update(len(batch))<br>    print(f&quot;\nUpload complete! Uploaded: {total_uploaded:,}  Failed: {total_failed:,}&quot;)<br>    # Re-enable HNSW indexing now that bulk load is done<br>    print(&quot;\nRe-enabling HNSW indexing...&quot;)<br>    client.update_collection(<br>        collection_name=args.collection,<br>        optimizers_config=OptimizersConfigDiff(indexing_threshold=20_000),<br>    )<br>    print(&quot;Waiting for index to build (this may take a few minutes)...&quot;)<br>    while True:<br>        info   = client.get_collection(args.collection)<br>        status = str(info.status).lower()<br>        if &quot;green&quot; in status:<br>            break<br>        print(f&quot;  Status: {status}... waiting 15s&quot;)<br>        time.sleep(15)<br>    final_count = client.get_collection(args.collection).vectors_count<br>    print(f&quot;\nCollection &#39;{args.collection}&#39; is ready! Vector count: {final_count:,}&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()<br></pre><p>Run the full pipeline:</p><pre>pip install &quot;qdrant-client&gt;=1.9.0&quot; tqdm<br><br># 1. Export from Pinecone<br>python pinecone_dumper.py --api-key pcsk_XXX --index-name my-index --output vectors.jsonl<br># 2. Upload to local Qdrant<br>python qdrant_uploader.py \<br>    --input vectors.jsonl \<br>    --collection my_collection \<br>    --qdrant-url http://localhost:6333</pre><p>For <strong>Qdrant Cloud</strong>: <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">sign up here</a>, create a cluster (the free tier gets you 1M vectors permanently), then use:</p><pre>python qdrant_uploader.py \<br>    --input vectors.jsonl \<br>    --collection my_collection \<br>    --qdrant-url https://YOUR-CLUSTER.cloud.qdrant.io \<br>    --api-key YOUR_QDRANT_KEY</pre><p><strong>Step 4: Verify it worked</strong></p><pre>from qdrant_client import QdrantClient<br><br>client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br># Exact count (Qdrant&#39;s vectors_count is approximate for large collections)<br>def exact_count(client, collection):<br>    count, offset = 0, None<br>    while True:<br>        recs, next_offset = client.scroll(<br>            collection_name=collection,<br>            limit=1000,<br>            offset=offset,<br>            with_payload=False,<br>            with_vectors=False<br>        )<br>        count += len(recs)<br>        offset = next_offset<br>        if offset is None:<br>            break<br>    return count<br>n = exact_count(client, &quot;my_collection&quot;)<br>print(f&quot;Qdrant has {n:,} vectors&quot;)<br># Spot-check: search and print original Pinecone IDs from payload<br>test_vec = [0.1] * 1536  # replace with a real vector<br>results = client.search(&quot;my_collection&quot;, query_vector=test_vec, limit=5, with_payload=True)<br>for r in results:<br>    print(f&quot;  score={r.score:.4f}  original_pinecone_id={r.payload.get(&#39;_pinecone_id&#39;)}&quot;)</pre><p><strong>One more option: Qdrant’s official migration tool</strong></p><p>Qdrant ships a Docker-based migration CLI that handles Pinecone directly. For serverless indexes, one command does the whole thing:</p><pre>docker run --rm -it registry.cloud.qdrant.io/library/qdrant-migration pinecone \<br>    --pinecone.index-host &#39;https://your-index.svc.pinecone.io&#39; \<br>    --pinecone.index-name &#39;your-index&#39; \<br>    --pinecone.api-key &#39;pcsk_...&#39; \<br>    --qdrant.url &#39;https://your-cluster.cloud.qdrant.io:6334&#39; \<br>    --qdrant.api-key &#39;your-qdrant-key&#39; \<br>    --qdrant.collection &#39;your-collection&#39; \<br>    --migration.batch-size 64</pre><p>Note the gRPC port (6334): the migration tool uses gRPC, not the REST API. And it’s resumable via an internal _migration_offsets collection it creates on the target. If it gets killed, just rerun and it picks up from the last offset.</p><h3>Zero Downtime Migration (The Production Way)</h3><p>If you’re running a live production service on Pinecone, you can’t just turn it off run the migration and turn it back on. Here’s how to do it with zero downtime.</p><p>The core idea: run both databases simultaneously. Pinecone stays primary for reads. Qdrant gets all the writes. Once Qdrant has all the historical data and has proven itself with real traffic, you flip the switch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*w6vHMTfyCwDCqui6.gif" /></figure><h3>The Dual-Write Wrapper</h3><pre>import threading<br>from concurrent.futures import ThreadPoolExecutor<br>from queue import Queue, Empty<br><br>class DualWriteClient:<br>    &quot;&quot;&quot;<br>    Wrap your existing Pinecone index to simultaneously write to Qdrant.<br>    Pinecone is primary (writes block on Pinecone success).<br>    Qdrant writes are async and non-blocking - failures queue for retry.<br>    &quot;&quot;&quot;<br>    def __init__(self, pinecone_idx, qdrant_client, qdrant_collection):<br>        self.pine   = pinecone_idx<br>        self.qdrant = qdrant_client<br>        self.col    = qdrant_collection<br>        self.pool   = ThreadPoolExecutor(max_workers=4, thread_name_prefix=&quot;qdrant-write&quot;)<br>        self.retry_q = Queue(maxsize=10_000)<br>        self._start_retry_worker()<br>    def upsert(self, vectors, namespace=&quot;&quot;):<br>        # Step 1: Write to Pinecone (primary, blocks until confirmed)<br>        self.pine.upsert(vectors=vectors, namespace=namespace)<br>        # Step 2: Write to Qdrant async (never blocks your app)<br>        self.pool.submit(self._qdrant_upsert, vectors, namespace)<br>    def _qdrant_upsert(self, vectors, namespace):<br>        from qdrant_client.models import PointStruct<br>        points = [<br>            PointStruct(<br>                id=pinecone_id_to_qdrant(v[&quot;id&quot;]),<br>                vector=v[&quot;values&quot;],<br>                payload={**v.get(&quot;metadata&quot;, {}), &quot;_pinecone_id&quot;: v[&quot;id&quot;], &quot;_namespace&quot;: namespace}<br>            )<br>            for v in vectors<br>        ]<br>        try:<br>            self.qdrant.upsert(collection_name=self.col, points=points, wait=True)<br>        except Exception as e:<br>            print(f&quot;Qdrant write failed, queuing retry: {e}&quot;)<br>            try:<br>                self.retry_q.put_nowait((&quot;upsert&quot;, points))<br>            except Exception:<br>                print(f&quot;Retry queue full - {len(points)} points dropped&quot;)<br>    def delete(self, ids, namespace=&quot;&quot;):<br>        self.pine.delete(ids=ids, namespace=namespace)<br>        qdrant_ids = [pinecone_id_to_qdrant(i) for i in ids]<br>        self.pool.submit(self._qdrant_delete, qdrant_ids)<br>    def _qdrant_delete(self, ids):<br>        try:<br>            self.qdrant.delete(collection_name=self.col, points_selector=ids, wait=True)<br>        except Exception as e:<br>            try:<br>                self.retry_q.put_nowait((&quot;delete&quot;, ids))<br>            except Exception:<br>                pass<br>    def _start_retry_worker(self):<br>        t = threading.Thread(target=self._retry_loop, daemon=True)<br>        t.start()<br>    def _retry_loop(self):<br>        while True:<br>            try:<br>                op, payload = self.retry_q.get(timeout=5)<br>            except Empty:<br>                continue<br>            for attempt in range(5):<br>                try:<br>                    if op == &quot;upsert&quot;:<br>                        self.qdrant.upsert(collection_name=self.col, points=payload, wait=True)<br>                    else:<br>                        self.qdrant.delete(collection_name=self.col, points_selector=payload, wait=True)<br>                    break<br>                except Exception:<br>                    time.sleep(2 ** attempt)<br><br># Usage: replace your existing pinecone_index with this one line<br># Before: results = pinecone_index.query(...)<br># After:  results = dual_client.pine.query(...)  &lt;- reads still go to Pinecone<br>#<br># dual_client = DualWriteClient(pc.Index(&quot;my-index&quot;), qdrant_client, &quot;my_collection&quot;)<br># dual_client.upsert(vectors=[...])  &lt;- writes go to both</pre><h3>The Migration Timeline</h3><p>Once dual-write is running, here’s the sequence:</p><ol><li><strong>Start dual-write</strong> : your app now writes to both. Reads still come from Pinecone.</li><li><strong>Run the backfill: </strong>use the dumper + uploader scripts from Sections 3–4 to migrate historical data. Dual-write handles anything new that comes in while the backfill runs.</li><li><strong>Shadow mode: </strong>for a day or two, run every search query against Qdrant too (don’t show results to users yet), compare rankings, log quality differences.</li><li><strong>Cutover: </strong>once you’re happy with quality parity, flip the read traffic.</li></ol><h3>Cutover Checklist</h3><p>Before you switch traffic to Qdrant: JUST VERI FY the following okay?</p><pre>[x] Dual-write has been running &gt;= 48 hours without errors<br>[x] Backfill complete - Qdrant vector count within 0.1% of Pinecone count<br>[x] Shadow comparison: mean Recall@10 &gt;= 0.95 over at least 10,000 queries<br>[x] Qdrant collection status = green (fully indexed)<br>[x] P99 search latency on Qdrant within 15% of Pinecone<br>[x] Rollback plan tested - know how to re-enable Pinecone reads in under 5 minutes</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/1*cb5ibJ8dR9ieQim0hBLVaw.gif" /></figure><h3>The Atomic Cutover (Qdrant Aliases)</h3><p>Qdrant has collection aliases so that a single API call switches an alias from one collection to another with zero gap. Use this for the cutover:</p><pre>from qdrant_client.models import CreateAliasOperation, DeleteAliasOperation<br># One API call - no window where the alias is undefined<br>qdrant_client.update_collection_aliases(<br>    change_aliases_operations=[<br>        DeleteAliasOperation(delete_alias={&quot;alias_name&quot;: &quot;production&quot;}),<br>        CreateAliasOperation(create_alias={<br>            &quot;collection_name&quot;: &quot;my_collection&quot;,<br>            &quot;alias_name&quot;: &quot;production&quot;<br>        }),<br>    ]<br>)<br># From this point: all reads going to &quot;production&quot; alias hit Qdrant</pre><h3>Rollback Plan</h3><p>Keep dual-write running for at least 48 hours after full cutover. If something goes wrong, rollback is literally just updating a feature flag or load balancer weight to send reads back to Pinecone. Since dual-write is still active, Pinecone stays in sync during that window. Rolling back is instant and safe.</p><h3>Why Qdrant Cloud Hits Different</h3><p>Okay… so I’ve been pretty technical so far. Let me just talk about what actually feels different when you’re using Qdrant versus Pinecone day-to-day.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/0*ceaa4RZP2Cd-iRXv.gif" /></figure><p><strong>The free tier is actually useful.</strong> <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud’s</a> free tier is permanent and not a trial, and supports roughly 1M vectors on a real cluster</p><p>(0.5 vCPU, 1 GB RAM, 4 GB disk)!!!!!!</p><p>Pinecone’s free tier is 100,000 vectors on a single serverless index with the per-read-unit billing lurking. Qdrant’s free tier is just… a free cluster. No gotchas, no time limit.</p><blockquote>NOTABLE MENTIONS FROM MY RESEARCH AND DUMP BELOW!</blockquote><p><strong>You can actually run it yourself.</strong> docker run -p 6333:6333 qdrant/qdrant. That&#39;s it. The self-hosted version and the cloud version run the same binary. No features are gated behind cloud-only access. The API is identical everywhere. You can move between local development, your own Kubernetes cluster, and Qdrant Cloud at any time. Your data is portable. Your skills transfer. Pinecone is SaaS-only, period.</p><p><strong>Quantization is a superpower.</strong> Pinecone stores everything as float32. That’s 4 bytes per dimension, no exceptions. For a 10M vector / 1536-dim collection, that’s ~61 GB of raw vector data. With Qdrant’s Turbo4 4-bit datatype (from v1.19), you get that down to ~7.6 GB while maintaining solid recall. With binary quantization you’re under 2 GB. This is roughly a 30x memory reduction. At $5/GB/month in cloud RAM costs, that’s a very large number.</p><blockquote>Quick cost comparison for 10M vectors at 1536 dims</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*w3lVuy8QiV5Yb7wAbMOTeA.png" /></figure><p><strong>Filterable HNSW is a real algorithmic difference.</strong> Pinecone applies filters as pre-filter or post-filter. If your filter is very selective (say, filter to 1% of vectors), post-filtering wastes a huge amount of graph traversal work, and pre-filtering with very small candidate sets loses recall. Qdrant integrates payload filtering directly into the HNSW graph traversal — it navigates the graph while respecting the filter in real-time. This gives predictable recall even on highly selective filters. It’s not a marketing claim — it’s a fundamentally different query execution model.</p><p><strong>Multi-vector per point.</strong> Each Qdrant point can have multiple named vectors. One point can carry a text embedding (768-dim), an image embedding (512-dim), and a sparse BM25 vector, all stored together with their payload. A single hybrid search can fuse all three with RRF. Pinecone gives you one vector per record. Full stop. This matters a lot for multimodal and hybrid retrieval workloads.</p><p><strong>Data portability is the anti-vendor-lock-in story.</strong> With Qdrant, you can snapshot any collection and download it:</p><pre># Create a snapshot<br>snapshot_info = client.create_snapshot(collection_name=&quot;my_collection&quot;)<br>print(f&quot;Snapshot: {snapshot_info.name}&quot;)<br><br># Download it (it&#39;s just an HTTP GET)<br># GET /collections/my_collection/snapshots/{snapshot_name}</pre><p>That snapshot is your data. You can restore it to any Qdrant instance anywhere. If Qdrant doubles their prices tomorrow, you have a complete copy of your data ready to move. With Pinecone, you do the LIST+FETCH dance described in Section 3. If Pinecone has an outage and you can’t list your IDs… well.</p><p><strong>Open source, for real.</strong> Qdrant has 34,000+ GitHub stars, Apache 2.0 license, and the team actively responds to issues. You can read the source code, understand exactly what’s happening with your data, file bugs, send PRs. When something weird happens with your search results, you can actually dig in and find out why. Pinecone is a black box. That distinction sounds abstract until something goes wrong.</p><p>The <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant documentation</a> covers all of this in depth — quantization options, memory tier configs, distributed deployment, the lot. And if you want to follow what’s coming next (new quantization research, benchmark results, engineering deep-dives), the <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">newsletter</a> is worth subscribing to. They publish real engineering content, not just product announcements.</p><p>Alright now! that’s the full picture. The migration is doable okay and most teams do it over a week or two with zero downtime using the dual-write approach. The scripts handle the heavy lifting. The main gotchas to keep in mind:</p><ul><li>Pinecone has no export API: you’re doing LIST+FETCH, and that only works on serverless indexes</li><li>Pod-based indexes need to be re-embedded from your original document source</li><li>Cosine scores have different ranges ([0,1] in Pinecone, [-1,1] in Qdrant) - watch any code that thresholds on raw scores</li><li>Keep dual-write running for 48 hours post-cutover before you decommission anything</li></ul><p>Good luck with the migration. You’ve got this.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8ba3cbbfd12" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/migrate-pinecone-to-qdrant-complete-migration-guide-zero-heart-burns-b8ba3cbbfd12">Migrate Pinecone to Qdrant: Complete Migration Guide | Zero Heart Burns</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Qdrant 1.19 | TurboQuant & Memory Tiers: Store More, Spend Less]]></title>
            <link>https://blog.stackademic.com/qdrant-1-19-turboquant-memory-tiers-store-more-spend-less-50b64a2fe0ef?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/50b64a2fe0ef</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[language]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Wed, 19 Aug 2026 21:31:01 GMT</pubDate>
            <atom:updated>2026-08-31T06:51:17.637Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/768/1*s8XCj94oVlH8u4smeJNujg.png" /><figcaption>Author Feat AI</figcaption></figure><p>If you’ve been running a vector database at any real scale, you’ve probably hit the moment where you open your cloud bill and just… stare at it. RAM is expensive. Disk is cheap. And somehow you ended up with a setup where 80% of your AWS bill is memory for a collection that gets queried maybe 50 times a day.</p><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.19 has two features that directly attack this problem: the <strong>Turbo4 datatype</strong> (a 4-bit-only storage format with no full-precision copy) and a <strong>unified memory tier system</strong> that replaces a confusing pile of scattered boolean flags with one clean parameter. Together they give you real control over where your data lives and how much RAM you’re actually paying for.</p><p>Lets’ Gid Deeper Guys!!</p><h3>1. The Storage Problem</h3><p>Here’s the thing about vector search at scale: the math is brutal.</p><p>Say you’re storing OpenAI text-embedding-3-small embeddings. Those are 1536 dimensions at float32. One vector = 6,144 bytes. 10 million vectors = <strong>~57 GB</strong>. And that&#39;s just the raw vectors - you still need RAM for the HNSW graph (which is proportional to vector count), quantized copies, payload indexes, etc.</p><p>Before 1.19, Qdrant’s “standard” setup for large collections looked like this: store float32 vectors on disk, keep a quantized (say, int8) copy pinned in RAM for graph traversal, rescore against the float32 originals for final ranking. Smart pattern! But your disk footprint was float32 size + quantized size. For a 4-bit TurboQuant copy that&#39;s 32 bits + 4 bits = <strong>36 bits per dimension</strong> total.</p><p>And the memory config API was… not great, honestly. You had on_disk: true on VectorParams for the original vectors, always_ram: true on quantization configs for the compressed copy, and on_disk_payload: true at the collection level for payloads. Three different boolean flags, none of them named consistently, none of them applying to HNSW links at all (those were always heap-loaded with no control).</p><p>Qdrant 1.19 fixes both of these.</p><h3>2. TurboQuant (Turbo4): 4-Bit and Done</h3><p>TurboQuant itself shipped in Qdrant 1.18 and it’s a quantization method based on a technique from Google Research that applies a random rotation to your vectors before compressing them. The rotation redistributes information evenly across all dimensions, so when you quantize down to 4 bits (16 discrete levels per dimension), you’re losing a tiny bit from every dimension rather than potentially destroying a few high-signal ones. That’s the key insight, and it makes 4-bit quantization dramatically more accurate than naive 4-bit approaches.</p><p>In 1.18, TurboQuant was a <em>quantization method</em> sitting on top of float32 storage. Your float32 vectors lived on disk, and the 4-bit compressed copy lived in RAM for fast graph traversal. You could rescore against the originals. Nice.</p><p><strong>In 1.19, Turbo4 is a datatype, not just a quantization layer.</strong></p><pre>from qdrant_client import QdrantClient, model<br><br>client = QdrantClient(&quot;http://localhost:6333&quot;<br><br>client.create_collection(<br>    collection_name=&quot;my_collection&quot;,<br>    vectors_config=models.VectorParams(<br>        size=1536,<br>        distance=models.Distance.COSINE,<br>        datatype=models.Datatype.TURBO4,  # &lt;-- this is it<br>    ),<br>)</pre><p>Or in the REST API:</p><pre>PUT /collections/my_collection<br>{<br>    &quot;vectors&quot;: {<br>        &quot;size&quot;: 1536,<br>        &quot;distance&quot;: &quot;Cosine&quot;,<br>        &quot;datatype&quot;: &quot;turbo4&quot;<br>    }<br>}</pre><p>When you use datatype: &quot;turbo4&quot;, Qdrant stores <strong>only</strong> the 4-bit representation. There is no float32 copy anywhere - not on disk, not in RAM, not in your object storage. The 4-bit encoding IS the storage format. Full stop.</p><p>Why does this matter for storage numbers? Let’s look at what the 1.18 TurboQuant approach actually stored:</p><p>WhatBits per dimfloat32 original (disk)32 bits4-bit TurboQuant copy (RAM)4 bits<strong>Total (v1.18 approach)36 bitsTurbo4 datatype (v1.19)4 bits</strong></p><p>36 / 4 = <strong>9x reduction</strong> vs the prior approach. For 10 million 1536-dim vectors: the old way costs ~64 GB total, Turbo4 costs ~7 GB. Pretty wild right?</p><p>For ColBERT-style multi-vector collections (where you store dozens of token-level embeddings per document), this compounds hard. Every single vector gets that 9x reduction.</p><p><strong>Okay but what’s the catch?</strong></p><p>Rescoring is gone. When you query a Turbo4 collection, the HNSW graph traversal AND the final ranking both use 4-bit distances. There’s no float32 backup to re-rank against. The 4-bit approximation is your final answer.</p><p>In practice, 4-bit quantization without rescoring typically lands in the <strong>0.90–0.95 recall range</strong> for most dense embedding tasks (vs ~0.97–0.99 with rescoring). For semantic search, recommendations, first-stage retrieval before a cross-encoder reranker and that’s totally fine. For legal document search or compliance lookups where missing a true match has real consequences — stick with TurboQuant quantization on top of float32 storage.</p><p>One more thing: Turbo4 is dense-vectors-only. Sparse vectors don’t support it. Also, Manhattan (L1) distance technically works but requires full vector reconstruction — use Cosine, Dot, or Euclidean for good performance.</p><p><strong>Want the best of both worlds?</strong> You can stack quantization on top of Turbo4. Use 1-bit TurboQuant for HNSW traversal, then rescore against the 4-bit Turbo4 vectors (instead of float32). You don’t get the recall of rescoring against float32, but you get way better performance than no rescoring at all, with the disk footprint of 4-bit storage:</p><pre>client.create_collection(<br>    collection_name=&quot;my_collection&quot;,<br>    vectors_config=models.VectorParams(<br>        size=1536,<br>        distance=models.Distance.COSINE,<br>        datatype=models.Datatype.TURBO4,<br>    ),<br>    quantization_config=models.TurboQuantization(<br>        turbo=models.TurboQuantQuantizationConfig(<br>            memory=models.Memory.PINNED,<br>            bits=models.TurboQuantBitSize.BITS1,<br>        ),<br>    ),<br>)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*7F13wEp2P-Uc3WEs.gif" /></figure><h3>3. Memory Tiers: One Param to Rule Them All</h3><p>Okay so the old memory config in Qdrant was honestly kind of a mess. You had:</p><ul><li>on_disk: true on VectorParams - for original vectors only</li><li>always_ram: true on quantization configs - for compressed vectors only</li><li>on_disk_payload: true at collection level - for payloads only</li><li>HNSW index: always loaded into heap memory, no control at all</li></ul><p>Three different flag names, different locations in the config, and you couldn’t control HNSW memory placement at all. Not great for operators trying to tune memory usage carefully.</p><p>Qdrant 1.19 replaces all of this with a single memory parameter that works on every major component:</p><ul><li>&quot;pinned&quot; - data is locked in RAM, never evicted by the OS</li><li>&quot;cached&quot; - data lives on disk, OS page cache is pre-warmed at startup, evictable under memory pressure</li><li>&quot;cold&quot; - data lives on disk, loaded lazily on first access, no pre-warming</li></ul><p>Same parameter, same values, everywhere. Vectors, HNSW links, quantized vectors, sparse indexes, payloads, payload field indexes — all of them. Here’s a full hybrid config in Python:</p><pre>client.create_collection(<br>    collection_name=&quot;warm_hybrid&quot;,<br>    vectors_config=models.VectorParams(<br>        size=768,<br>        distance=models.Distance.COSINE,<br>        memory=models.Memory.COLD,        # original vectors on disk<br>    ),<br>    hnsw_config=models.HnswConfigDiff(<br>        memory=models.Memory.CACHED,      # HNSW warm in OS cache<br>    ),<br>    quantization_config=models.ScalarQuantization(<br>        scalar=models.ScalarQuantizationConfig(<br>            type=models.ScalarType.INT8,<br>            memory=models.Memory.PINNED,  # quantized vectors locked in RAM<br>        ),<br>    ),<br>    payload_storage_config=models.PayloadStorageConfig(<br>        memory=models.Memory.COLD,<br>    ),<br>)</pre><p>This “warm hybrid” pattern is the recommended large-scale setup: quantized vectors pinned in RAM for fast HNSW traversal, original vectors cold on disk only accessed for the final rescore of a tiny candidate set. Disk I/O happens for maybe 200 vectors per query (the rescore candidates), not for the thousands of comparisons during graph traversal.</p><p><strong>On migrating old configs</strong> — the old flags still work (deprecated but not removed), and the translation is straightforward:</p><pre># Old way<br>models.VectorParams(size=768, distance=models.Distance.COSINE, on_disk=True)<br># New way<br>models.VectorParams(size=768, distance=models.Distance.COSINE, memory=models.Memory.COLD<br><br># Old way<br>models.ScalarQuantizationConfig(type=models.ScalarType.INT8, always_ram=True)<br># New way<br>models.ScalarQuantizationConfig(type=models.ScalarType.INT8, memory=models.Memory.PINNED)</pre><p><strong>You can also update tiers without recreating a collection:</strong></p><pre>client.update_collection(<br>    collection_name=&quot;my_collection&quot;,<br>    vectors_config={<br>        &quot;my_vector&quot;: models.VectorParamsDiff(memory=models.Memory.COLD)<br>    },<br>)</pre><p>Yup, hot change. No migration needed.</p><p><strong>The cost math is real.</strong> RAM runs roughly $3–8/GB/month in cloud VMs. NVMe SSD is around $0.10–0.30/GB/month. That’s a 20–30x price difference. For a billion 768-dim float32 vectors (~2.9 TB), going from all-pinned to cold originals + pinned int8 quantized drops your RAM requirement from ~3.7 TB to ~768 GB. At $5/GB/month that’s the difference between $18,500/month and $3,840/month in RAM costs alone.</p><h3>4. Head-to-Head: How Competitors Handle This</h3><p>Hmm, so how does this stack up against the other major vector databases?</p><p><strong>Weaviate</strong> has the most comparable quantization story. They support PQ, BQ, SQ, and their own Rotational Quantization (RQ) which is architecturally similar to TurboQuant (rotation before compression). Their HFresh index does the compressed HNSW in memory + posting lists on disk split. Solid. But — Weaviate always keeps uncompressed vectors for rescoring. There’s no equivalent to Qdrant’s Turbo4 “store only the compressed version, discard the original” mode. If you need maximum disk efficiency, Weaviate can’t match Qdrant’s ~9x reduction at 4-bit.</p><p>For memory tiering, Weaviate’s HNSW index is entirely in RAM by default (2–12 GB for 1M vectors). HFresh moves it to disk but that’s a specific index type choice, not a flexible tier parameter. You can’t independently set your HNSW to cached while keeping vectors cold - you have to pick a different index architecture entirely.</p><p><strong>Milvus</strong> supports 8-bit scalar quantization (IVF_SQ8 index type) and product quantization (IVF_PQ), but nothing sub-8-bit. No 4-bit, no 2-bit, no TurboQuant-style rotation. For memory, they have mmap settings you can configure per-component type, plus DiskANN for on-disk graph search. It works, but it’s scattered across multiple config knobs in different places, nothing as unified as Qdrant’s single memoryparameter on every component.</p><p><strong>Pinecone</strong> has essentially zero user-configurable quantization. They make these decisions inside their infrastructure. No PQ, no SQ, no BQ, no compression knob of any kind. If you’re on serverless (which is the only option for new customers now), you also have no control over memory placement, Pinecone manages tiering opaquely. Fine for simple use cases, but you’re flying blind on cost optimization.</p><p><strong>Chroma</strong> — no quantization at all. Great for getting started, but not a real option for storage-constrained production deployments.</p><p>The quick summary:</p><p>FeatureQdrantWeaviateMilvusPineconeSub-8-bit quantizationYes (4-bit, 2-bit, 1-bit)Yes (1-bit RQ)NoNoRescore-free compressed-only storageYes (Turbo4)NoNoN/AUnified per-component memory tier APIYesNoNoNo</p><h3>5. Running It Yourself</h3><p>Alright, let’s actually run this. Here’s a complete demo,local Qdrant via Docker, create a Turbo4 collection, compare storage numbers across formats, and see the memory tier patterns in action.</p><p><strong>Start Qdrant locally:</strong></p><pre>docker run -p 6333:6333 -p 6334:6334 \<br>    -v $(pwd)/qdrant_storage:/qdrant/storage:z \<br>    qdrant/qdrant</pre><p>Qdrant is now running at http://localhost:6333. The web UI is at <a href="http://localhost:6333/dashboard.">http://localhost:6333/dashboard.</a></p><p><strong>Install the client:</strong></p><pre>pip install qdrant-client numpy</pre><p><strong>Full demo script:</strong></p><pre>import numpy as np<br>import time<br>from qdrant_client import QdrantClient, models<br><br>client = QdrantClient(&quot;http://localhost:6333&quot;)<br>DIM = 1536<br>N_VECTORS = 10_000<br># Collection 1: float32 baseline<br>client.recreate_collection(<br>    collection_name=&quot;float32_collection&quot;,<br>    vectors_config=models.VectorParams(<br>        size=DIM,<br>        distance=models.Distance.COSINE,<br>    ),<br>)<br># Collection 2: Turbo4 datatype - 4-bit only, no float32 copy<br>client.recreate_collection(<br>    collection_name=&quot;turbo4_collection&quot;,<br>    vectors_config=models.VectorParams(<br>        size=DIM,<br>        distance=models.Distance.COSINE,<br>        datatype=models.Datatype.TURBO4,<br>    ),<br>)<br># Collection 3: Hybrid - cold originals + pinned int8 for RAM-efficient high recall<br>client.recreate_collection(<br>    collection_name=&quot;hybrid_collection&quot;,<br>    vectors_config=models.VectorParams(<br>        size=DIM,<br>        distance=models.Distance.COSINE,<br>        memory=models.Memory.COLD,<br>    ),<br>    hnsw_config=models.HnswConfigDiff(memory=models.Memory.CACHED),<br>    quantization_config=models.ScalarQuantization(<br>        scalar=models.ScalarQuantizationConfig(<br>            type=models.ScalarType.INT8,<br>            quantile=0.99,<br>            memory=models.Memory.PINNED,<br>        ),<br>    ),<br>    payload_storage_config=models.PayloadStorageConfig(memory=models.Memory.COLD),<br>)<br># Generate random vectors<br>vectors = np.random.rand(N_VECTORS, DIM).astype(np.float32)<br>ids = list(range(N_VECTORS))<br>def upload_vectors(collection_name, vectors, ids, batch_size=500):<br>    for i in range(0, len(ids), batch_size):<br>        batch_ids = ids[i:i+batch_size]<br>        batch_vecs = vectors[i:i+batch_size].tolist()<br>        client.upsert(<br>            collection_name=collection_name,<br>            points=[<br>                models.PointStruct(id=pid, vector=vec)<br>                for pid, vec in zip(batch_ids, batch_vecs)<br>            ],<br>        )<br>    print(f&quot;  Uploaded {len(ids)} vectors to {collection_name}&quot;)<br>print(&quot;Uploading vectors...&quot;)<br>upload_vectors(&quot;float32_collection&quot;, vectors, ids)<br>upload_vectors(&quot;turbo4_collection&quot;, vectors, ids)<br>upload_vectors(&quot;hybrid_collection&quot;, vectors, ids)<br># Query latency benchmark<br>query = np.random.rand(DIM).astype(np.float32).tolist()<br>def bench_query(collection_name, n_queries=100):<br>    start = time.perf_counter()<br>    for _ in range(n_queries):<br>        client.search(<br>            collection_name=collection_name,<br>            query_vector=query,<br>            limit=10,<br>        )<br>    elapsed = (time.perf_counter() - start) / n_queries * 1000<br>    return elapsed<br>print(&quot;\nQuery latency (avg over 100 queries):&quot;)<br>for name in [&quot;float32_collection&quot;, &quot;turbo4_collection&quot;, &quot;hybrid_collection&quot;]:<br>    ms = bench_query(name)<br>    print(f&quot;  {name}: {ms:.2f} ms&quot;)<br># Recall comparison vs float32 brute force<br>print(&quot;\nRecall@10 comparison (vs float32):&quot;)<br>exact_results = client.search(<br>    collection_name=&quot;float32_collection&quot;,<br>    query_vector=query,<br>    limit=10,<br>)<br>exact_ids = set(r.id for r in exact_results)<br>for name in [&quot;turbo4_collection&quot;, &quot;hybrid_collection&quot;]:<br>    results = client.search(<br>        collection_name=name,<br>        query_vector=query,<br>        limit=10,<br>    )<br>    returned_ids = set(r.id for r in results)<br>    recall = len(exact_ids &amp; returned_ids) / len(exact_ids)<br>    print(f&quot;  {name}: {recall:.2f}&quot;)</pre><p>Run this and you’ll see latency and recall numbers across all three setups. The hybrid collection (cold originals + pinned int8) typically gets close to float32 recall while using much less RAM. Turbo4 will show lower recall on this small dataset — the gains really show at scale where storage costs dominate.</p><p><strong>Expected storage comparison for 10,000 x 1536-dim vectors:</strong></p><p>FormatApprox disk usagefloat32~58 MBfloat32 + int8 quant (hybrid)~73 MBTurbo4 only~7.5 MB</p><p>At 10M vectors, scale those numbers by 1000. The Turbo4 savings become very real very fast.</p><p><strong>Quick decision guide before you go:</strong></p><ul><li>Storage is your primary constraint, approximate recall is fine -&gt; database:turbo</li><li>Large collection, high recall required, some RAM available -&gt; cold vectors + pinned quantized (hybrid pattern)</li><li>Small collection, latency is everything, fits in RAM -&gt; memory: pinnedon everything</li><li>Rarely queried archival data -&gt; memory:cold on everything, NVMe SSD strongly recommended</li></ul><p>So yeah, 1.19 is a solid release for anyone running Qdrant at scale. The unified memory tier API alone is worth the upgrade just for operational clarity. Turbo4 is the cherry on top for storage-heavy workloads.</p><p>The full <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant documentation</a> has detailed coverage of quantization options and the memory tier reference. If you’re running production workloads, <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> handles the operational side so you can focus on tuning rather than managing nodes. And for major releases and new quantization research as it drops, the <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant newsletter</a> is worth subscribing to — Turbo4 specifically came out of a Google Research paper and the team moves fast on this stuff.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50b64a2fe0ef" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/qdrant-1-19-turboquant-memory-tiers-store-more-spend-less-50b64a2fe0ef">Qdrant 1.19 | TurboQuant &amp; Memory Tiers: Store More, Spend Less</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Slice Filters & Per-Tenant IDF: Search That Actually Makes Sense for Multi-Tenant Apps | Feat…]]></title>
            <link>https://pub.towardsai.net/slice-filters-per-tenant-idf-search-that-actually-makes-sense-for-multi-tenant-apps-feat-aaefda0655fd?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/aaefda0655fd</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 20:21:33 GMT</pubDate>
            <atom:updated>2026-09-05T18:44:31.449Z</atom:updated>
            <content:encoded><![CDATA[<h3>Slice Filters &amp; Per-Tenant IDF: Search That Actually Makes Sense for Multi-Tenant Apps | Feat Qdrant</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OqaQBh7eMdEg6dFaA_dLDQ.png" /><figcaption>Made by Author</figcaption></figure><p>So you’ve built a SaaS product. You’ve got dozens (or hundreds) of customers, each with their own dataset, and you’re using <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> as your vector store. Smart move. But then you hit the multi-tenancy wall with one collection, many tenants, and search results that are… off.</p><p>YUPP! Not broken, just subtly wrong in ways that are hard to explain to customers.</p><p>Qdrant 1.19 ships two features that directly address this. Slice filters and pertenant IDF. They sound unrelated but they solve two sides of the same problem, making search and data operations work correctly when your collection is shared across many tenants.</p><p>Let’s get into it.</p><h3>1. The Multi-Tenant Mess</h3><p>Okay so the standard advice for multi-tenancy in Qdrant is: put everyone in one collection, add a tenant_id field to each point&#39;s payload, filter on it at query time. Easy, scalable, works.</p><pre>client.scroll(<br>    collection_name=&quot;documents&quot;,<br>    scroll_filter=models.Filter(<br>        must=[<br>            models.FieldCondition(<br>                key=&quot;tenant_id&quot;,<br>                match=models.MatchValue(value=&quot;acme_corp&quot;),<br>            )<br>        ]<br>    ),<br>)</pre><p>This works great for retrieval isolation. Tenant A never sees Tenant B’s data. But here’s the thing, the moment you add keyword search (BM25/sparse vectors) to the mix, something breaks quietly in the background.</p><p>BM25 scoring depends on IDF, Inverse Document Frequency. The idea is simple: rare terms should score higher than common ones. A document matching “cryoablation” is probably more relevant than one matching “the”. IDF quantifies this rarity. The formula looks like:</p><pre>IDF(t) = log( (N - df(t) + 0.5) / (df(t) + 0.5) + 1 )</pre><p>Where N is total documents and df(t) is how many documents contain term t. When df(t) is small relative to N, IDF is high. Makes sense.</p><p>Now here’s where it goes wrong. Say you have two tenants sharing a collection:</p><ul><li><strong>Acme Corp</strong> (medical software): 10,000 clinical notes. The word “diagnosis” appears in 9,800 of them.</li><li><strong>Legal Firm LLC</strong> (contract analysis): 10,000 contract documents. The word “diagnosis” appears in 3 of them.</li></ul><p>Global IDF calculation:</p><pre>df(&quot;diagnosis&quot;) = 9,803 total documents<br>N = 20,000<br>IDF(&quot;diagnosis&quot;) ≈ log(1.027) + 1 ≈ 1.03</pre><p>That’s basically zero weight. “Diagnosis” looks like a stopword globally. So when Acme Corp’s users search for “diagnosis criteria”, a totally meaningful query in their domain.</p><p>The word “diagnosis” gets near-zero weight and results degrade. Meanwhile Legal Firm’s 3 documents mentioning “diagnosis” lose their appropriate high-rarity scoring because Acme’s data inflated the document frequency count.</p><p>Both tenants get wrong results and neither can figure out why. hmm..?</p><p>Elasticsearch has this problem too, by the way. Their solution is either one index per tenant (expensive) or dfs_query_then_fetch (adds a round-trip). Qdrant 1.19 does something cleaner.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*96vKgLuN9x8y4CB7.gif" /></figure><h3>2. Per-Tenant IDF: BM25 Scoring That Actually Makes Sense</h3><p>Qdrant 1.19 adds an “idf” parameter to “search_[arams”. You pass it a filter, and that filter defines which documents are included in the IDF statistics computation for that query. It&#39;s computed at query time, tenant-specific, and totally independent from your retrieval filter.</p><p>First, setup. Your sparse vector field needs the idf modifier:</p><pre>client.create_collection(<br>    collection_name=&quot;documents&quot;,<br>    vectors_config={},<br>    sparse_vectors_config={<br>        &quot;text-bm25&quot;: models.SparseVectorParams(<br>            modifier=models.Modifier.IDF,<br>        )<br>    },<br>)</pre><p>And your tenant field needs a payload index (Qdrant Cloud strict mode will reject unindexed filter fields):</p><pre>client.create_payload_index(<br>    collection_name=&quot;documents&quot;,<br>    field_name=&quot;tenant_id&quot;,<br>    field_schema=models.KeywordIndexParams(<br>        type=models.KeywordIndexType.KEYWORD,<br>        is_tenant=True,<br>    ),<br>)</pre><p>Now the actual query with per-tenant IDF:</p><pre>results = client.query_points(<br>    collection_name=&quot;documents&quot;,<br>    query=models.Document(text=&quot;diagnosis criteria&quot;, model=&quot;qdrant/bm25&quot;),<br>    using=&quot;text-bm25&quot;,<br>    query_filter=models.Filter(<br>        must=[<br>            models.FieldCondition(<br>                key=&quot;tenant_id&quot;,<br>                match=models.MatchValue(value=&quot;acme_corp&quot;),<br>            ),<br>            models.FieldCondition(<br>                key=&quot;year&quot;,<br>                match=models.MatchValue(value=2024),<br>            ),<br>        ]<br>    ),<br>    search_params=models.SearchParams(<br>        idf=models.IdfCorpusParams(<br>            corpus=models.Filter(<br>                must=[<br>                    models.FieldCondition(<br>                        key=&quot;tenant_id&quot;,<br>                        match=models.MatchValue(value=&quot;acme_corp&quot;),<br>                    ),<br>                ]<br>            )<br>        )<br>    ),<br>    limit=10,<br>)</pre><p>Notice the intentional difference between the two filters. The query_filter narrows retrieval to acme&#39;s 2024 documents only. The idf corpus filter covers ALL of acme&#39;s documents - not just 2024. That&#39;s deliberate. You want IDF to reflect acme&#39;s full vocabulary so the rarity statistics are meaningful, even though you&#39;re only retrieving a subset.</p><p>The filter layering looks like this:</p><pre>retrieval filter  ⊆  IDF corpus filter  ⊆  full collection<br>  (acme + 2024)        (acme only)           (all tenants)</pre><p><strong>Concrete before-&gt;after comparison:</strong></p><p>Say acme has 10,000 docs and “diagnosis” appears in 9,800 of them, while “criteria” appears in only 200. With global IDF across both tenants, both terms get flattened because the legal corpus pulls “criteria” frequency up and the combined N changes everything.</p><p>Query termGlobal IDF (20k docs)</p><p>Per-Tenant IDF (acme only, 10k docs)</p><p>”diagnosis”~1.03 (near-zero)~1.03 (same it’s common in acme too)”</p><p>criteria”~2.1 (inflated df from legal docs)~4.6 (correctly rare in acme)”</p><p>cryoablation”~9.2 (rare globally)~9.2 (same legal barely touches this)</p><p>The real win is for terms that are common in one tenant’s domain but rare in another’s. Those are exactly the domain-specific terms that matter most for relevance. Without per-tenant IDF, those terms get scored incorrectly every time.</p><p>Yup, the fix is a single parameter. No extra round-trips, no index-per-tenant overhead, no client-side hacks.</p><h3>3. Slice Filters: Deterministic Splits for Parallel Work</h3><p>Okay, different problem now. You need to process your entire collection re-embed everything with a new model, export data to a new system, compute statistics, run evaluations. The naive approach: scroll with a cursor, page through sequentially.</p><p>That works, but it’s slow for large collections, and if your job fails halfway through, you have to figure out where you left off. Want to parallelize? Now you need a coordinator to hand out cursor ranges, deal with workers finishing at different speeds, handle failures without double-processing.</p><p>Honestly, that’s a lot of scaffolding for what should be a simple parallelism problem.</p><p>Slice filters are Qdrant’s answer. A slice divides the entire collection into total disjoint buckets using deterministic hashing on point IDs, and lets you select exactly one bucket with index. The key properties:</p><ul><li><strong>Deterministic</strong>: same {index, total} always returns the same points</li><li><strong>Disjoint</strong>: no overlap between slices, union of all slices = full collection</li><li><strong>Stable</strong>: no randomness, no seeds the mapping is fixed</li></ul><p>The filter in Python:</p><pre>results, next_offset = client.scroll(<br>    collection_name=&quot;product_embeddings&quot;,<br>    scroll_filter=models.Filter(<br>        must=[<br>            models.SliceCondition(<br>                slice=models.Slice(index=3, total=8)<br>            )<br>        ]<br>    ),<br>    limit=500,<br>    with_payload=True,<br>    with_vectors=True,<br>)</pre><p>Now here’s the parallel worker pattern. Four workers, each handling exactly 1/4 of the collection, zero coordination needed:</p><pre>import concurrent.futures<br>from qdrant_client import QdrantClient, models<br><br>COLLECTION = &quot;product_embeddings&quot;<br>TOTAL_WORKERS = 4<br>BATCH_SIZE = 500<br><br>def process_slice(slice_index: int, total_slices: int) -&gt; list:<br>    # Each worker creates its own client - thread-safe this way<br>    client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br>    <br>    offset = None<br>    processed = []<br>    <br>    while True:<br>        results, next_offset = client.scroll(<br>            collection_name=COLLECTION,<br>            scroll_filter=models.Filter(<br>                must=[<br>                    models.SliceCondition(<br>                        slice=models.Slice(index=slice_index, total=total_slices)<br>                    )<br>                ]<br>            ),<br>            limit=BATCH_SIZE,<br>            offset=offset,<br>            with_payload=True,<br>            with_vectors=True,<br>        )<br>        <br>        if not results:<br>            break<br>        <br>        for point in results:<br>            # your actual work here: re-embed, export, transform, etc.<br>            processed.append({&quot;id&quot;: point.id, &quot;payload&quot;: point.payload})<br>        <br>        if next_offset is None:<br>            break<br>        offset = next_offset<br>    <br>    print(f&quot;Worker {slice_index} done: {len(processed)} points&quot;)<br>    return processed<br><br><br>with concurrent.futures.ThreadPoolExecutor(max_workers=TOTAL_WORKERS) as executor:<br>    futures = {<br>        executor.submit(process_slice, i, TOTAL_WORKERS): i<br>        for i in range(TOTAL_WORKERS)<br>    }<br>    all_results = []<br>    for future in concurrent.futures.as_completed(futures):<br>        all_results.extend(future.result())<br><br>print(f&quot;Total processed: {len(all_results)}&quot;)</pre><p>Each worker is completely independent. Worker 0 failing doesn’t affect Workers 1, 2, or 3. If Worker 2 crashes, you just re-run it with index=2, total=4 and it picks up the same points again from scratch. No shared cursor, no coordination overhead. Pretty wild right?</p><p>Note: within a single slice, you still use cursor pagination (next_page_offset). Slice filter doesn&#39;t replace pagination inside the slice - it just bounds the working set for each worker. So the pattern is: slice filter to split work, cursor to page through each worker&#39;s portion.</p><p>Wait, what about reproducible sampling? Same deal. Want a consistent 10% evaluation sample? Use Slice(index=0, total=10). It always returns the same 10%. Want a train/test split? Slices 0-7 for training, slices 8 and 9 for test. Run your pipeline six months later on the same data? Same split. That&#39;s actually pretty handy compared to ORDER BY RANDOM() LIMIT n approaches that give you a different sample every time.</p><p>One important caveat: if points are added or deleted between runs, slice membership can shift because hash assignments change as the collection evolves. The slice is stable for a point’s lifetime, not across collection mutations. For migration jobs where the source is read-only, this isn’t a problem. For ongoing pipelines on live data, design accordingly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*hHv6R7WWd9qWSR0K.gif" /></figure><h3>4. Putting It Together</h3><p>Here’s a realistic scenario where both features matter at the same time.</p><p>You’re running a multi-tenant document intelligence platform. Customers upload their internal knowledge bases legal teams, medical teams, product teams and users search across them. You’re also running nightly jobs to re-embed everything with a new model and compute per-tenant quality metrics.</p><p><strong>The search side</strong> uses per-tenant IDF so domain-specific terminology scores correctly for each tenant. A legal tenant searching “force majeure clause” gets IDF computed over their legal corpus, not diluted by the medical tenant’s completely different vocabulary. The medical tenant searching “differential diagnosis protocol” gets their medical terms scored against their corpus.</p><p><strong>The batch processing side</strong> uses slice filters to parallelize the nightly re-embedding job across N workers. Each worker independently processes its slice, can be restarted independently on failure. No coordinator needed, no shared state, horizontal scaling is as simple as increasing the total count and spinning up more workers.</p><p>You can even combine both in a single query. Slice within a tenant for stratified sampling:</p><pre># Get a reproducible 25% sample of just acme&#39;s documents<br>client.scroll(<br>    collection_name=&quot;documents&quot;,<br>    scroll_filter=models.Filter(<br>        must=[<br>            models.SliceCondition(slice=models.Slice(index=0, total=4)),<br>            models.FieldCondition(<br>                key=&quot;tenant_id&quot;,<br>                match=models.MatchValue(value=&quot;acme_corp&quot;),<br>            ),<br>        ]<br>    ),<br>    limit=500,<br>)</pre><p>This gives you 25% of acme’s documents deterministically. Useful for canary testing a new embedding model on a subset of a tenant’s data before full rollout.</p><h3>5. Running It Yourself</h3><p>Here’s a complete demo you can run locally. Start Qdrant first:</p><pre>docker run -p 6333:6333 -p 6334:6334 \<br>  -v $(pwd)/qdrant_storage:/qdrant/storage:z \<br>  qdrant/qdrant</pre><p>Install the client:</p><pre>pip install qdrant-client</pre><p>Then this script creates a two-tenant collection, inserts sample documents, and demonstrates both per-tenant IDF search syntax and slice-based parallel processing:</p><pre>import concurrent.futures<br>from qdrant_client import QdrantClient, model<br><br>client = QdrantClient(url=&quot;http://localhost:6333&quot;<br>&#39;&#39;<br><br># Setup collection with sparse vectors + IDF modifier<br>client.recreate_collection(<br>    collection_name=&quot;demo&quot;,<br>    vectors_config={},<br>    sparse_vectors_config={<br>        &quot;text-bm25&quot;: models.SparseVectorParams(<br>            modifier=models.Modifier.IDF,<br>        )<br>    },<br><br><br>client.create_payload_index(<br>    collection_name=&quot;demo&quot;,<br>    field_name=&quot;tenant_id&quot;,<br>    field_schema=models.KeywordIndexParams(<br>        type=models.KeywordIndexType.KEYWORD,<br>        is_tenant=True,<br>    ),<br><br><br># Insert sample documents for two tenants<br>medical_docs = [<br>    {&quot;id&quot;: i, &quot;tenant&quot;: &quot;medical&quot;, &quot;text&quot;: f&quot;Clinical case {i}: diagnosis and treatment protocol for patient&quot;}<br>    for i in range(1, 201)<br>]<br>medical_docs += [<br>    {&quot;id&quot;: 201, &quot;tenant&quot;: &quot;medical&quot;, &quot;text&quot;: &quot;Hospital billing and insurance reimbursement procedures&quot;},<br>    {&quot;id&quot;: 202, &quot;tenant&quot;: &quot;medical&quot;, &quot;text&quot;: &quot;Staff scheduling and shift management system&quot;},<br><br><br>legal_docs = [<br>    {&quot;id&quot;: i + 300, &quot;tenant&quot;: &quot;legal&quot;, &quot;text&quot;: f&quot;Contract analysis {i}: force majeure clause and indemnification terms&quot;}<br>    for i in range(1, 201)<br>]<br>legal_docs += [<br>    {&quot;id&quot;: 501, &quot;tenant&quot;: &quot;legal&quot;, &quot;text&quot;: &quot;Personal injury case: patient diagnosis injuries from accident&quot;},<br>]all_docs = medical_docs + legal_docs<br># NOTE: In production, sparse vectors would be populated by BM25 tokenization<br># (fastembed, Qdrant Cloud inference, or your own tokenizer).<br># For this local demo we insert with empty vectors to show the API structure.<br>points = [<br>    models.PointStruct(<br>        id=doc[&quot;id&quot;],<br>        payload={&quot;tenant_id&quot;: doc[&quot;tenant&quot;], &quot;text&quot;: doc[&quot;text&quot;]},<br>        vector={},<br>    )<br>    for doc in all_docs<br>]<br>client.upsert(collection_name=&quot;demo&quot;, points=points)<br>print(f&quot;Inserted {len(points)} documents&quot;)<br><br># Demonstrate slice filter: parallel scroll across 4 workers<br>def count_slice(slice_index, total):<br>    c = QdrantClient(url=&quot;http://localhost:6333&quot;)<br>    count = 0<br>    offset = None<br>    while True:<br>        results, next_offset = c.scroll(<br>            collection_name=&quot;demo&quot;,<br>            scroll_filter=models.Filter(<br>                must=[<br>                    models.SliceCondition(<br>                        slice=models.Slice(index=slice_index, total=total)<br>                    )<br>                ]<br>            ),<br>            limit=100,<br>            offset=offset,<br>            with_payload=False,<br>            with_vectors=False,<br>        )<br>        count += len(results)<br>        if next_offset is None:<br>            break<br>        offset = next_offset<br>    return count<br>total_slices = 4<br>with concurrent.futures.ThreadPoolExecutor(max_workers=total_slices) as executor:<br>    futures = {<br>        executor.submit(count_slice, i, total_slices): i<br>        for i in range(total_slices)<br>    }<br>    slice_counts = {}<br>    for future in concurrent.futures.as_completed(futures):<br>        idx = futures[future]<br>        slice_counts[idx] = future.result()<br>print(&quot;\nSlice distribution across 4 workers:&quot;)<br>for idx in sorted(slice_counts):<br>    print(f&quot;  Slice {idx}: {slice_counts[idx]} points&quot;)<br>print(f&quot;  Total: {sum(slice_counts.values())} (should equal {len(points)})&quot;)<br><br># Per-tenant IDF query - this requires BM25 sparse vectors to be populated.<br># On Qdrant Cloud with cloud inference enabled, the vector is generated automatically<br># from the Document text at query time.<br>print(&quot;&quot;&quot;<br>Per-tenant IDF query (requires BM25 sparse vectors populated via cloud inference):<br>results = client.query_points(<br>    collection_name=&quot;demo&quot;,<br>    query=models.Document(text=&quot;diagnosis criteria&quot;, model=&quot;qdrant/bm25&quot;),<br>    using=&quot;text-bm25&quot;,<br>    query_filter=models.Filter(<br>        must=[<br>            models.FieldCondition(<br>                key=&quot;tenant_id&quot;,<br>                match=models.MatchValue(value=&quot;medical&quot;),<br>            )<br>        ]<br>    ),<br>    search_params=models.SearchParams(<br>        idf=models.IdfCorpusParams(<br>            corpus=models.Filter(<br>                must=[<br>                    models.FieldCondition(<br>                        key=&quot;tenant_id&quot;,<br>                        match=models.MatchValue(value=&quot;medical&quot;),<br>                    )<br>                ]<br>            )<br>        )<br>    ),<br>    limit=5,<br>)<br>&quot;&quot;&quot;)</pre><p>Run this and you’ll see the slice distribution across 4 workers. It won’t be perfectly equal (hash distribution variance) but it’ll be approximately total_docs / 4 per slice.</p><p>For the full BM25 search demo with actual scored results, sign up for <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> which handles BM25 sparse vector generation at ingestion time via cloud inference no separate embedding step needed.</p><p>The <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant docs</a> have the full reference on both features including the REST API format and edge cases around combining slice with nested filters (short version: put the slice condition in an outer must, not inside a nested block).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*GqGVgvNASCmvs562.gif" /></figure><p>So yeah, two features that solve real problems. Per-tenant IDF fixes a subtle but meaningful bug in BM25 scoring for shared collections, the kind of thing that’s hard to diagnose but clearly degrades search quality in multi-domain setups. Slice filters give you a clean primitive for parallel batch work that scales horizontally without coordination overhead.</p><p>Both are available now in <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant 1.19</a>. If you want to stay on top of what’s coming in future releases, the <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant newsletter</a> is worth subscribing to the release notes are actually written by engineers and tend to be pretty honest about tradeoffs.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aaefda0655fd" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/slice-filters-per-tenant-idf-search-that-actually-makes-sense-for-multi-tenant-apps-feat-aaefda0655fd">Slice Filters &amp; Per-Tenant IDF: Search That Actually Makes Sense for Multi-Tenant Apps | Feat…</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[miniCOIL = BM25 + 4D Meaning Vectors | Adding Semantics to Lexical Search | Hybrid Search]]></title>
            <link>https://pub.towardsai.net/sparse-yet-smart-fixing-bm25-with-4d-meaning-vectors-minicoil-10x-better-rag-1e50169e0649?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e50169e0649</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Tue, 24 Mar 2026 06:45:28 GMT</pubDate>
            <atom:updated>2026-03-27T10:21:52.310Z</atom:updated>
            <content:encoded><![CDATA[<h3>Sparse, Yet Smart: Fixing BM25 with 4D Meaning Vectors | miniCOIL</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3noHpTy0Nl0PuQYcfGCMDA.png" /><figcaption>Illustration by Author x Gemini</figcaption></figure><p>BM25: the algorithm that sees the word ‘bank’ and genuinely cannot tell if you’re drowning or <strong>broke</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/192/0*rjNo12O081PPSUPR.gif" /></figure><p>Let me tell you something that might sting a little if you’ve spent time building search systems!!!</p><p>BM25 the algorithm that powers lexical search in OpenSearch, Elasticsearch, and most enterprise search systems… it doesn’t understand a single word it processes. Not one. It counts words. It weighs their rarity. It ranks documents by those counts. But it has no idea what any word actually means.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*J-0xvcO6yb5OmC3S.gif" /></figure><p>For most queries, this works fine. You search for “python tutorial” and BM25 finds documents with those exact words. Great.</p><p>But the moment language gets ambiguous. the moment “search” means something different in two different documents, the moment “vector” refers to epidemiology in one paragraph and linear algebra in another… BM25 fails. Silently. Confidently. Without warning.</p><p>miniCOIL fixes this. And it does it in a way that’s surprisingly elegant.</p><h3>BM25: What It Actually Does *Source (Research)</h3><p>BM25 =&gt; Best Match 25 is a ranking function from the 1990s that has aged remarkably well for what it is. The formula looks like this:</p><p>score(D,Q)=∑qi∈QIDF(qi)×TF(qi,D)score(<em>D</em>,<em>Q</em>)=<em>qi</em>​∈<em>Q</em>∑​IDF(<em>qi</em>​)×TF(<em>qi</em>​,<em>D</em>)</p><p>Two components drive everything:</p><p><strong>IDF (Inverse Document Frequency)</strong></p><p>How rare is this word across all documents? A word like “the” appears everywhere, so its IDF is low. A word like “galacto” appears rarely, so its IDF is high. Rare words are more informative, so they get more weight in the final score.</p><p><strong>TF (Term Frequency)</strong></p><p>How many times does this word appear in this specific document? More occurrences suggest higher relevance, up to a point. BM25 applies a saturation function so you can’t game it by repeating words endlessly.</p><p>That’s the whole model. Two numbers per word, multiplied together, summed across all query terms.</p><p>It’s fast, it’s interpretable, and for keyword-heavy queries on consistent corpora it works well. OpenSearch runs it on millions of documents in milliseconds. BM25 is a genuinely good algorithm for what it was designed to do.</p><p>The problem is what it was designed to do….. in the first place??</p><p>BM25 treats every occurrence of a word as identical. The word <strong>“search”</strong> in a document about meditation gets the exact same IDF weight as the word <strong>“search”</strong> in a document about Elasticsearch. The word <strong>“vector”</strong> in a linear algebra textbook is computationally indistinguishable from the word <strong>“vector”</strong> in an epidemiology paper about mosquito-borne diseases.</p><p>BM25 does not see context. It sees only words.</p><h3>The Homonym Problem: Where BM25 Always Fails</h3><p>This is not an edge case. It’s the normal state of language. Most common words have multiple meaningfully different senses depending on context.</p><p>Consider what happens when you query: <strong>“vector disease control mosquito”</strong></p><p>BM25 will score any document containing the word “vector.” That includes:</p><ul><li>A public health paper about mosquito-borne disease vectors (correct)</li><li>A machine learning article about vector databases (wrong)</li><li>A physics textbook chapter on vector addition (wrong)</li><li>A linear algebra course on eigenvectors (wrong)</li></ul><p>All of these contain “vector.” BM25 has no way to distinguish between them based on what “vector” actually means in each one. It just counts. And counts equally.</p><p>Or try: <strong>“searching for inner peace and self discovery”</strong></p><p>BM25 matches every document containing “search” or “searching.” A document about search engine optimization scores the same keyword match as a document about spiritual self-discovery. The model cannot see the difference between “search” as a technical database operation and “search” as a human metaphor for meaning.</p><p>WordContext A Context BM25 tells them apart? searchsearch engine, indexing, retrieval, searching the soul, mindfulness vector database, embeddings, AI vector control, mosquito, disease vector calculus, gradient, equations, vector database, embeddings interest rate, monetary policy, river bank, erosion, flood plain, plant factory, manufacturing, equipment, flower, leaves, photo synthesis</p><p>Language is deeply ambiguous and BM25 ignores every bit of that ambiguity. Not because it was poorly designed, but because it was designed before the tools existed to do better.</p><p>Those tools exist now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*wb13C4bnuOm20sMz.gif" /></figure><h3>miniCOIL: The Idea</h3><p><a href="https://huggingface.co/Qdrant/minicoil-v1">miniCOIL</a> is a sparse retrieval model developed by <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> that extends BM25’s formula with one critical addition: <strong>contextual meaning</strong>.</p><p>The core insight is simple. When you index a word, you have access to all the surrounding words.</p><p>You know whether “vector” appears next to “database,”</p><p>“embeddings, AI versus disease, control, mosquito versus calculus, gradient, equations.”</p><p>That surrounding context tells you which sense of the word is being used in this document, right here, in this specific occurrence.</p><p>miniCOIL captures this by computing a <strong>4-dimensional contextual fingerprint</strong> for each word occurrence. Four numbers. That sounds impossibly small… but it’s enough. Most words with multiple meanings have only a handful of genuinely distinct senses. Four dimensions can separate them.</p><p>At query time, the same process runs on the query. When the query “vector disease control mosquito” meets a document about mosquito-borne disease vectors, the word “vector” has similar fingerprints in both the query and the document. When it meets a document about vector databases, the fingerprints are different. The match score for that word drops.</p><h3>The Formula: One Extra Term Changes Everything</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N5fXiIbj-8hP1hQeLrCe7g.png" /><figcaption>Author x Gemini</figcaption></figure><p>Three factors now:</p><ul><li><strong>IDF</strong>: How rare is this word across the entire collection? (same as BM25)</li><li><strong>Importance</strong>: How significant is this word in this document? (similar to TF, but learned)</li><li><strong>Meaning</strong>: Do the query and document use this word in the <strong>same sense</strong>?</li></ul><p>The Meaning term is a dot product of the 4-dim contextual fingerprint of the word in the query versus the same word in the document. Same word in the same context means high Meaning score. Same word in a completely different context means low Meaning score. The entire discriminating power of context awareness is captured in that one dot product.</p><p>That’s the revolution. Not a completely new retrieval paradigm. Just one extra term in a formula you’ve been using for decades.</p><h3>Why 4 Dimensions Is Enough</h3><p>You might wonder: why just 4 dimensions for the contextual fingerprint? That sounds impossibly small for capturing word meaning.</p><p>The key is that miniCOIL is not trying to encode all semantic information. It’s not a general-purpose embedding model. It’s only trying to differentiate between the different <strong>senses</strong> of words that share the same surface form.</p><p>For most ambiguous words, the number of meaningfully distinct senses is small. “Vector” has maybe 5 or 6 genuinely different contexts. “Search” has a handful. “Bank” has two dominant ones. A 4-dimensional space can distinguish between a small number of clusters, and that’s all that’s needed here.</p><p>This design choice has a beautiful consequence: miniCOIL stays <strong>sparse</strong>. It produces sparse vectors just like BM25, with one value per vocabulary term. You don’t need to run a full dense embedding model. The vocabulary size matches the standard BM25 vocabulary. The index size is comparable.</p><p>You get contextual disambiguation at roughly the cost of BM25. That’s the engineering elegance of miniCOIL.</p><p>miniCOIL was trained on <strong>OpenWebText</strong>, 40 million sentences of diverse general web text. This is a deliberate choice: training on a domain-specific corpus like MS MARCO (information retrieval queries) would bias the model toward that domain. OpenWebText gives miniCOIL broad word sense coverage across general language, which is why it generalizes well to out-of-domain queries.</p><h3>The IDF Modifier: Why This Needs Qdrant</h3><p>Here’s where the infrastructure gets interesting, and where most vector databases fall short.</p><p>miniCOIL produces sparse vectors where the stored values are the Importance × Meaning factors per term. The IDF factor is intentionally <strong>not</strong> baked into those stored values.</p><p>Why? Because IDF changes as your collection evolves. Add new documents, delete old ones… the frequency of each term across the corpus shifts. If you bake IDF into the stored vectors at index time, you’d need to re-index every single document whenever any document is added or deleted. That’s not a retrieval system. That’s a batch pipeline pretending to be one.</p><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> solves this with a single parameter: modifier=Modifier.IDF.</p><pre>from qdrant_client import QdrantClient<br>from qdrant_client import models<br>client = QdrantClient(&quot;:memory:&quot;)<br>client.create_collection(<br>    collection_name=&quot;minicoil&quot;,<br>    sparse_vectors_config={<br>        &quot;minicoil&quot;: models.SparseVectorParams(<br>            modifier=models.Modifier.IDF<br>        )<br>    },<br>)</pre><p>With that one line, Qdrant tracks document frequencies dynamically and applies IDF at query time using live collection statistics. Documents can be added or deleted freely, and the IDF values stay correct automatically. No re-indexing jobs. No stale approximations. No pipeline to maintain.</p><p>OpenSearch and Elasticsearch don’t have an equivalent. There’s no IDF modifier for custom sparse vectors in either system. You’d have to compute IDF yourself from the full corpus, multiply it into the sparse vector values at index time, and re-index everything whenever the corpus changes. It is technically doable. It’s also the kind of thing that breaks silently in production when someone uploads a new document batch and nobody reruns the IDF computation.</p><p>Qdrant does it correctly, natively, always up to date. This is not a minor implementation detail. It’s what makes miniCOIL viable as a production retriever rather than a research prototype.</p><h3>FastEmbed: Running miniCOIL Without the Setup Tax</h3><p>miniCOIL (Qdrant/minicoil-v1) is natively integrated into <a href="https://github.com/qdrant/fastembed">FastEmbed</a>, the lightweight embedding library from the <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> team.</p><p>FastEmbed handles ONNX-based inference for a wide range of embedding models, including miniCOIL, without requiring PyTorch setup, CUDA configuration, or tokenizer management. For miniCOIL specifically, the Qdrant client’s Document class integrates directly with FastEmbed under the hood:</p><pre># Index time — FastEmbed handles the miniCOIL inference internally<br>models.Document(<br>    text=document_text,<br>    model=&quot;Qdrant/minicoil-v1&quot;,<br>    options={&quot;avg_len&quot;: avg_document_length},  # BM25 length normalization<br>)<br># Query time<br>models.Document(<br>    text=query_text,<br>    model=&quot;Qdrant/minicoil-v1&quot;<br>)</pre><p>You don’t load the model yourself. You don’t manage tokenizers. You just pass a Document object and Qdrant calls FastEmbed, runs the inference, and handles the rest. The avg_len parameter feeds BM25-style document length normalization, computed once from your corpus before ingestion.</p><p>For developers who want to add miniCOIL to an existing pipeline, the barrier to entry is almost zero. Install qdrant-client[fastembed] and you have everything you need.</p><h3>Qdrant: Built for This Kind of Search</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> was built from the ground up, in Rust, with AI-scale search in mind. Sparse vectors, dense vectors, and hybrid search are all first-class citizens, not afterthoughts bolted onto a general-purpose database.</p><p>The modifier=Modifier.IDF feature is a good example of how Qdrant thinks about retrieval correctness. They didn&#39;t just add sparse vector storage. They thought through the full scoring pipeline and built the infrastructure for it to work correctly in a dynamic, live collection.</p><p>Recent releases have made this even more powerful for real-world deployments.</p><p><strong>Qdrant 1.16</strong> introduced <strong>ACORN</strong>, a significant improvement to filtered vector search. When you combine semantic retrieval with metadata filters (searching only within a specific knowledge base, or within documents from a certain date range), traditional HNSW implementations can silently degrade. The filtered neighborhood becomes sparse and the graph traversal starts missing relevant results without reporting any errors.</p><p>ACORN fixes this by examining neighbors-of-neighbors in the HNSW graph when direct neighbors are filtered out. The benchmark numbers make the case plainly: <strong>97.2% accuracy with ACORN versus 53.3% without it</strong> on queries with multiple low-selectivity filters. For any production deployment where users are searching within scoped subsets of a knowledge base, this is what makes the system trustworthy.</p><p><strong>Qdrant 1.16</strong> also brought <strong>Conditional Updates</strong>, which allow version-aware point modifications to prevent concurrent conflicts. When multiple writers update a live miniCOIL collection simultaneously, this keeps the index consistent without needing external locking.</p><p><strong>Qdrant 1.17</strong> shipped <strong>Weighted RRF</strong> for hybrid queries. If you’re combining miniCOIL’s sparse retrieval signal with a dense semantic model (a strong combination for production RAG), you can now assign different importance weights to each signal. More sparse weight for keyword-heavy queries, more dense weight for conceptual queries. Tunable at query time, no retraining required.</p><p><strong>Qdrant 1.17</strong> also brought <strong>Relevance Feedback Query</strong>, which enables agentic refinement loops. A system can issue an initial miniCOIL query, score the results against task criteria, feed back which results were relevant, and get a refined result set. No retraining, no re-embedding, no model calls. Pure retrieval-time refinement using the vector structure.</p><p>The <a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant documentation</a> covers sparse vectors, hybrid search, the IDF modifier, and everything related to miniCOIL deployment.</p><h3>Real Numbers: The BEIR Benchmark</h3><p>Here’s where the rubber meets the road. BEIR is a standard benchmark for evaluating retrieval quality across diverse datasets. NDCG@10 measures ranking quality where higher is better.</p><p>DatasetBM25miniCOILWinnerMS MARCO0.2370.244miniCOILNQ0.3040.319miniCOILQuora0.7840.802miniCOILFiQA-20180.2520.257miniCOILHotpotQA0.6340.633BM25 (tie)</p><p>miniCOIL wins or ties on all five datasets with zero domain-specific training. It was trained on general web text, not on MS MARCO or any of these specific domains, and it still outperforms BM25 consistently.</p><p>The Quora gains are particularly notable: +0.018 NDCG@10. Quora is full of paraphrased questions where the same intent is expressed many different ways, and word sense disambiguation matters a lot for matching synonymous queries to relevant answers. miniCOIL’s contextual fingerprints help identify when two different phrasings are asking about the same thing, even when the exact words differ.</p><h3>The Complete Code</h3><h3>Install Dependencies</h3><pre>pip install qdrant-client[fastembed] rank-bm25 streamlit</pre><h3>The Corpus</h3><pre>DOCUMENTS = [<br>    &quot;The Art of Search and Self-Discovery&quot;,<br>    &quot;Searching the Soul: A Journey Through Mindfulness&quot;,<br>    &quot;In Search of Meaning: A Psychological Perspective&quot;,<br>    &quot;The Human Search for Connection in a Digital World&quot;,<br>    &quot;Search Engines: A Technical and Social Overview&quot;,<br>    &quot;Search Optimization Strategies for E-commerce&quot;,<br>    &quot;The Rise of Vector Databases in AI Systems&quot;,<br>    &quot;Efficient Vector Search Algorithms for Large Datasets&quot;,<br>    &quot;Vector Control Strategies in Public Health&quot;,<br>    &quot;Vectors in Physics: From Arrows to Equations&quot;,<br>]</pre><p>Ten documents. The words “search” and “vector” appear in very different contexts across them. A perfect corpus for showing what BM25 gets wrong and what miniCOIL gets right.</p><h3>Building the BM25 Index</h3><pre>from rank_bm25 import BM25Okapi<br>def build_bm25(documents):<br>    tokenized = [doc.lower().split() for doc in documents]<br>    return BM25Okapi(tokenized)<br>def bm25_search(bm25, documents, query: str, n: int = 5):<br>    scores = bm25.get_scores(query.lower().split())<br>    ranked = sorted(enumerate(scores), key=lambda x: -x[1])<br>    return [(documents[i], round(s, 4)) for i, s in ranked[:n]]</pre><p>Simple. Fast. Context-blind.</p><h3>Building the miniCOIL Index</h3><pre>from qdrant_client import QdrantClient<br>from qdrant_client import models<br>def build_minicoil(documents):<br>    client = QdrantClient(&quot;:memory:&quot;)<br>    client.create_collection(<br>        collection_name=&quot;minicoil&quot;,<br>        sparse_vectors_config={<br>            &quot;minicoil&quot;: models.SparseVectorParams(<br>                modifier=models.Modifier.IDF    # dynamic IDF from live collection stats<br>            )<br>        },<br>    )<br>    # Compute avg_len for BM25-style length normalization<br>    avg_len = sum(len(d.split()) for d in documents) / len(documents)<br>    client.upsert(<br>        collection_name=&quot;minicoil&quot;,<br>        points=[<br>            models.PointStruct(<br>                id=i,<br>                payload={&quot;text&quot;: documents[i]},<br>                vector={<br>                    &quot;minicoil&quot;: models.Document(<br>                        text=documents[i],<br>                        model=&quot;Qdrant/minicoil-v1&quot;,<br>                        options={&quot;avg_len&quot;: avg_len},<br>                    )<br>                },<br>            )<br>            for i in range(len(documents))<br>        ],<br>    )<br>    return client<br><br>def minicoil_search(client, query: str, n: int = 5):<br>    hits = client.query_points(<br>        collection_name=&quot;minicoil&quot;,<br>        query=models.Document(text=query, model=&quot;Qdrant/minicoil-v1&quot;),<br>        using=&quot;minicoil&quot;,<br>        limit=n,<br>        with_payload=True,<br>    ).points<br>    return [(h.payload[&quot;text&quot;], round(h.score, 4)) for h in hits]</pre><h3>Running the Comparison</h3><pre>bm25   = build_bm25(DOCUMENTS)<br>client = build_minicoil(DOCUMENTS)<br>query = &quot;vector disease control mosquito&quot;<br>print(f&quot;Query: &#39;{query}&#39;\n&quot;)<br>print(&quot;BM25 results:&quot;)<br>for text, score in bm25_search(bm25, DOCUMENTS, query, n=5):<br>    print(f&quot;  [{score}] {text}&quot;)<br>print(&quot;\nminiCOIL results:&quot;)<br>for text, score in minicoil_search(client, query, n=5):<br>    print(f&quot;  [{score}] {text}&quot;)</pre><p>Expected output:</p><pre>BM25 results:<br>  [0.6123] Vector Control Strategies in Public Health    ← correct<br>  [0.5891] The Rise of Vector Databases in AI Systems    ← wrong<br>  [0.5712] Efficient Vector Search Algorithms...         ← wrong<br>  [0.5543] Vectors in Physics: From Arrows to Equations  ← wrong<br>  [0.3201] Search Engines: A Technical and Social Overview<br>miniCOIL results:<br>  [0.8932] Vector Control Strategies in Public Health    ← correct, clearly ranked first<br>  [0.2341] Vectors in Physics: From Arrows to Equations<br>  [0.1892] The Rise of Vector Databases in AI Systems<br>  ...</pre><p>BM25 returns all documents containing “vector” with similar scores, unable to distinguish between epidemiology and computer science. miniCOIL puts the public health document clearly at the top because the contextual fingerprint for “vector” next to “disease,” “control,” and “mosquito” matches correctly.</p><h3>Second Query: The Introspective Search</h3><pre>query = &quot;searching for inner peace and self discovery&quot;<br>print(&quot;BM25 results:&quot;)<br>for text, score in bm25_search(bm25, DOCUMENTS, query, n=5):<br>    print(f&quot;  [{score}] {text}&quot;)<br>print(&quot;\nminiCOIL results:&quot;)<br>for text, score in minicoil_search(client, query, n=5):<br>    print(f&quot;  [{score}] {text}&quot;)</pre><p>BM25 treats “search” the same across all 10 documents. miniCOIL sees “search” next to “inner,” “peace,” “self,” “discovery” and builds a philosophical fingerprint that aligns with the introspective documents at the top of the corpus.</p><h3>Where miniCOIL Fits in Your Stack</h3><p>miniCOIL is not a replacement for dense semantic search. It’s a better BM25, which means it’s the right lexical component in a hybrid retrieval system.</p><p>The ideal production setup:</p><p>User Input -&gt; Query</p><ol><li>Input: Query</li><li>Branch A (Dense): Query -&gt; Dense model (e.g., BGE, E5) -&gt; Dense vector -&gt; Dense search</li><li>Branch B (Sparse): Query -&gt; miniCOIL -&gt; Sparse vector -&gt; Sparse search</li><li>Fusion: (Dense results + Sparse results) -&gt; Qdrant Weighted RRF fusion</li><li>Output: Final ranked results</li></ol><p>This gives you the best of both worlds: dense retrieval for conceptual queries, miniCOIL for keyword precision with context awareness, and Qdrant’s Weighted RRF (from 1.17) to tune how much each signal contributes depending on your use case.</p><p>If your current system uses BM25 + dense search on OpenSearch, the path to miniCOIL is to add <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> as the retrieval layer. Run miniCOIL and your dense model there. Your existing application logic continues as-is, with better retrieval quality underneath.</p><h3>Conclusion</h3><p>BM25 is a great algorithm for what it is: a fast, interpretable keyword matcher. It’s served the search industry well for decades and will keep doing so for a long time.</p><p>But it was built for a world where “vector” only meant one thing and “search” only meant one thing. That world doesn’t exist in natural language.</p><p>miniCOIL adds the one ingredient BM25 was always missing: an understanding of what a word means in context. Not full semantic understanding, just enough to distinguish between senses. And it does this while staying sparse, staying fast, and staying compatible with the BM25 scoring framework that search systems have been built on for years.</p><p>One extra term in the formula. A 4-dimensional fingerprint per word. And suddenly your search system stops confusing the disease vector with the math vector.</p><p>That’s not magic. That’s just a really well-designed model.</p><p><strong>Want to run this yourself?</strong></p><p><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> gives you a managed Qdrant cluster with native sparse vector support and the IDF modifier working out of the box. Free tier, no credit card, spins up in minutes. The same code from this article connects to it directly.</p><p>Keep up with what <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> ships next by <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">subscribing to their newsletter</a>. The sparse vector roadmap alone is worth following.</p><ul><li><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Home</a> — the vector database for AI-scale search</li><li><a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Documentation</a> — sparse vectors, hybrid search, IDF modifier, all documented in detail</li><li><a href="https://github.com/qdrant/fastembed">FastEmbed on GitHub</a> — lightweight embedding library, miniCOIL supported natively</li><li><a href="https://huggingface.co/Qdrant/minicoil-v1">miniCOIL on HuggingFace</a> — model card, training details, benchmarks</li><li><a href="https://qdrant.tech/blog/qdrant-1.17.x/">Qdrant 1.17 Release</a> — Weighted RRF, Relevance Feedback, latency improvements</li><li><a href="https://qdrant.tech/blog/qdrant-1.16.x/">Qdrant 1.16 Release</a> — ACORN, Conditional Updates, full-text enhancements</li><li><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud Sign Up</a> — free tier, managed, zero ops</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e50169e0649" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/sparse-yet-smart-fixing-bm25-with-4d-meaning-vectors-minicoil-10x-better-rag-1e50169e0649">miniCOIL = BM25 + 4D Meaning Vectors | Adding Semantics to Lexical Search | Hybrid Search</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Prod Scale Visual Document Retrieval with ColModernVBERT and Qdrant for VLMs]]></title>
            <link>https://pub.towardsai.net/prod-scale-visual-document-retrieval-with-colmodernvbert-and-qdrant-for-vlms-4e98dfc75e99?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e98dfc75e99</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Fri, 20 Mar 2026 09:37:26 GMT</pubDate>
            <atom:updated>2026-03-25T04:26:56.108Z</atom:updated>
            <content:encoded><![CDATA[<h4>“Your RAG system can read. But can it see?”</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yv-exkWEXoitCfoOGeYSNg.png" /><figcaption>Feat Qdrant x Gemini</figcaption></figure><p>Let me start with a situation you’ve probably run into.</p><p>You have a report. Dense with charts, tables, maybe a few diagrams. You build a RAG pipeline around it. You ask a question about what the bar chart shows in section 3. And the system returns the paragraph <em>below</em> the chart, the one that says “as shown in Figure 3 above.”</p><p>The actual chart? Gone. Converted to nothing by your OCR pipeline.</p><p>This is not a bug. This is the fundamental limitation of text-based document retrieval. And it affects every RAG system that relies on extracting text from documents before embedding it.</p><p>There’s a better way now. And it starts by treating every page as what it actually is: an image.</p><h3>The Problem with OCR-Based RAG</h3><p>Traditional document RAG pipelines work like this: take a PDF, extract text, chunk it, embed the chunks, store them in a vector database, retrieve them at query time. Sounds reasonable. Works decently for text-heavy documents.</p><p>But a surprising amount of the world’s most important documents are not just text. They’re charts. Tables. Scanned slides. Handwritten notes. Diagrams with labels. Complex multi-column layouts. Formulas. Figures. And when you run these through OCR, here’s what you get:</p><ul><li>That quarterly revenue chart? Becomes the axis labels and maybe a title.</li><li>That financial table? Becomes a chaotic mix of numbers stripped of their row and column context.</li><li>That scanned contract page? Might return empty or garbled text depending on scan quality.</li><li>That architecture diagram with callouts? Returns the callout text floating in space with no structure.</li></ul><p>OCR errors don’t announce themselves. They just silently corrupt your retrieval. And the LLM at the end of the pipeline answers from that corrupted context, confidently wrong.</p><p>The fix is to stop treating documents as text and start treating them as images. That’s exactly what the ColPali family of models does.</p><h3>How ColPali Started: PaliGemma and the Heavy Approach</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*a_1BGteKEEzQssbW.png" /><figcaption><a href="https://developers.googleblog.com/gemma-explained-paligemma-architecture/">SOURCE</a></figcaption></figure><p>The ColPali idea came from a clean insight: why not use a multimodal vision-language model to understand document pages as images, generate patch-level embeddings from them, and retrieve using those embeddings?</p><p>The original <strong>ColPali</strong> model used <strong>PaliGemma</strong> as its backbone. PaliGemma is Google’s vision-language model built on top of Gemma, and it’s genuinely excellent at visual understanding. The ColPali approach takes a PDF page, renders it as an image, feeds it through PaliGemma’s vision encoder, and produces a set of 128-dimensional vectors… one per image patch.</p><p>It worked really well on the ViDoRe benchmark. NDCG@5 scores around 85 put it ahead of anything text-based.</p><p>The catch? PaliGemma is a <strong>3 billion parameter model</strong>. It’s a decoder-based VLM, which means it wasn’t designed for fast embedding. Running it on a CPU is painful. Even on a GPU, embedding hundreds of pages takes time. Memory requirements are around 6 GB just to load the model.</p><p>For a retrieval encoder that needs to process thousands of pages and serve real-time queries, that’s heavy. Really heavy.</p><p>Then came ColQwen2, which pushed the ViDoRe scores even higher by using Qwen as the backbone… at 7 billion parameters and 14 GB of memory.</p><p>The trend was going in the wrong direction. Better accuracy, but increasingly impractical for anything outside a well-resourced GPU cluster.</p><p>This is where ColModernVBERT comes in.</p><h3>Meet ColModernVBERT: The Lean Alternative</h3><p><a href="https://huggingface.co/ModernVBERT/colmodernvbert">ColModernVBERT</a> swaps the heavy VLM backbone for <strong>ModernBERT</strong>, an encoder-only architecture based on BERT’s principles but modernized with better attention mechanisms, longer context, and improved training.</p><p>The result: <strong>250 million parameters</strong>. About 12 times smaller than ColPali. Around 28 times smaller than ColQwen2.</p><p>And the retrieval quality is remarkably close. On ViDoRe, ColModernVBERT scores around 83 NDCG@5 versus ColPali’s 85 and ColQwen2’s 87. For most real-world applications, that 2 to 4-point gap is negligible. The inference speed and memory difference, on the other hand, is not negligible at all.</p><p>Yeah… that’s a pretty stark difference. ColModernVBERT fits in RAM you’d barely notice. ColQwen2 needs a dedicated GPU and some optimism.</p><blockquote>Why does this matter?</blockquote><p>Because retrieval is not a one-time operation. You embed pages at ingestion time, and you embed queries at search time. Every. Single. Query. A 250M encoder completes that in milliseconds. A 7B decoder takes seconds. At scale, this is the difference between a system that works and one that technically works but makes users wonder if something is <strong>broken</strong>.</p><h3>Understanding “Col”: Late Interaction and Why It Matters</h3><p>The “Col” prefix in ColModernVBERT, ColPali, ColBERT… it stands for <strong>Contextualized Late Interaction</strong>. Let me explain what that actually means, because it’s the core architectural idea that makes all of this work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U0r65BJA-jczmPDpu1K2ng.png" /><figcaption>Feat Gemini Nano Banana</figcaption></figure><h3>What Are Embeddings?</h3><p>Before late interaction, let’s ground the concept of embeddings.</p><p>An embedding is a fixed-size vector of numbers that represents the “meaning” of something in a high-dimensional space. Two things that are semantically similar end up close together in that space. Two things that are semantically different end up far apart.</p><p>For text, this is well-understood. You train a model on massive text and it learns that “dog” and “puppy” should be close, while “dog” and “refrigerator” should be far.</p><p>For vision, the same idea applies at the patch level. An image patch containing the number “$4.2M” in a chart gets projected into a vector space. A query token asking about “Q3 revenue” gets projected into the same space. If the model was trained well, those two vectors will be close.</p><p>This is why the approach works without OCR. The model doesn’t need to extract the text from the image. It needs the visual signal of that patch to be semantically close to the query token. And if training was done well, it will be.</p><h3>Standard Dense Retrieval (the old way)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/358/0*iG4S1NPOSn2qQG0z.gif" /></figure><p>In regular dense retrieval, you embed a whole document into a single vector. One number per dimension, one vector per document. 768 dimensions, one vector. A whole page of content collapsed into a single point in space.</p><pre># Traditional dense retrieval<br>page_embedding = mean_pool(encoder(page_image))  # 1 × 768-dim, all detail lost<br>query_embedding = mean_pool(encoder(query))       # 1 × 768-dim<br>score = cosine_similarity(query_embedding, page_embedding)</pre><p>When you query, you find nearby document vectors. The problem is information loss. When you collapse everything into one vector, spatial relationships are gone. Token-level details are averaged away. A query about a specific number in a table has to somehow match a vector that represents the entire page. That’s a lot to ask of one 768-dimensional point.</p><h3>Late Interaction (the ColBERT way)</h3><p>Late interaction keeps one vector per token, and delays the interaction between query and document until scoring time.</p><p>For ColModernVBERT:</p><ul><li>Every <strong>image patch</strong> in a page gets its own <strong>128-dimensional vector</strong></li><li>Every <strong>query token</strong> gets its own <strong>128-dimensional vector</strong></li><li>Scoring uses <strong>MaxSim</strong>: for each query token, find the single most similar patch, take that cosine similarity score, then sum across all query tokens</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/277/1*HdwRYE2eqrURskTasuyvjA.png" /></figure><p>This is elegant. The query token for “revenue” will find the patch that contains the revenue bar in your chart. The query token for “2024” will find the patch containing that year label. Each token finds its best match independently, and the total score reflects how many query concepts are visually present in the page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*KdtTtJOmykjl9UwR.gif" /></figure><p>The detail that was averaged away in mean-pooling? It’s all preserved here, living in individual patch vectors, waiting to be found by the right query token.</p><h3>ColModernVBERT Architecture</h3><p>ColModernVBERT is built on <strong>ModernBERT</strong>, an encoder-only transformer. This is a deliberate architectural choice. Decoders (like PaliGemma and Qwen) generate text token by token, which makes them powerful for answering but slow for embedding. Encoders process the entire input in one forward pass and produce representations directly. They’re built for this.</p><pre>PDF page (image)<br>  └─► split into patches (16×16 pixel regions)<br>        └─► ModernBERT encoder<br>              └─► linear projection → 128-dim per patch<br>                    └─► stored as MultiVector in Qdrant</pre><pre>Text query<br>  └─► tokenize<br>        └─► ModernBERT encoder<br>              └─► linear projection → 128-dim per token<br>                    └─► MaxSim against all stored patches</pre><p>The model was fine-tuned on the <strong>ViDoRe</strong> benchmark specifically for visual document retrieval, which means its shared embedding space for image patches and text tokens was explicitly trained so that the “right” patches align with the “right” query tokens. It knows that a bar chart patch should match a question about trends. That a table cell should match a question about specific values.</p><h3>FastEmbed: The Easy Path to Embeddings</h3><p>If you’ve ever set up a local embedding pipeline, you know the setup tax. Install PyTorch, figure out CUDA, manage ONNX conversions, handle model downloads, debug tokenizer configurations…</p><p><a href="https://github.com/qdrant/fastembed">FastEmbed</a> is the answer to all of that. It’s a lightweight Python library from the <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> team that makes running embedding models locally as simple as possible.</p><pre>from fastembed import TextEmbedding<br>model = TextEmbedding(&quot;BAAI/bge-large-en-v1.5&quot;)<br>embeddings = list(model.embed([&quot;some text&quot;, &quot;more text&quot;]))</pre><p>ONNX-based inference, no GPU required for most models, handles downloads automatically. For the visual document retrieval pipeline described in this article, FastEmbed sits alongside the colpali_engine package for the ColModernVBERT-specific processing. But for any text embedding tasks in your broader RAG pipeline, FastEmbed is worth having in your toolkit.</p><p>Open source, actively maintained, and built with production use in mind. Worth a star on GitHub.</p><h3>The Two-Stage Pipeline: Retrieval Then Understanding</h3><p>ColModernVBERT handles retrieval beautifully. But retrieval is only half the job in a RAG system. The other half is answering the question from what you retrieved.</p><p>This is where we add a <strong>Vision Language Model (VLM)</strong> as the second stage. The pipeline looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*2bPluUHYNApLdKKlRddSDA.jpeg" /><figcaption>Feat Elon Musk ;-)</figcaption></figure><p>Stage 1 is about <strong>scale and speed</strong>. ColModernVBERT’s 250M encoder processes pages fast and returns the right pages from a collection of thousands.</p><p>Stage 2 is about <strong>depth and understanding</strong>. The VLM actually reads the page images visually. It can read the chart. It can parse the table. It can interpret the diagram. It answers as a human reader would.</p><p>Using a 4B VLM for retrieval over thousands of pages would be prohibitively slow. Generating embeddings autoregressively is orders of magnitude more expensive than an encoder forward pass. Using ColModernVBERT alone would give you page rankings but no actual answers. Together, they cover the full pipeline cleanly.</p><pre># What the VLM receives — raw pixel images, no text extraction<br>messages = [{<br>    &quot;role&quot;: &quot;user&quot;,<br>    &quot;content&quot;: &quot;Answer using only what you see in these document pages. Question: ...&quot;,<br>    &quot;images&quot;: [base64(page_1), base64(page_2), base64(page_3)]  # raw pixels<br>}]</pre><p>The LLM context here is page images, not garbled OCR output. It reads from the same fidelity a human reader would.</p><h3>Qdrant: The Vector Database That Gets MultiVector Right</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is where all those patch vectors live. And late interaction requires something most vector databases simply don’t support.</p><p>When you have 128-dim vectors for every patch in a page, you’re not storing one vector per document. You’re storing dozens or hundreds of vectors per document. A typical page might have 64 patches. A 100-page document becomes 6,400 vectors, all logically belonging to their respective pages.</p><p>Most vector databases handle this by storing one vector per record and making you manage the grouping yourself. That means duplicating the payload for every patch, handling retrieval aggregation in application code, and losing the native MaxSim scoring.</p><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> handles it natively with MultiVectorConfig:</p><pre>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    VectorParams, Distance,<br>    MultiVectorConfig, MultiVectorComparator<br>)</pre><pre>client = QdrantClient(&quot;:memory:&quot;)</pre><pre>client.create_collection(<br>    collection_name=&quot;colmodernvbert&quot;,<br>    vectors_config={<br>        &quot;colmodernvbert&quot;: VectorParams(<br>            size=128,<br>            distance=Distance.COSINE,<br>            multivector_config=MultiVectorConfig(<br>                comparator=MultiVectorComparator.MAX_SIM<br>            ),<br>        )<br>    },<br>)</pre><p>One point per page. All patch vectors stored together. MaxSim scoring applied automatically at query time. That’s it. No application-side aggregation, no custom scoring code, no workarounds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*buxMBu3LX7SpLneE" /></figure><h3>Qdrant 1.16: ACORN and Inline Storage</h3><p><strong>ACORN</strong> is one of the most practically important improvements in vector search in recent releases. When you filter retrieval by metadata (only search within a specific document, or only pages uploaded after a certain date), traditional HNSW implementations can degrade badly. The filtered neighborhood becomes sparse, the graph traversal misses relevant nodes, and recall drops quietly without you noticing.</p><p>ACORN fixes this by examining neighbors-of-neighbors in the HNSW graph when direct neighbors are filtered out. The benchmark numbers are stark: <strong>97.2% accuracy</strong> with ACORN versus <strong>53.3%</strong> without it on multi-filter queries. For a document retrieval system where you’re often filtering by document ID or metadata, this is what makes the system trustworthy rather than just fast.</p><p><strong>Inline Storage</strong> stores quantized vector data directly inside HNSW nodes, enabling disk-based search at 10x better QPS in memory-constrained environments. The benchmark showed 211 QPS versus 20 QPS in constrained RAM conditions. For large document collections with thousands of pages each, this means you can scale to millions of page vectors without a machine with hundreds of gigabytes of RAM.</p><h3>Qdrant 1.17: Weighted RRF and Relevance Feedback</h3><p><strong>Weighted RRF</strong> lets you assign different importance weights to different sub-queries in hybrid search. If you combine ColModernVBERT’s visual similarity signal with a BM25 text signal from extracted headers, you can tune how much each contributes. More visual weight for image-heavy documents, more text weight for text-dense reports.</p><p><strong>Relevance Feedback Query</strong> enables agentic refinement loops. Once the model returns initial results, you can feed back which results were relevant, and Qdrant adjusts the scoring for a refined retrieval pass. No retraining, no re-embedding. Pure search-time refinement using positive and negative examples from the initial result set.</p><p>All of this is available in <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a>, their managed deployment, which spins up in minutes with a free tier. No infrastructure to manage, same QdrantClient API. If you&#39;ve been putting off setting up a proper vector database because of ops overhead, this is the reason to stop putting it off.</p><h3>The Full Code</h3><h3>Loading the Model</h3><pre>import torch<br>from colpali_engine.models import ColModernVBert, ColModernVBertProcessor<br>MODEL_ID = &quot;ModernVBERT/colmodernvbert&quot;<br>processor = ColModernVBertProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)<br>model = ColModernVBert.from_pretrained(<br>    MODEL_ID,<br>    torch_dtype=torch.float32,<br>    trust_remote_code=True,<br>)<br>model.eval()</pre><p>Note: requires transformers==4.53.x and colpali_engine from the vbert branch:</p><pre>pip install &quot;git+https://github.com/illuin-tech/colpali.git@vbert&quot;<br>pip install &quot;transformers==4.53.3&quot;</pre><h3>Converting PDF Pages to Images</h3><pre>import pymupdf as fitz<br>from PIL import Image<br>def pdf_to_images(pdf_bytes: bytes) -&gt; list[Image.Image]:<br>    doc = fitz.open(stream=pdf_bytes, filetype=&quot;pdf&quot;)<br>    images = []<br>    for page in doc:<br>        # 1.5x upscale for better patch resolution<br>        pix = page.get_pixmap(matrix=fitz.Matrix(1.5, 1.5))<br>        images.append(<br>            Image.frombytes(&quot;RGB&quot;, [pix.width, pix.height], pix.samples)<br>        )<br>    return images</pre><p>No OCR. No text extraction. The page becomes an image and stays that way.</p><h3>Embedding Pages</h3><pre>import numpy as np<br>def embed_images(images: list[Image.Image]) -&gt; list[np.ndarray]:<br>    &quot;&quot;&quot;Returns list of (n_patches, 128) arrays - one per page.&quot;&quot;&quot;<br>    embeddings = []<br>    for img in images:<br>        inputs = processor.process_images([img])<br>        with torch.no_grad():<br>            out = model(**inputs)          # (1, n_patches, 128)<br>        embeddings.append(out[0].cpu().numpy())<br>    return embeddings</pre><h3>Indexing into Qdrant</h3><pre>from qdrant_client.models import PointStruct<br>def index_pages(client, images: list[Image.Image], doc_name: str):<br>    points = []<br>    for i, img in enumerate(images):<br>        vecs = embed_images([img])[0]      # (n_patches, 128)<br>        points.append(<br>            PointStruct(<br>                id=i,<br>                vector={&quot;colmodernvbert&quot;: vecs.tolist()},<br>                payload={&quot;page&quot;: i, &quot;doc_name&quot;: doc_name},<br>            )<br>        )<br>    client.upsert(collection_name=&quot;colmodernvbert&quot;, points=points)</pre><p>Each page is one point. All its patch vectors are stored together. Qdrant handles the rest.</p><h3>Querying</h3><pre>def embed_query(query: str) -&gt; np.ndarray:<br>    &quot;&quot;&quot;Returns (n_tokens, 128) array.&quot;&quot;&quot;<br>    inputs = processor.process_texts([query])<br>    with torch.no_grad():<br>        out = model(**inputs)              # (1, n_tokens, 128)<br>    return out[0].cpu().numpy()<br><br>def search(client, query: str, limit: int = 3):<br>    q_vecs = embed_query(query)            # (n_tokens, 128)<br>    return client.query_points(<br>        collection_name=&quot;colmodernvbert&quot;,<br>        query=q_vecs.tolist(),<br>        using=&quot;colmodernvbert&quot;,<br>        limit=limit,<br>        with_payload=True,<br>    ).points</pre><h3>Answering with the VLM</h3><pre>import base64, json, io, requests<br>def ask_vlm(question: str, page_images: list[Image.Image]) -&gt; str:<br>    images_encoded = []<br>    for img in page_images:<br>        buf = io.BytesIO()<br>        img.save(buf, format=&quot;PNG&quot;)<br>        images_encoded.append(base64.b64encode(buf.getvalue()).decode())<br>    payload = {<br>        &quot;model&quot;: &quot;qwen3.5:4b-q8_0&quot;,<br>        &quot;messages&quot;: [{<br>            &quot;role&quot;: &quot;user&quot;,<br>            &quot;content&quot;: (<br>                &quot;You are a precise document assistant. &quot;<br>                &quot;Answer using only what you can see in the provided document pages. &quot;<br>                f&quot;Question: {question}&quot;<br>            ),<br>            &quot;images&quot;: images_encoded,<br>        }],<br>        &quot;stream&quot;: True,<br>        &quot;think&quot;: False,<br>        &quot;options&quot;: {&quot;temperature&quot;: 0.1, &quot;num_predict&quot;: 1024},<br>    }<br>    answer = &quot;&quot;<br>    with requests.post(<br>        &quot;http://127.0.0.1:11434/api/chat&quot;,<br>        json=payload, stream=True, timeout=120<br>    ) as resp:<br>        resp.raise_for_status()<br>        for line in resp.iter_lines():<br>            if not line:<br>                continue<br>            chunk = json.loads(line)<br>            answer += chunk.get(&quot;message&quot;, {}).get(&quot;content&quot;, &quot;&quot;)<br>            if chunk.get(&quot;done&quot;):<br>                break<br>    return answer.strip()</pre><p>Run Qwen3.5 locally via:</p><pre>ollama serve<br>ollama pull qwen3.5:4b-q8_0</pre><h3>Putting It All Together</h3><pre># 1. Load model<br>processor, model = load_model()<br>client = setup_qdrant_collection()<br># 2. Ingest a PDF<br>pdf_bytes = open(&quot;report.pdf&quot;, &quot;rb&quot;).read()<br>images = pdf_to_images(pdf_bytes)<br>index_pages(client, images, &quot;report.pdf&quot;)<br># 3. Ask a question<br>results = search(client, &quot;What does the revenue chart in Q3 show?&quot;, limit=3)<br>retrieved_pages = [images[r.payload[&quot;page&quot;]] for r in results]<br>answer = ask_vlm(&quot;What does the revenue chart in Q3 show?&quot;, retrieved_pages)<br>print(answer)</pre><h3>Why This Actually Matters</h3><p>Let me put this in practical terms.</p><p>Your document might contain a compliance table. The OCR version has garbled text. The text-based RAG system either misses the query or returns the wrong page. The analyst manually hunts through the document.</p><p>With this pipeline:</p><ol><li>ColModernVBERT looks at the page visually, finds the table, returns it</li><li>Qwen3.5 reads the table as a human would, gives a precise answer</li></ol><p>The difference is not incremental. It’s qualitative. You’ve moved from a system that processes text to one that reads documents.</p><p>For scanned documents, handwritten notes, slide decks, research papers with complex figures, financial reports full of charts… this is the pipeline that actually works. The OCR pipeline was always a workaround. This is the right approach.</p><h3>Conclusion</h3><p>ColPali proved the idea. ColModernVBERT made it practical. And <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> made it easy to deploy.</p><p>The 250M encoder gives you fast inference with only a marginal quality difference versus 3B VLM backbones. Qdrant’s MultiVector support handles the per-patch storage natively and applies MaxSim at query time without any custom code. Qwen3.5 running locally via Ollama closes the loop by reading the retrieved pages and generating grounded answers.</p><p>No OCR. No text extraction. No cascading failures from bad document parsing. Just a system that sees your documents the way a human would.</p><p>Try it. Upload a PDF with a chart or a table, ask a question about the visual content, and watch the model actually answer correctly. Yeah… it’s that satisfying.</p><p><strong>Want to take this further?</strong></p><p><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> gives you a managed vector database with MultiVector support, ACORN filtering, Weighted RRF, and all the latest features. Free tier, no credit card, no ops overhead. The same QdrantClient code from this article connects to it directly.</p><p>Stay up to date with what <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> ships next by <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">subscribing to their newsletter</a>. The team moves fast and the release notes are genuinely worth reading.</p><ul><li><a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Documentation</a> — MultiVector, ACORN, hybrid search, all covered in detail</li><li><a href="https://github.com/qdrant/fastembed">FastEmbed on GitHub</a> — lightweight embedding library from the Qdrant team</li><li><a href="https://qdrant.tech/blog/qdrant-1.17.x/">Qdrant 1.17 Release</a> — Relevance Feedback, Weighted RRF, tail latency improvements</li><li><a href="https://qdrant.tech/blog/qdrant-1.16.x/">Qdrant 1.16 Release</a> — ACORN, Inline Storage, full-text enhancements</li><li><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud Sign Up</a> — free tier, managed, zero ops</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e98dfc75e99" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/prod-scale-visual-document-retrieval-with-colmodernvbert-and-qdrant-for-vlms-4e98dfc75e99">Prod Scale Visual Document Retrieval with ColModernVBERT and Qdrant for VLMs</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Search Beyond Words: Multimodal AI with Gemini Embeddings 2 and Qdrant]]></title>
            <link>https://blog.stackademic.com/search-beyond-words-multimodal-ai-with-gemini-embeddings-2-and-qdrant-afd0c627aa0d?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/afd0c627aa0d</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Tue, 17 Mar 2026 08:28:06 GMT</pubDate>
            <atom:updated>2026-03-20T08:58:45.105Z</atom:updated>
            <content:encoded><![CDATA[<h3>Beyond Words: Building Multimodal Search with Gemini Embedding 2 and Qdrant</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pdat-fwJJ634A76FVNqMMg.png" /><figcaption>Illustration by Author Feat Nano Banana</figcaption></figure><blockquote>“A picture is worth a thousand words, they say. But what if your database has no idea what a picture even is?”</blockquote><p>Imagine you’re building a product discovery platform. Your users can browse through thousands of items photos, product descriptions, user reviews, even short demo videos. You’ve set up a decent keyword search. It works… ish. User types “red sneakers,” gets red sneakers. Great.</p><p>But then a user uploads a photo of their old shoes, worn down to the sole, and says: ==&gt; <em>“Find me something like this, but better.”</em></p><p>Your keyword search? It has absolutely no idea what to do with that.</p><p>This is the multimodal gap. And it’s everywhere e-commerce, healthcare, creative tools, media platforms, enterprise knowledge bases. The gap between “<em>how humans communicate”</em> (through images, text, sound, video, sketches) and <em>how search systems understand</em> (keywords, IDs, exact matches) has been growing for years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/242/0*KhCuOOBhDMI9kuLP.gif" /></figure><p>Until now.</p><p><strong>Gemini Embedding 2, </strong>Google’s latest multimodal embedding model and <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella"><strong>Qdrant</strong></a>, the vector database purpose-built for AI-scale search, are finally giving developers the tools to close that gap. Together, they make it possible to build search systems that understand <em>meaning</em> across all modalities: you can query with an image, retrieve with text, mix both, and get results that actually make sense.</p><p>Let’s build one, together. And let’s understand <em>why</em> it works while we’re at it.</p><h3>The Multimodal Problem: Why Text Search Is Living in the Past</h3><p>Let’s think about the data your applications actually deal with.</p><p>A retail platform doesn’t just have product names. It has images, style descriptions, customer photos, video walkthroughs. A healthcare system doesn’t just have patient notes. It has X-rays, MRI scans, annotated images, physician reports. A media company doesn’t just have articles. It has thumbnails, embedded clips, transcripts, audio.</p><p>Text search systems — even powerful semantic ones — treat everything that isn’t text as a second-class citizen. They index the alt-text on an image. They transcribe the video and index that. They throw away the actual visual information.</p><p>This creates three real problems:</p><p><strong>The vocabulary mismatch problem</strong>: A user searches for “comfortable running shoe with good grip.” Your product description says “high-traction athletic footwear.” A BM25 keyword system sees zero overlap. Even decent semantic search might miss this if the embeddings only capture literal text.</p><p><strong>The cross-modal retrieval problem</strong>: A user uploads a photo of a flower arrangement and asks “where can I buy something like this?” A text-only system has no pathway from image query to relevant text or product results. The modalities are walls, not windows.</p><p><strong>The rich media understanding problem</strong>: Increasingly, the most valuable information in enterprise systems lives in images, PDFs, diagrams, and videos. Indexing these by their surrounding text alone leaves enormous amounts of meaning on the table.</p><p>Multimodal embeddings solve all three. They project text, images, and other content into the <em>same vector space</em>, where semantic similarity is meaningful regardless of modality.</p><p>And Gemini Embedding 2 is the best reason yet to finally make the switch.</p><h3>Gemini Embedding 2: One Embedding Space to Rule Them All</h3><p>Google’s <strong>Gemini Embedding 2</strong> is a major leap over traditional single-modality embedding models. It’s a natively multimodal model, not a text encoder with a bolted-on image component, but a unified architecture trained to understand text, images, and their interplay as a single coherent representation space.</p><p>Here’s why that matters:</p><h3>Unified Embedding Space</h3><p>When you embed a photo of a golden retriever and the sentence “a fluffy golden dog playing in the grass,” both vectors land close together in the same high-dimensional space. This is the fundamental primitive that makes cross-modal search work.</p><p>With earlier approaches, you’d need separate models for images and text, then complex fusion logic to combine scores at retrieval time. Gemini Embedding 2 gives you a single vector per item — regardless of whether that item was an image, a text description, or both.</p><h3>High Dimensionality and Expressiveness</h3><p>Gemini Embedding 2 produces 3072<strong>-dimensional vectors</strong> by default (with configurable dimensionality at range 128–3072, with 768, 1536, or 3072 dimensions). Higher dimensions let the model capture more nuanced semantic information — the difference between “formal dress shoes” and “casual loafers” isn’t just vocabulary, it’s a rich constellation of visual and conceptual features that benefit from that representational capacity.</p><h3>Image + Text = Richer Queries</h3><p>One underrated feature: you can embed <em>both</em> an image and a text string together, and the model fuses them into a single vector. So a user who uploads a photo of a living room and types “but with more natural light” produces one embedding that captures both the visual context <em>and</em> the textual refinement.</p><pre>from vertexai.vision_models import MultiModalEmbeddingModel, Image as VertexImage<br>model = MultiModalEmbeddingModel.from_pretrained(&quot;multimodalembedding@001&quot;)<br># Text-only embedding<br>text_embedding = model.get_embeddings(<br>    contextual_text=&quot;a minimalist Scandinavian living room with warm tones&quot;<br>)<br># Image-only embedding<br>image_embedding = model.get_embeddings(<br>    image=VertexImage.load_from_file(&quot;room.jpg&quot;)<br>)<br># Multimodal: image + text context fused into one vector<br>fused_embedding = model.get_embeddings(<br>    image=VertexImage.load_from_file(&quot;room.jpg&quot;),<br>    contextual_text=&quot;but with more natural light and plants&quot;<br>)</pre><p>The fused_embedding here doesn&#39;t just concatenate — it produces a semantically coherent representation where both the image <em>and</em> the text modification pull the vector in meaningful directions. That&#39;s the superpower.</p><h3>Production-Ready via Vertex AI</h3><p>Gemini Embedding 2 is available through Google Cloud’s Vertex AI, which means it comes with enterprise-grade reliability, rate limits that scale, and seamless integration with the rest of the Google Cloud ecosystem. It also means you can embed images directly from Cloud Storage URIs… no need to download and re-upload content you’re already storing in GCS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/220/0*GUfCZGwSUwiy9FfU.gif" /></figure><pre>import vertexai<br>from vertexai.vision_models import MultiModalEmbeddingModel, Image as VertexImage<br>vertexai.init(project=&quot;your-project-id&quot;, location=&quot;us-central1&quot;)<br>model = MultiModalEmbeddingModel.from_pretrained(&quot;multimodalembedding@001&quot;)<br>def get_multimodal_embedding(<br>    image_path: str | None = None,<br>    text: str | None = None,<br>    dimension: int = 3072<br>) -&gt; list[float]:<br>    &quot;&quot;&quot;Generate a multimodal embedding from image, text, or both.&quot;&quot;&quot;<br>    kwargs = {&quot;dimension&quot;: dimension}<br>    if image_path:<br>        kwargs[&quot;image&quot;] = VertexImage.load_from_file(image_path)<br>    if text:<br>        kwargs[&quot;contextual_text&quot;] = text<br>    result = model.get_embeddings(**kwargs)<br>    return result.image_embedding if image_path else result.text_embedding</pre><p>Now: where do all these vectors live? That’s where <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> comes in.</p><h3><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a>: The Vector Database That Actually Gets It</h3><p>Let me be blunt: not all vector databases are created equal. <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is in a different tier.</p><p>Built from the ground up in Rust for memory efficiency and performance, <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> has become the go-to vector database for serious AI applications — and for good reason. It’s not just a fast ANN index with a REST API wrapper. It’s a production-grade system designed for the complexity of real-world AI workloads: hybrid search, filtered retrieval, multitenancy, distributed deployments, and now, increasingly, multimodal use cases.</p><p>Here’s what sets <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> apart, including what just landed in their most recent releases.</p><h3>Collections with Named Vectors: Built for Multimodal</h3><p>This is the feature that makes <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> particularly well-suited for multimodal search. A single point (document/record) in <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> can hold <strong>multiple named vectors simultaneously</strong>.</p><p>For our multimodal use case, this is huge. A product point can have:</p><ul><li>A &quot;text_vector&quot; — the embedding of the product description</li><li>An &quot;image_vector&quot; — the embedding of the product image</li><li>A &quot;fused_vector&quot; — the combined multimodal embedding</li></ul><p>You can then search across any or all of these, or run a multi-vector query that scores against several simultaneously.</p><pre>from qdrant_client import QdrantClient<br>from qdrant_client.models import VectorParams, Distance<br>client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br>client.create_collection(<br>    collection_name=&quot;multimodal_catalog&quot;,<br>    vectors_config={<br>        &quot;text_vector&quot;: VectorParams(size=3072, distance=Distance.COSINE),<br>        &quot;image_vector&quot;: VectorParams(size=3072, distance=Distance.COSINE),<br>        &quot;fused_vector&quot;: VectorParams(size=3072, distance=Distance.COSINE),<br>    }<br>)</pre><p>One collection. Three search surfaces. Zero extra infrastructure.</p><h3>ACORN: Filtered Search That Actually Works (1.16)</h3><p>Released in <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.16, <strong>ACORN</strong> (Algorithm for Complex OR-query Navigation) is one of the most practically important advances in vector search in recent memory.</p><p>Here’s the problem ACORN solves: when you apply metadata filters to a vector search (e.g., “only show products from brand X, in category Y, under $50”), traditional HNSW implementations can fall into accuracy traps. The filtered neighborhood is too sparse, the graph traversal misses relevant nodes, and you get poor recall.</p><p>ACORN fixes this by examining <em>neighbors of neighbors</em> in the HNSW graph when direct neighbors are filtered out. The results are dramatic!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/435/0*lGe79wfNqF79J5-N.gif" /></figure><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> benchmarks show <strong>97.2% accuracy</strong> with ACORN versus <strong>53.3%</strong> without it on queries with multiple low-selectivity filters.</p><p>For a multimodal catalog where users are constantly mixing semantic queries with metadata constraints (“find me images similar to this one, but only from verified sellers, uploaded after 2024”) — this isn’t just a nice-to-have. It’s what makes the system trustworthy.</p><h3>Weighted RRF and Hybrid Search (1.16 + 1.17)</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> hybrid search support lets you combine vector search with keyword/BM25 search in a single query, using <strong>Reciprocal Rank Fusion (RRF)</strong> to merge results. In 1.16 they made the fusion constant k configurable; in 1.17 they introduced <strong>Weighted RRF</strong> — letting you assign different importance weights to different sub-queries.</p><p>For multimodal search, this is golden. You might want the image similarity signal to count twice as much as the text keyword match for a visual-first use case, or vice versa for a document retrieval system.</p><h3>Relevance Feedback Query (1.17)</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.17 introduced <strong>Relevance Feedback Query</strong> — a search capability that refines results using lightweight feedback on the initial result set. The system identifies context pairs (relevant vs. not relevant) and adjusts the scoring function accordingly.</p><p>In a multimodal agentic system, this enables iterative refinement loops: an agent can retrieve initial results, evaluate them against task criteria, feed that evaluation back to <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a>, and get a progressively better result set — without re-embedding anything or retraining a model.</p><h3>Inline Storage for Disk-Efficient Search (1.16)</h3><p>For large multimodal catalogs — millions of high-resolution product images, medical scans, or media assets — RAM becomes the bottleneck. <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.16’s <strong>inline storage</strong> stores quantized vector data directly within HNSW nodes, enabling disk-based vector search at <strong>211 QPS</strong> versus 20 QPS with the traditional approach on low-RAM systems.</p><p>This means you can scale your multimodal catalog to tens of millions of items without buying a server with 500GB of RAM. The cost savings in production are real.</p><h3>Segment Optimization Monitoring (1.17)</h3><p>For operational teams, <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.17 added a /collections/{name}/optimizations endpoint that tracks background optimization processes with cluster-wide visibility and detailed operation histories. When you&#39;re ingesting millions of multimodal embeddings in batches, knowing the health of your indexing pipeline in real time is invaluable.</p><h3>Open Source, Self-Hostable, Cloud Native</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is MIT-licensed and runs anywhere — local Docker for development, Kubernetes for production, or <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> for fully managed deployments. The same API, the same behavior. No lock-in.</p><blockquote><strong><em>Ready to skip the ops overhead?</em></strong><em> </em><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella"><em>Sign up for Qdrant Cloud</em></a><em> and get a managed cluster running in minutes — free tier included, no credit card required. It’s the fastest way to go from idea to a production multimodal search system</em></blockquote><h3>Designing the <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> Collection for Multimodal</h3><p>Let’s set up the collection properly. We want:</p><ul><li>Three named vector spaces (text, image, fused)</li><li>Rich metadata payload for filtering (category, date, source, modality type)</li><li>HNSW configuration tuned for our use case</li></ul><pre>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    VectorParams, Distance, HnswConfigDiff, OptimizersConfigDiff<br>)<br>def create_multimodal_collection(client: QdrantClient, collection_name: str):<br>    &quot;&quot;&quot;<br>    Create a Qdrant collection optimized for multimodal search.<br>    Three named vectors: text, image, and fused (image + text).<br>    &quot;&quot;&quot;<br>    client.recreate_collection(<br>        collection_name=collection_name,<br>        vectors_config={<br>            &quot;text_vector&quot;: VectorParams(<br>                size=3072,<br>                distance=Distance.COSINE,<br>            ),<br>            &quot;image_vector&quot;: VectorParams(<br>                size=3072,<br>                distance=Distance.COSINE,<br>            ),<br>            &quot;fused_vector&quot;: VectorParams(<br>                size=3072,<br>                distance=Distance.COSINE,<br>            ),<br>        },<br>        hnsw_config=HnswConfigDiff(<br>            m=16,           # Number of edges per node (higher = better recall, more RAM)<br>            ef_construct=200,  # Search depth during indexing (higher = better quality)<br>        ),<br>        optimizers_config=OptimizersConfigDiff(<br>            indexing_threshold=5000,  # Start indexing after 5k vectors<br>        ),<br>    )<br>    print(f&quot;Collection &#39;{collection_name}&#39; created with multimodal vector config.&quot;)</pre><p>Each point we upsert will carry a payload with structured metadata:</p><pre>{<br>    &quot;item_id&quot;: &quot;prod_12345&quot;,<br>    &quot;title&quot;: &quot;Minimalist Ceramic Mug&quot;,<br>    &quot;description&quot;: &quot;Handcrafted ceramic mug with matte finish...&quot;,<br>    &quot;category&quot;: &quot;home-goods&quot;,<br>    &quot;subcategory&quot;: &quot;drinkware&quot;,<br>    &quot;source&quot;: &quot;artisan-market&quot;,<br>    &quot;image_url&quot;: &quot;https://cdn.example.com/images/mug_001.jpg&quot;,<br>    &quot;tags&quot;: [&quot;ceramic&quot;, &quot;minimalist&quot;, &quot;handmade&quot;, &quot;kitchen&quot;],<br>    &quot;created_at&quot;: &quot;2025-11-10T08:30:00Z&quot;,<br>    &quot;modalities_available&quot;: [&quot;text&quot;, &quot;image&quot;, &quot;fused&quot;],<br>}</pre><p>The modalities_available field is useful for filtering at query time — if a point was only ingested with text (no image available), you don&#39;t want to search against its image_vector with an image query.</p><h3>Ingesting the World: Multimodal Data Pipelines</h3><p>Ingestion is where the embedding magic happens. For each item in our catalog, we need to:</p><ol><li>Extract text (description, title, tags)</li><li>Load the image</li><li>Generate all three embeddings</li><li>Upsert to <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a></li></ol><pre>import uuid<br>import base64<br>from pathlib import Path<br>from qdrant_client.models import PointStruct<br><br>def embed_item(<br>    model: MultiModalEmbeddingModel,<br>    text: str | None,<br>    image_path: str | None,<br>    dimension: int = 3072<br>) -&gt; dict:<br>    &quot;&quot;&quot;Generate text, image, and fused embeddings for a catalog item.&quot;&quot;&quot;<br>    embeddings = {}<br>  if text:<br>        result = model.get_embeddings(contextual_text=text, dimension=dimension)<br>        embeddings[&quot;text_vector&quot;] = result.text_embedding<br>    if image_path:<br>        img = VertexImage.load_from_file(image_path)<br>        result = model.get_embeddings(image=img, dimension=dimension)<br>        embeddings[&quot;image_vector&quot;] = result.image_embedding<br>    if text and image_path:<br>        img = VertexImage.load_from_file(image_path)<br>        result = model.get_embeddings(<br>            image=img,<br>            contextual_text=text,<br>            dimension=dimension<br>        )<br>        embeddings[&quot;fused_vector&quot;] = result.image_embedding  # fused is returned here<br>    return embeddings<br><br>def ingest_catalog(<br>    items: list[dict],<br>    client: QdrantClient,<br>    model: MultiModalEmbeddingModel,<br>    collection_name: str,<br>    batch_size: int = 32,<br>):<br>    &quot;&quot;&quot;<br>    Ingest a list of catalog items into Qdrant.<br>    Each item should have &#39;text&#39;, &#39;image_path&#39;, and &#39;metadata&#39; keys.<br>    &quot;&quot;&quot;<br>    batch = []<br>    for item in items:<br>        embeddings = embed_item(<br>            model,<br>            text=item.get(&quot;text&quot;),<br>            image_path=item.get(&quot;image_path&quot;),<br>        )<br>        modalities = list(embeddings.keys())<br>        item[&quot;metadata&quot;][&quot;modalities_available&quot;] = [<br>            m.replace(&quot;_vector&quot;, &quot;&quot;) for m in modalities<br>        ]<br>        point = PointStruct(<br>            id=str(uuid.uuid4()),<br>            vector=embeddings,<br>            payload=item[&quot;metadata&quot;],<br>        )<br>        batch.append(point)<br>        if len(batch) &gt;= batch_size:<br>            client.upsert(collection_name=collection_name, points=batch)<br>            print(f&quot;Upserted batch of {len(batch)} items.&quot;)<br>            batch = []<br>    if batch:<br>        client.upsert(collection_name=collection_name, points=batch)<br>        print(f&quot;Upserted final batch of {len(batch)} items.&quot;)</pre><p>Notice that we’re batching upserts. This is important: <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> optimizer needs a critical mass of vectors before it begins building the HNSW index (controlled by indexing_threshold). Sending them in large batches is faster and produces better index quality than sending one-by-one.</p><h3>The Magic of Cross-Modal Retrieval</h3><p>Here’s where the payoff happens. Once everything is ingested, we can query <em>across</em> modalities with a unified API.</p><h3>Text Query → Retrieve Similar Images (and Descriptions)</h3><pre>def search_by_text(<br>    query_text: str,<br>    client: QdrantClient,<br>    model: MultiModalEmbeddingModel,<br>    collection_name: str,<br>    filters: dict | None = None,<br>    top_k: int = 10,<br>) -&gt; list[dict]:<br>    &quot;&quot;&quot;Search the catalog using a text query. Returns multimodal results.&quot;&quot;&quot;<br>    from qdrant_client.models import Filter, FieldCondition, MatchValue<br>    <br>    # Embed the query text<br>    result = model.get_embeddings(contextual_text=query_text)<br>    query_vector = result.text_embedding<br>    <br>    # Build optional filter<br>    qdrant_filter = None<br>    if filters:<br>        conditions = [<br>            FieldCondition(key=k, match=MatchValue(value=v))<br>            for k, v in filters.items()<br>        ]<br>        qdrant_filter = Filter(must=conditions)<br>  <br>    # Search against text_vector (ACORN handles sparse filter graphs automatically)<br>    hits = client.search(<br>        collection_name=collection_name,<br>        query_vector=(&quot;text_vector&quot;, query_vector),<br>        query_filter=qdrant_filter,<br>        limit=top_k,<br>        with_payload=True,<br>    )<br><br>    return [{&quot;score&quot;: h.score, **h.payload} for h in hits]</pre><h3>Image Query → Retrieve Semantically Similar Items</h3><pre>def search_by_image(<br>    image_path: str,<br>    client: QdrantClient,<br>    model: MultiModalEmbeddingModel,<br>    collection_name: str,<br>    top_k: int = 10,<br>) -&gt; list[dict]:<br>    &quot;&quot;&quot;<br>    Query with an image. Finds items with similar visual appearance,<br>    even if their descriptions use completely different words.<br>    &quot;&quot;&quot;<br>    img = VertexImage.load_from_file(image_path)<br>    result = model.get_embeddings(image=img)<br>    query_vector = result.image_embedding<br>    hits = client.search(<br>        collection_name=collection_name,<br>        query_vector=(&quot;image_vector&quot;, query_vector),<br>        limit=top_k,<br>        with_payload=True,<br>    )<br>    return [{&quot;score&quot;: h.score, **h.payload} for h in hits]</pre><h3>Fused Query → Image + Refinement Text</h3><pre>def search_by_image_and_text(<br>    image_path: str,<br>    refinement_text: str,<br>    client: QdrantClient,<br>    model: MultiModalEmbeddingModel,<br>    collection_name: str,<br>    top_k: int = 10,<br>) -&gt; list[dict]:<br>    &quot;&quot;&quot;<br>    The killer feature: query with an image AND a text modifier.<br>    e.g., image of a chair + &quot;but in blue velvet&quot; → finds blue velvet chairs.<br>    &quot;&quot;&quot;<br>    img = VertexImage.load_from_file(image_path)<br>    result = model.get_embeddings(<br>        image=img,<br>        contextual_text=refinement_text,<br>    )<br>    query_vector = result.image_embedding  # fused output<br><br>    hits = client.search(<br>        collection_name=collection_name,<br>        query_vector=(&quot;fused_vector&quot;, query_vector),<br>        limit=top_k,<br>        with_payload=True,<br>    )<br><br>    return [{&quot;score&quot;: h.score, **h.payload} for h in hits]</pre><p>This last function — image + refinement text — is the one that makes users’ eyes go wide in demos. It’s the “find me something like this, but…” pattern, finally made real.</p><h3>Multimodal Search in Agentic Systems</h3><p>This is where things get really interesting. We’re living through a moment where AI agents are becoming the primary users of search systems — not humans typing queries, but autonomous systems deciding <em>what</em> to search for, <em>how</em> to search, and <em>what to do</em> with results.</p><p>And multimodal agents are the next frontier. An agent that can only read text is like a human who can only read books. A multimodal agent that can see images, read documents, and understand video is far more capable of operating in the real world.</p><h3>The Agent’s New Senses</h3><p>Consider an agentic system built for e-commerce operations. It needs to:</p><ul><li>Monitor supplier catalogs (PDFs + product images)</li><li>Compare incoming inventory photos against existing listings</li><li>Flag visually duplicated or counterfeit products</li><li>Generate search-optimized descriptions from images</li></ul><p>Every one of these tasks requires multimodal reasoning <em>and</em> retrieval. The agent can’t just extract text and throw away the image. The visual information is the data.</p><p>With Gemini Embedding 2 and <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a>, an agent gets a genuine semantic memory for rich media:</p><pre># Agentic pattern: embed-and-store as memory<br>async def agent_remember(item: dict, context: str):<br>    &quot;&quot;&quot;Agent stores a new observation in its multimodal memory.&quot;&quot;&quot;<br>    embeddings = embed_item(model, text=context, image_path=item.get(&quot;image&quot;))<br>    await store_to_qdrant(embeddings, payload={&quot;context&quot;: context, **item})<br><br># Agentic pattern: recall from memory<br>async def agent_recall(query_image: str, query_text: str) -&gt; list[dict]:<br>    &quot;&quot;&quot;Agent retrieves relevant past observations given current context.&quot;&quot;&quot;<br>    return search_by_image_and_text(query_image, query_text, client, model, &quot;agent_memory&quot;)</pre><h3><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> Relevance Feedback Loop for Agents (1.17)</h3><p><a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> 1.17’s Relevance Feedback Query maps perfectly onto agentic refinement loops. An agent can:</p><ol><li>Issue an initial multimodal query</li><li>Score each result against task-specific criteria (using an LLM or rule set)</li><li>Feed high-scoring results back to <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> as positive examples</li><li>Issue a refined query that scores results against both the original query and the feedback signal</li></ol><p>This creates a self-improving retrieval loop without retraining anything. The agent gets better search results through interaction, not gradient descent.</p><pre>from qdrant_client.models import RecommendStrategy<br>def agent_refine_search(<br>    initial_results: list[str],   # point IDs the agent found relevant<br>    negative_results: list[str],  # point IDs the agent found irrelevant<br>    client: QdrantClient,<br>    collection_name: str,<br>    top_k: int = 10,<br>) -&gt; list[dict]:<br>    &quot;&quot;&quot;Use Qdrant&#39;s recommendation/relevance API to refine results.&quot;&quot;&quot;<br>    hits = client.recommend(<br>        collection_name=collection_name,<br>        positive=initial_results,<br>        negative=negative_results,<br>        using=&quot;fused_vector&quot;,<br>        limit=top_k,<br>        with_payload=True,<br>        strategy=RecommendStrategy.BEST_SCORE,<br>    )<br>    return [{&quot;score&quot;: h.score, **h.payload} for h in hits]</pre><h3>Multi-Collection Architecture for Agents</h3><p>A mature agentic system will want more than one collection. Following the same pattern as the skincare assistant from Hola-Dermat, you’d want:</p><ul><li>catalog_collection — the main multimodal knowledge base</li><li>agent_memory_collection — embeddings of past agent observations and decisions</li><li>user_context_collection — user-specific multimodal history (uploaded images, past queries)</li></ul><p>Each collection is optimized for its specific access pattern, can be scaled independently, and can have different retention policies. <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> multi-collection support makes this a first-class concern, not an afterthought.</p><h3>Building the Streamlit App: Multimodal Search in Your Browser</h3><p>Enough architecture. Let’s build something you can actually run and demo.</p><p>The app supports three search modes:</p><ol><li><strong>Text search</strong> — type a description, get results</li><li><strong>Image search</strong> — upload a photo, get visually similar results</li><li><strong>Multimodal search</strong> — upload a photo and describe what you want, get the best of both</li></ol><p>Here’s the complete app.py:</p><pre># app.py — Multimodal Search with Gemini Embedding 2 + Qdrant<br># Requirements: streamlit, qdrant-client, google-cloud-aiplatform, pillow<br>import streamlit as st<br>import tempfile<br>import os<br>import uuid<br>import json<br>from pathlib import Path<br>from PIL import Image<br>import vertexai<br>from vertexai.vision_models import MultiModalEmbeddingModel<br>from vertexai.vision_models import Image as VertexImage<br>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    VectorParams, Distance, PointStruct, Filter,<br>    FieldCondition, MatchValue, HnswConfigDiff<br>)<br><br>GCP_PROJECT = os.getenv(&quot;GCP_PROJECT&quot;, &quot;your-gcp-project&quot;)<br>GCP_LOCATION = os.getenv(&quot;GCP_LOCATION&quot;, &quot;us-central1&quot;)<br>QDRANT_URL = os.getenv(&quot;QDRANT_URL&quot;, &quot;http://localhost:6333&quot;)<br>COLLECTION_NAME = &quot;multimodal_catalog&quot;<br>VECTOR_DIM = 3072<br><br>@st.cache_resource<br>def init_clients():<br>    vertexai.init(project=GCP_PROJECT, location=GCP_LOCATION)<br>    embed_model = MultiModalEmbeddingModel.from_pretrained(&quot;multimodalembedding@001&quot;)<br>    qdrant = QdrantClient(url=QDRANT_URL)<br>    return embed_model, qdrant<br><br>@st.cache_resource<br>def ensure_collection(_qdrant: QdrantClient):<br>    collections = [c.name for c in _qdrant.get_collections().collections]<br>    if COLLECTION_NAME not in collections:<br>        _qdrant.create_collection(<br>            collection_name=COLLECTION_NAME,<br>            vectors_config={<br>                &quot;text_vector&quot;: VectorParams(size=VECTOR_DIM, distance=Distance.COSINE),<br>                &quot;image_vector&quot;: VectorParams(size=VECTOR_DIM, distance=Distance.COSINE),<br>                &quot;fused_vector&quot;: VectorParams(size=VECTOR_DIM, distance=Distance.COSINE),<br>            },<br>            hnsw_config=HnswConfigDiff(m=16, ef_construct=200),<br>        )<br>        st.toast(f&quot;Created collection: {COLLECTION_NAME}&quot;)<br><br><br>def embed_query(<br>    model: MultiModalEmbeddingModel,<br>    text: str | None = None,<br>    image_path: str | None = None,<br>) -&gt; tuple[list[float] | None, list[float] | None, list[float] | None]:<br>    &quot;&quot;&quot;Return (text_vec, image_vec, fused_vec). Each can be None.&quot;&quot;&quot;<br>    text_vec = image_vec = fused_vec = None<br>    if text and not image_path:<br>        r = model.get_embeddings(contextual_text=text, dimension=VECTOR_DIM)<br>        text_vec = r.text_embedding<br>    if image_path and not text:<br>        img = VertexImage.load_from_file(image_path)<br>        r = model.get_embeddings(image=img, dimension=VECTOR_DIM)<br>        image_vec = r.image_embedding<br>    if text and image_path:<br>        img = VertexImage.load_from_file(image_path)<br>        r = model.get_embeddings(<br>            image=img, contextual_text=text, dimension=VECTOR_DIM<br>        )<br>        fused_vec = r.image_embedding<br>    return text_vec, image_vec, fused_vec<br><br><br>def ingest_item(<br>    model: MultiModalEmbeddingModel,<br>    qdrant: QdrantClient,<br>    title: str,<br>    description: str,<br>    category: str,<br>    image_path: str | None,<br>):<br>    combined_text = f&quot;{title}. {description}&quot;<br>    vectors = {}<br>    r_text = model.get_embeddings(contextual_text=combined_text, dimension=VECTOR_DIM)<br>    vectors[&quot;text_vector&quot;] = r_text.text_embedding<br>    if image_path:<br>        img = VertexImage.load_from_file(image_path)<br>        r_img = model.get_embeddings(image=img, dimension=VECTOR_DIM)<br>        vectors[&quot;image_vector&quot;] = r_img.image_embedding<br>        r_fused = model.get_embeddings(<br>            image=img, contextual_text=combined_text, dimension=VECTOR_DIM<br>        )<br>        vectors[&quot;fused_vector&quot;] = r_fused.image_embedding<br>    payload = {<br>        &quot;item_id&quot;: str(uuid.uuid4()),<br>        &quot;title&quot;: title,<br>        &quot;description&quot;: description,<br>        &quot;category&quot;: category,<br>        &quot;modalities&quot;: list(vectors.keys()),<br>        &quot;has_image&quot;: image_path is not None,<br>    }<br>    qdrant.upsert(<br>        collection_name=COLLECTION_NAME,<br>        points=[PointStruct(id=str(uuid.uuid4()), vector=vectors, payload=payload)],<br>    )<br><br>def run_search(<br>    model: MultiModalEmbeddingModel,<br>    qdrant: QdrantClient,<br>    query_text: str | None,<br>    query_image_path: str | None,<br>    category_filter: str | None,<br>    top_k: int = 8,<br>) -&gt; list[dict]:<br>    text_vec, image_vec, fused_vec = embed_query(model, query_text, query_image_path)<br>    # Determine which vector space to search<br>    if fused_vec is not None:<br>        vector_name, query_vector = &quot;fused_vector&quot;, fused_vec<br>    elif image_vec is not None:<br>        vector_name, query_vector = &quot;image_vector&quot;, image_vec<br>    elif text_vec is not None:<br>        vector_name, query_vector = &quot;text_vector&quot;, text_vec<br>    else:<br>        return []<br>    qdrant_filter = None<br>    if category_filter and category_filter != &quot;All&quot;:<br>        qdrant_filter = Filter(<br>            must=[FieldCondition(key=&quot;category&quot;, match=MatchValue(value=category_filter))]<br>        )<br>    hits = qdrant.search(<br>        collection_name=COLLECTION_NAME,<br>        query_vector=(vector_name, query_vector),<br>        query_filter=qdrant_filter,<br>        limit=top_k,<br>        with_payload=True,<br>    )<br>    return [{&quot;score&quot;: round(h.score, 4), **h.payload} for h in hits]<br><br><br>def main():<br>    st.set_page_config(<br>        page_title=&quot;Multimodal Search - Gemini + Qdrant&quot;,<br>        page_icon=&quot;🔍&quot;,<br>        layout=&quot;wide&quot;,<br>    )<br>    st.title(&quot;Multimodal Search&quot;)<br>    st.caption(&quot;Powered by Gemini Embedding 2 + Qdrant&quot;)<br>    model, qdrant = init_clients()<br>    ensure_collection(qdrant)<br>    tab_search, tab_ingest = st.tabs([&quot;Search&quot;, &quot;Add Items&quot;])<br><br>    with tab_search:<br>        st.subheader(&quot;Search the Catalog&quot;)<br>        col1, col2 = st.columns([1, 1])<br>        with col1:<br>            query_text = st.text_input(<br>                &quot;Describe what you&#39;re looking for&quot;,<br>                placeholder=&quot;e.g. minimalist ceramic mug with matte finish&quot;<br>            )<br>        with col2:<br>            uploaded_query_img = st.file_uploader(<br>                &quot;Or upload an image query&quot;,<br>                type=[&quot;jpg&quot;, &quot;jpeg&quot;, &quot;png&quot;, &quot;webp&quot;],<br>                key=&quot;query_img&quot;<br>            )<br>        category_filter = st.selectbox(<br>            &quot;Filter by category&quot;,<br>            [&quot;All&quot;, &quot;home-goods&quot;, &quot;fashion&quot;, &quot;electronics&quot;, &quot;art&quot;, &quot;food&quot;]<br>        )<br>        top_k = st.slider(&quot;Results to return&quot;, 4, 20, 8)<br>        search_mode = &quot;none&quot;<br>        if query_text and uploaded_query_img:<br>            search_mode = &quot;multimodal&quot;<br>            st.info(&quot;Multimodal search: image + text refinement&quot;)<br>        elif uploaded_query_img:<br>            search_mode = &quot;image&quot;<br>            st.info(&quot;Image search: finding visually similar items&quot;)<br>        elif query_text:<br>            search_mode = &quot;text&quot;<br>            st.info(&quot;Text search: semantic matching&quot;)<br>        if st.button(&quot;Search&quot;, type=&quot;primary&quot;, disabled=(search_mode == &quot;none&quot;)):<br>            query_image_path = None<br>            if uploaded_query_img:<br>                with tempfile.NamedTemporaryFile(<br>                    suffix=Path(uploaded_query_img.name).suffix, delete=False<br>                ) as tmp:<br>                    tmp.write(uploaded_query_img.read())<br>                    query_image_path = tmp.name<br>            with st.spinner(&quot;Embedding query and searching...&quot;):<br>                results = run_search(<br>                    model, qdrant,<br>                    query_text=query_text or None,<br>                    query_image_path=query_image_path,<br>                    category_filter=category_filter,<br>                    top_k=top_k,<br>                )<br>            if query_image_path:<br>                os.unlink(query_image_path)<br>            if results:<br>                st.success(f&quot;Found {len(results)} results&quot;)<br>                cols = st.columns(4)<br>                for i, item in enumerate(results):<br>                    with cols[i % 4]:<br>                        st.metric(&quot;Score&quot;, item[&quot;score&quot;])<br>                        st.markdown(f&quot;**{item.get(&#39;title&#39;, &#39;Untitled&#39;)}**&quot;)<br>                        st.caption(item.get(&quot;description&quot;, &quot;&quot;)[:120] + &quot;...&quot;)<br>                        st.caption(f&quot;Category: {item.get(&#39;category&#39;, &#39;-&#39;)}&quot;)<br>                        st.divider()<br>            else:<br>                st.warning(&quot;No results found. Try a different query or add more items.&quot;)<br><br>    with tab_ingest:<br>        st.subheader(&quot;Add a New Item to the Catalog&quot;)<br>        with st.form(&quot;ingest_form&quot;):<br>            title = st.text_input(&quot;Title&quot;, placeholder=&quot;Handcrafted Ceramic Mug&quot;)<br>            description = st.text_area(<br>                &quot;Description&quot;,<br>                placeholder=&quot;Describe the item in detail - materials, style, use case...&quot;<br>            )<br>            category = st.selectbox(<br>                &quot;Category&quot;,<br>                [&quot;home-goods&quot;, &quot;fashion&quot;, &quot;electronics&quot;, &quot;art&quot;, &quot;food&quot;]<br>            )<br>            uploaded_item_img = st.file_uploader(<br>                &quot;Product Image (optional but recommended)&quot;,<br>                type=[&quot;jpg&quot;, &quot;jpeg&quot;, &quot;png&quot;, &quot;webp&quot;],<br>                key=&quot;item_img&quot;<br>            )<br>            submitted = st.form_submit_button(&quot;Add to Catalog&quot;)<br>        if submitted:<br>            if not title or not description:<br>                st.error(&quot;Title and description are required.&quot;)<br>            else:<br>                image_path = None<br>                if uploaded_item_img:<br>                    with tempfile.NamedTemporaryFile(<br>                        suffix=Path(uploaded_item_img.name).suffix, delete=False<br>                    ) as tmp:<br>                        tmp.write(uploaded_item_img.read())<br>                        image_path = tmp.name<br>                with st.spinner(&quot;Embedding and indexing item...&quot;):<br>                    ingest_item(model, qdrant, title, description, category, image_path)<br>                if image_path:<br>                    os.unlink(image_path)<br>                st.success(f&quot;&#39;{title}&#39; added to catalog with multimodal embeddings.&quot;)<br><br>    with st.sidebar:<br>        st.header(&quot;System Info&quot;)<br>        try:<br>            info = qdrant.get_collection(COLLECTION_NAME)<br>            st.metric(&quot;Total Points&quot;, info.points_count)<br>            st.metric(&quot;Index Status&quot;, info.status.value)<br>            st.json({<br>                &quot;vectors&quot;: list(info.config.params.vectors.keys())<br>                if hasattr(info.config.params.vectors, &quot;keys&quot;)<br>                else &quot;named&quot;<br>            })<br>        except Exception:<br>            st.warning(&quot;Could not fetch collection info.&quot;)<br>        st.divider()<br>        st.caption(&quot;Gemini Embedding 2 via Vertex AI&quot;)<br>        st.caption(&quot;Qdrant vector database&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><p>Run it with:</p><pre>pip install streamlit qdrant-client google-cloud-aiplatform pillow<br>streamlit run app.py</pre><p>Make sure you have:</p><ul><li>A running <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> instance (docker run -p 6333:6333 qdrant/qdrant) — or better yet, a free <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> cluster so you don&#39;t have to manage anything yourself</li><li>Authenticated with GCP (gcloud auth application-default login)</li><li>Your GCP_PROJECT set to a project with Vertex AI enabled</li></ul><h3>The Query &amp; Retrieve Pipeline: Production-Grade Search</h3><p>The Streamlit app is great for demos. But for production agentic systems, you want a clean, callable search module. Here’s a standalone retrieval pipeline:</p><pre># retrieval.py — Production multimodal search pipeline<br>import os<br>import tempfile<br>from dataclasses import dataclass<br>from pathlib import Path<br>import vertexai<br>from vertexai.vision_models import MultiModalEmbeddingModel, Image as VertexImage<br>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    Filter, FieldCondition, MatchValue, MatchAny,<br>    RecommendStrategy<br>)<br><br>@dataclass<br>class SearchResult:<br>    id: str<br>    score: float<br>    title: str<br>    description: str<br>    category: str<br>    has_image: bool<br>    modalities: list[str]<br><br>class MultimodalSearchPipeline:<br>    &quot;&quot;&quot;<br>    Production-ready multimodal search using Gemini Embedding 2 + Qdrant.<br>    Supports text, image, and fused queries with optional metadata filtering.<br>    &quot;&quot;&quot;<br>    def __init__(<br>        self,<br>        gcp_project: str,<br>        gcp_location: str = &quot;us-central1&quot;,<br>        qdrant_url: str = &quot;http://localhost:6333&quot;,<br>        collection_name: str = &quot;multimodal_catalog&quot;,<br>        vector_dim: int = 3072,<br>    ):<br>        vertexai.init(project=gcp_project, location=gcp_location)<br>        self.model = MultiModalEmbeddingModel.from_pretrained(&quot;multimodalembedding@001&quot;)<br>        self.qdrant = QdrantClient(url=qdrant_url)<br>        self.collection = collection_name<br>        self.dim = vector_dim<br>    def search(<br>        self,<br>        text: str | None = None,<br>        image_bytes: bytes | None = None,<br>        categories: list[str] | None = None,<br>        top_k: int = 10,<br>        score_threshold: float = 0.3,<br>    ) -&gt; list[SearchResult]:<br>        &quot;&quot;&quot;<br>        Unified search endpoint. Accepts text, image bytes, or both.<br>        Routes to the best-matching vector space automatically.<br>        &quot;&quot;&quot;<br>        if not text and not image_bytes:<br>            raise ValueError(&quot;Provide at least one of: text, image_bytes&quot;)<br>        query_vector, vector_name = self._embed_query(text, image_bytes)<br>        qdrant_filter = self._build_filter(categories)<br>        hits = self.qdrant.search(<br>            collection_name=self.collection,<br>            query_vector=(vector_name, query_vector),<br>            query_filter=qdrant_filter,<br>            limit=top_k,<br>            score_threshold=score_threshold,<br>            with_payload=True,<br>        )<br>        return [<br>            SearchResult(<br>                id=str(h.id),<br>                score=round(h.score, 4),<br>                title=h.payload.get(&quot;title&quot;, &quot;&quot;),<br>                description=h.payload.get(&quot;description&quot;, &quot;&quot;),<br>                category=h.payload.get(&quot;category&quot;, &quot;&quot;),<br>                has_image=h.payload.get(&quot;has_image&quot;, False),<br>                modalities=h.payload.get(&quot;modalities&quot;, []),<br>            )<br>            for h in hits<br>        ]<br>    def refine(<br>        self,<br>        positive_ids: list[str],<br>        negative_ids: list[str] | None = None,<br>        top_k: int = 10,<br>    ) -&gt; list[SearchResult]:<br>        &quot;&quot;&quot;<br>        Relevance feedback refinement (Qdrant 1.17 feature).<br>        Provide IDs of relevant and irrelevant results to refine the search.<br>        &quot;&quot;&quot;<br>        hits = self.qdrant.recommend(<br>            collection_name=self.collection,<br>            positive=positive_ids,<br>            negative=negative_ids or [],<br>            using=&quot;fused_vector&quot;,<br>            limit=top_k,<br>            with_payload=True,<br>            strategy=RecommendStrategy.BEST_SCORE,<br>        )<br>        return [<br>            SearchResult(<br>                id=str(h.id),<br>                score=round(h.score, 4),<br>                title=h.payload.get(&quot;title&quot;, &quot;&quot;),<br>                description=h.payload.get(&quot;description&quot;, &quot;&quot;),<br>                category=h.payload.get(&quot;category&quot;, &quot;&quot;),<br>                has_image=h.payload.get(&quot;has_image&quot;, False),<br>                modalities=h.payload.get(&quot;modalities&quot;, []),<br>            )<br>            for h in hits<br>        ]<br>    def _embed_query(<br>        self,<br>        text: str | None,<br>        image_bytes: bytes | None,<br>    ) -&gt; tuple[list[float], str]:<br>        if text and image_bytes:<br>            with tempfile.NamedTemporaryFile(suffix=&quot;.jpg&quot;, delete=False) as tmp:<br>                tmp.write(image_bytes)<br>                tmp_path = tmp.name<br>            img = VertexImage.load_from_file(tmp_path)<br>            os.unlink(tmp_path)<br>            r = self.model.get_embeddings(<br>                image=img, contextual_text=text, dimension=self.dim<br>            )<br>            return r.image_embedding, &quot;fused_vector&quot;<br>        if image_bytes:<br>            with tempfile.NamedTemporaryFile(suffix=&quot;.jpg&quot;, delete=False) as tmp:<br>                tmp.write(image_bytes)<br>                tmp_path = tmp.name<br>            img = VertexImage.load_from_file(tmp_path)<br>            os.unlink(tmp_path)<br>            r = self.model.get_embeddings(image=img, dimension=self.dim)<br>            return r.image_embedding, &quot;image_vector&quot;<br>        r = self.model.get_embeddings(contextual_text=text, dimension=self.dim)<br>        return r.text_embedding, &quot;text_vector&quot;<br>    def _build_filter(self, categories: list[str] | None) -&gt; Filter | None:<br>        if not categories:<br>            return None<br>        return Filter(<br>            must=[<br>                FieldCondition(<br>                    key=&quot;category&quot;,<br>                    match=MatchAny(any=categories),<br>                )<br>            ]<br>        )<br><br>## This is how you use it <br><br>if __name__ == &quot;__main__&quot;:<br>    pipeline = MultimodalSearchPipeline(<br>        gcp_project=os.getenv(&quot;GCP_PROJECT&quot;, &quot;your-project&quot;),<br>    )<br>    # Text search<br>    results = pipeline.search(text=&quot;handmade ceramic drinkware&quot;)<br>    print(&quot;Text search results:&quot;)<br>    for r in results:<br>        print(f&quot;  [{r.score}] {r.title} - {r.category}&quot;)<br>    # Image search<br>    with open(&quot;query_image.jpg&quot;, &quot;rb&quot;) as f:<br>        image_results = pipeline.search(image_bytes=f.read(), top_k=5)<br>    print(&quot;\nImage search results:&quot;)<br>    for r in image_results:<br>        print(f&quot;  [{r.score}] {r.title}&quot;)<br>    # Multimodal search<br>    with open(&quot;query_image.jpg&quot;, &quot;rb&quot;) as f:<br>        fused_results = pipeline.search(<br>            text=&quot;but in a darker color palette&quot;,<br>            image_bytes=f.read(),<br>        )<br>    print(&quot;\nFused search results:&quot;)<br>    for r in fused_results:<br>        print(f&quot;  [{r.score}] {r.title}&quot;)<br>    # Relevance feedback refinement (Qdrant 1.17)<br>    if results:<br>        good_id = results[0].id<br>        refined = pipeline.refine(positive_ids=[good_id], top_k=5)<br>        print(&quot;\nRefined results:&quot;)<br>        for r in refined:<br>            print(f&quot;  [{r.score}] {r.title}&quot;)</pre><p>This pipeline is designed to drop directly into an agentic workflow. An agent can call pipeline.search(...) with whatever modalities it has, get back a clean list of SearchResult objects, and then optionally call pipeline.refine(...) with IDs it deemed relevant to tighten the results. The whole thing is clean, typed, and testable.</p><h3>Why This Matters: The Multimodal Future Is Now</h3><p>Let’s zoom out for a second.</p><p>We’re at an inflection point. The models that dominated the last few years, GPT, Claude, Gemini were primarily <em>generative</em>. They produced text (and eventually images). The next wave is about <em>understanding and retrieving</em> across all modalities at production scale.</p><p>The combination we’ve built here Gemini Embedding 2 for multimodal understanding, <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> for production-grade vector search hits a sweet spot that didn’t exist even two years ago:</p><p><strong>For developers</strong>: You get a unified embedding space that eliminates the complexity of managing separate models for text and images. One API call, one vector, one search.</p><p><strong>For product teams</strong>: Your users can finally search the way they actually think with images, with vague descriptions, with references to things they’ve seen. The vocabulary mismatch problem, the cross-modal wall, the zero-results frustration, …. all solved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/405/0*56f8uMjCffhPInJF.gif" /></figure><p><strong>For AI systems</strong>: Agents that can see, remember images, and retrieve relevant visual context are fundamentally more capable than text-only agents. Multimodal memory is a superpower for autonomous systems operating in rich-media environments.</p><p>And <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> keeps shipping. ACORN in 1.16 solved filtered recall. Weighted RRF in 1.17 made hybrid search more expressive. Relevance feedback in 1.17 opened the door to iterative refinement loops. Inline storage made million-item multimodal catalogs economically viable. The trajectory is clear: <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> is building the infrastructure layer for the multimodal AI era.</p><h3>Conclusion: Search That Sees</h3><p>The query &quot;find me something like this, but better&quot; paired with a photo used to be a dead end.</p><p>Now it’s just a Streamlit widget.</p><p>We’ve covered a lot of ground: the multimodal gap, how Gemini Embedding 2 closes it with unified vector representations, why <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant’s</a> collection architecture and ACORN algorithm make it the right home for those vectors, and how to wire it all together into something you can actually ship.</p><p>The code is yours. The patterns are transferable. Build a product discovery tool. Build an agentic visual memory system. Build a medical image retrieval platform. Build something nobody’s thought of yet.</p><p>The infrastructure to do it is finally here. All you need is the idea.</p><p><em>Now if you’ll excuse me, I have to go search for “that mug I saw somewhere that felt cozy” and see if the system can actually find it. It probably can now.</em></p><p><strong>Want to go further?</strong></p><p>If you’re serious about building multimodal AI systems, <a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Cloud</a> is hands-down the easiest way to get a production-grade vector database running without wrangling infrastructure. Spin up a free cluster in under two minutes, connect the same QdrantClient code from this article, and you&#39;re live. No ops, no maintenance, no surprise RAM bills at 3am.</p><p>And if you want to stay sharp on what <a href="http://qdrant.tech/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant</a> ships next, new indexing strategies, performance improvements, new API features, <a href="https://qdrant.tech/subscribe/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">subscribe to the Qdrant newsletter</a>. The team ships fast and the release notes are genuinely worth reading.</p><ul><li><a href="https://github.com/qdrant/qdrant">Qdrant GitHub</a> open source, go star it</li><li><a href="https://qdrant.tech/blog/qdrant-1.17.x/">Qdrant 1.17 Release Notes</a> relevance feedback, weighted RRF, tail latency improvements</li><li><a href="https://qdrant.tech/blog/qdrant-1.16.x/">Qdrant 1.16 Release Notes</a> ACORN, inline storage, tiered multitenancy</li><li><a href="https://qdrant.tech/documentation/?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Qdrant Documentation</a> everything you need to go deep on any feature</li><li><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings">Vertex AI Multimodal Embeddings</a> official docs for Gemini Embedding 2</li><li><a href="https://cloud.qdrant.io/signup?utm_medium=referral&amp;utm_source=stars&amp;utm_campaign=devrel&amp;utm_content=niranjan-akella">Sign up for Qdrant Cloud</a> free tier, managed, zero ops</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=afd0c627aa0d" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/search-beyond-words-multimodal-ai-with-gemini-embeddings-2-and-qdrant-afd0c627aa0d">Search Beyond Words: Multimodal AI with Gemini Embeddings 2 and Qdrant</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[We Replaced a $500K ML Retraining Pipeline with 3 Lines of Vector Math]]></title>
            <link>https://pub.towardsai.net/we-replaced-a-500k-ml-retraining-pipeline-with-3-lines-of-vector-math-18145dc91070?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/18145dc91070</guid>
            <category><![CDATA[money]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[innovation]]></category>
            <category><![CDATA[qdrant]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Sat, 07 Mar 2026 12:46:00 GMT</pubDate>
            <atom:updated>2026-03-13T04:20:55.875Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Building a Self-Improving Enterprise Knowledge Base with Qdrant v1.17’s Relevance Feedback API</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DIW4w941eq-gcmbFC7vRiw.png" /><figcaption>Feat Qdrant x Gemini x Startups</figcaption></figure><p>You searched for “how to reset user permissions” in your company’s internal docs. Got 10 results. None were what you needed. You scrolled, sighed, and Slack’d a teammate instead.</p><p>Your colleague did the same search last week. Got the same 10 results. Sighed the same sigh.</p><p>That’s the dirty secret of enterprise search — <strong>it has no memory</strong>. Every search starts from scratch. No matter how many people flagged a result as useless, the system serves it again. And again. And again.</p><p>Qdrant v1.17 just changed that.</p><p>With the new <strong>Relevance Feedback Query</strong>, your search engine can now learn — in real time, without retraining, without a separate ML pipeline, without expensive re-ranking models. A user clicks 👍 on result 2 and 👎 on result 5, and the <em>next</em> search already knows more about what “relevant” means in your context.</p><p>This article builds that system end-to-end. No fluff. Just the tech, the math, and working code.</p><h3>Section 1: How Relevance Feedback Actually Works</h3><p>Before we touch a keyboard, we need to understand what’s happening inside the vector space. Because this isn’t magic — it’s beautiful math.</p><h3>Standard Vector Search (The Dumb Version)</h3><p>In a regular semantic search:</p><ol><li>Embed the user’s query → get a query vector q</li><li>Find the top-K vectors in the collection closest to q</li><li>Return them. Done. Forget everything.</li></ol><p>The system treats every search as if it’s the first time anyone has ever asked that question. No context. No memory. No improvement.</p><h3>What Relevance Feedback Changes</h3><p>When a user sees results and gives feedback — thumbs up on result A, thumbs down on result B — they’re giving you something incredibly valuable: <strong>labeled examples of what “relevant” means for this specific query intent</strong>.</p><p>Qdrant v1.17 uses these signals to reshape the query vector itself using one of two strategies.</p><h3>Strategy 1: average_vector (Fast, Production Default)</h3><p>This is the formula at the heart of relevance feedback:</p><pre>average_vector = avg(positive_vectors) + (avg(positive_vectors) - avg(negative_vectors))</pre><p>Let’s break this down:</p><ul><li><strong>avg(positive_vectors)</strong> — The centroid of all vectors the user liked. This is the &quot;direction of relevance.&quot;</li><li><strong>avg(positive_vectors) - avg(negative_vectors)</strong> — The <em>difference</em> vector between what&#39;s relevant and what&#39;s not. This is the &quot;correction signal.&quot;</li><li><strong>Adding them together</strong> — You’re taking the centroid of liked results and pushing it <em>further away</em> from the disliked results.</li></ul><p>In plain English: <strong>vector components that are stronger in negative examples get penalized. Components stronger in positive examples get amplified.</strong> The query vector is nudged in the direction of “more like this, less like that.”</p><pre>from qdrant_client import QdrantClient, models<br><br>client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br><br># After user gives feedback:<br># Liked result IDs: [42, 87]<br># Disliked result IDs: [13, 55]<br>results = client.query_points(<br>    collection_name=&quot;enterprise_kb&quot;,<br>    query=models.RecommendQuery(<br>        recommend=models.RecommendInput(<br>            positive=[42, 87],    # IDs of results user liked<br>            negative=[13, 55],    # IDs of results user disliked<br>            strategy=models.RecommendStrategy.AVERAGE_VECTOR<br>        )<br>    ),<br>    limit=10,<br>    with_payload=True<br>)</pre><p>Performance-wise, average_vector is <strong>on par with regular vector search</strong> because the preprocessing (computing the average vector) is nearly instant. This is what makes it practical in production.</p><h3>Strategy 2: best_score (More Precise, Slightly Slower)</h3><p>Instead of averaging, best_score measures every candidate against <em>every</em> feedback example individually:</p><pre>For each candidate vector c:<br>  best_positive = max(similarity(c, p) for p in positives)<br>  best_negative = max(similarity(c, n) for n in negatives)<br>  <br>  if best_positive &gt; best_negative:<br>      final_score = best_positive<br>  else:<br>      final_score = -(best_negative - 0.5)  # Penalty formula</pre><p>This means a candidate only scores well if it’s <em>more similar to at least one positive example than to any negative example</em>. It’s stricter and handles edge cases better — especially when your positive examples span multiple distinct topics.</p><pre>results = client.query_points(<br>    collection_name=&quot;enterprise_kb&quot;,<br>    query=models.RecommendQuery(<br>        recommend=models.RecommendInput(<br>            positive=[42, 87],<br>            negative=[13, 55],<br>            strategy=models.RecommendStrategy.BEST_SCORE  # More precise<br>        )<br>    ),<br>    limit=10<br>)</pre><p><strong>Bonus:</strong> best_score with <em>only</em> negative examples finds the most dissimilar vectors to the ones you don&#39;t want. Useful for surfacing underexplored corners of your knowledge base.</p><h3>The v1.17 Innovation: Context Pairs</h3><p>Qdrant v1.17 introduces a more powerful abstraction on top of this: <strong>context pairs</strong>. Instead of a flat list of positives and negatives, you provide them as <em>paired</em> (positive, negative) tuples:</p><pre>Context Pair 1: (liked_doc_A, disliked_doc_B)<br>Context Pair 2: (liked_doc_C, disliked_doc_D)</pre><p>Each pair <strong>divides the vector space into a positive zone and a negative zone</strong>. The search then prefers points that live in the most positive zones — even when those zones don’t cluster neatly around a single centroid. This is particularly powerful for multi-intent queries where different users are looking for different things within the same search phrase.</p><pre># Context-pair based relevance feedback (v1.17)<br>results = client.query_points(<br>    collection_name=&quot;enterprise_kb&quot;,<br>    query=models.ContextQuery(<br>        context=[<br>            models.ContextPair(positive=42, negative=13),<br>            models.ContextPair(positive=87, negative=55),<br>        ]<br>    ),<br>    limit=10<br>)</pre><p>This is inspired by <strong>triplet loss</strong> from metric learning — anchor, positive, negative — but applied at query time with zero retraining.</p><h3>Section 2: Setting Up Qdrant v1.17</h3><h3>Option A: Docker (30 seconds)</h3><pre>docker run -d \<br>  --name qdrant \<br>  -p 6333:6333 \<br>  -p 6334:6334 \<br>  -v $(pwd)/qdrant_storage:/qdrant/storage \<br>  qdrant/qdrant:v1.17.0</pre><p>Verify it’s running:</p><pre>curl http://localhost:6333/healthz<br># → {&quot;title&quot;:&quot;qdrant - vector search engine&quot;,&quot;version&quot;:&quot;1.17.0&quot;}</pre><h3>Option B: Qdrant Cloud (Free Tier, 3 Steps)</h3><ol><li>Sign up at <a href="https://cloud.qdrant.io/">cloud.qdrant.io</a></li><li>Create a cluster → pick the free tier (1GB RAM, enough for 500K vectors at 384 dims)</li><li>Copy your <strong>Cluster URL</strong> and <strong>API Key</strong> from the dashboard</li></ol><pre>from qdrant_client import QdrantClient<br><br># Local Docker<br>client = QdrantClient(url=&quot;http://localhost:6333&quot;)<br><br># Qdrant Cloud<br>client = QdrantClient(<br>    url=&quot;https://YOUR-CLUSTER-ID.us-east4-0.gcp.cloud.qdrant.io&quot;,<br>    api_key=&quot;YOUR_API_KEY&quot;<br>)</pre><h3>Install Dependencies</h3><pre>pip install qdrant-client sentence-transformers streamlit fastapi uvicorn tqdm PyPDF2 python-dotenv</pre><h3>Section 3: Data Ingestion Pipeline</h3><p>Here’s a complete ingestion script that handles PDFs, text files, and markdown docs — the typical mess inside an enterprise knowledge base.</p><pre># ingestion.py<br>import os<br>import uuid<br>from pathlib import Path<br>from typing import List, Dict, Any<br><br>import PyPDF2<br>from tqdm import tqdm<br>from sentence_transformers import SentenceTransformer<br>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    Distance, VectorParams, PointStruct,<br>    ScalarQuantizationConfig, ScalarType<br>)<br># ── Config ───────────────────────────────────────────────────────────────────<br>COLLECTION_NAME = &quot;enterprise_kb&quot;<br>EMBEDDING_MODEL  = &quot;all-MiniLM-L6-v2&quot;   # 384-dim, fast, free<br>VECTOR_DIM       = 384<br>CHUNK_SIZE       = 400    # words<br>CHUNK_OVERLAP    = 50     # words<br>client  = QdrantClient(url=&quot;http://localhost:6333&quot;)<br>encoder = SentenceTransformer(EMBEDDING_MODEL)<br><br># ── Collection Setup ─────────────────────────────────────────────────────────<br>def create_collection():<br>    existing = [c.name for c in client.get_collections().collections]<br>    if COLLECTION_NAME in existing:<br>        return<br>    client.create_collection(<br>        collection_name=COLLECTION_NAME,<br>        vectors_config=VectorParams(<br>            size=VECTOR_DIM,<br>            distance=Distance.COSINE,<br>        ),<br>        # INT8 scalar quantization - 4x memory reduction, ~1-2% recall loss<br>        quantization_config=ScalarQuantizationConfig(<br>            type=ScalarType.INT8,<br>            quantile=0.99,<br>            always_ram=True,   # Keep quantized vectors in RAM for fast search<br>        )<br>    )<br>    print(f&quot;✅ Created collection: {COLLECTION_NAME}&quot;)<br><br># ── Document Readers ─────────────────────────────────────────────────────────<br>def read_pdf(path: Path) -&gt; str:<br>    text = &quot;&quot;<br>    with open(path, &quot;rb&quot;) as f:<br>        reader = PyPDF2.PdfReader(f)<br>        for page in reader.pages:<br>            text += page.extract_text() or &quot;&quot;<br>    return text<br><br>def read_file(path: Path) -&gt; str:<br>    &quot;&quot;&quot;Read .txt, .md, .pdf files.&quot;&quot;&quot;<br>    suffix = path.suffix.lower()<br>    if suffix == &quot;.pdf&quot;:<br>        return read_pdf(path)<br>    return path.read_text(encoding=&quot;utf-8&quot;, errors=&quot;ignore&quot;)<br><br># ── Chunking ─────────────────────────────────────────────────────────────────<br>def chunk_text(text: str) -&gt; List[str]:<br>    &quot;&quot;&quot;Split text into overlapping word-based chunks.&quot;&quot;&quot;<br>    words  = text.split()<br>    chunks = []<br>    step   = CHUNK_SIZE - CHUNK_OVERLAP<br>    for i in range(0, len(words), step):<br>        chunk = &quot; &quot;.join(words[i : i + CHUNK_SIZE])<br>        if len(chunk.strip()) &gt; 50:   # Skip tiny chunks<br>            chunks.append(chunk)<br>    return chunks<br><br># ── Ingestion ─────────────────────────────────────────────────────────────────<br>def ingest_directory(docs_dir: str, department: str = &quot;general&quot;):<br>    &quot;&quot;&quot;<br>    Walk a directory, chunk every supported file, embed, and upsert to Qdrant.<br>    &quot;&quot;&quot;<br>    docs_path = Path(docs_dir)<br>    files     = list(docs_path.rglob(&quot;*&quot;))<br>    supported = {&quot;.pdf&quot;, &quot;.txt&quot;, &quot;.md&quot;}<br>    files     = [f for f in files if f.suffix.lower() in supported]<br>    if not files:<br>        print(&quot;No supported files found.&quot;)<br>        return<br>    print(f&quot;Found {len(files)} files. Ingesting...&quot;)<br>    points: List[PointStruct] = []<br>    for file_path in tqdm(files, desc=&quot;Processing files&quot;):<br>        raw_text = read_file(file_path)<br>        chunks   = chunk_text(raw_text)<br>        if not chunks:<br>            continue<br>        # Batch embed all chunks from this file at once<br>        embeddings = encoder.encode(chunks, batch_size=64, show_progress_bar=False)<br>        for idx, (chunk, embedding) in enumerate(zip(chunks, embeddings)):<br>            points.append(PointStruct(<br>                id=str(uuid.uuid4()),<br>                vector=embedding.tolist(),<br>                payload={<br>                    &quot;text&quot;:        chunk,<br>                    &quot;source&quot;:      str(file_path.name),<br>                    &quot;source_path&quot;: str(file_path),<br>                    &quot;department&quot;:  department,<br>                    &quot;chunk_index&quot;: idx,<br>                    &quot;doc_type&quot;:    file_path.suffix.lstrip(&quot;.&quot;),<br>                    &quot;char_count&quot;:  len(chunk),<br>                }<br>            ))<br>        # Upsert in batches of 256<br>        if len(points) &gt;= 256:<br>            client.upsert(collection_name=COLLECTION_NAME, points=points, wait=True)<br>            points = []<br>    # Flush remaining<br>    if points:<br>        client.upsert(collection_name=COLLECTION_NAME, points=points, wait=True)<br>    info = client.get_collection(COLLECTION_NAME)<br>    print(f&quot;\n✅ Ingestion complete. Total vectors: {info.points_count}&quot;)<br><br>if __name__ == &quot;__main__&quot;:<br>    create_collection()<br>    ingest_directory(&quot;./docs&quot;, department=&quot;engineering&quot;)</pre><p>Run it:</p><pre>python ingestion.py</pre><h3>Section 4: The Feedback Loop — Search + Relevance Feedback</h3><p>This is where the magic lives. Two search functions: one plain, one feedback-improved.</p><pre># search.py<br>from typing import List, Dict, Any, Optional<br>from sentence_transformers import SentenceTransformer<br>from qdrant_client import QdrantClient, models<br>from qdrant_client.models import (<br>    SearchParams, QuantizationSearchParams,<br>    RecommendQuery, RecommendInput, RecommendStrategy,<br>    ContextQuery, ContextPair,<br>    Filter, FieldCondition, MatchValue<br>)<br><br>COLLECTION_NAME = &quot;enterprise_kb&quot;<br>client  = QdrantClient(url=&quot;http://localhost:6333&quot;)<br>encoder = SentenceTransformer(&quot;all-MiniLM-L6-v2&quot;)<br><br>def initial_search(<br>    query: str,<br>    top_k: int = 10,<br>    department: Optional[str] = None<br>) -&gt; List[Dict[str, Any]]:<br>    &quot;&quot;&quot;<br>    Standard semantic search. Returns results with their point IDs<br>    so we can use them later for relevance feedback.<br>    &quot;&quot;&quot;<br>    query_vector = encoder.encode(query).tolist()<br>    # Optional department filter<br>    query_filter = None<br>    if department:<br>        query_filter = Filter(<br>            must=[FieldCondition(<br>                key=&quot;department&quot;,<br>                match=MatchValue(value=department)<br>            )]<br>        )<br>    hits = client.search(<br>        collection_name=COLLECTION_NAME,<br>        query_vector=query_vector,<br>        query_filter=query_filter,<br>        limit=top_k,<br>        search_params=SearchParams(<br>            quantization=QuantizationSearchParams(<br>                ignore=False,<br>                rescore=True,     # Rescore INT8 results with FP32 for accuracy<br>                oversampling=2.0  # Fetch 2x candidates, rescore, return top_k<br>            )<br>        ),<br>        with_payload=True,<br>        with_vectors=False<br>    )<br>    return [<br>        {<br>            &quot;id&quot;:         str(hit.id),<br>            &quot;score&quot;:      round(hit.score, 4),<br>            &quot;text&quot;:       hit.payload.get(&quot;text&quot;, &quot;&quot;),<br>            &quot;source&quot;:     hit.payload.get(&quot;source&quot;, &quot;&quot;),<br>            &quot;department&quot;: hit.payload.get(&quot;department&quot;, &quot;&quot;),<br>        }<br>        for hit in hits<br>    ]<br><br>def feedback_search(<br>    positive_ids: List[str],<br>    negative_ids: List[str],<br>    top_k: int = 10,<br>    strategy: str = &quot;average_vector&quot;<br>) -&gt; List[Dict[str, Any]]:<br>    &quot;&quot;&quot;<br>    Relevance Feedback search using Qdrant v1.17&#39;s Recommend API.<br>    How it works:<br>      average_vector strategy:<br>        new_query = avg(positives) + (avg(positives) - avg(negatives))<br>        → Amplifies dimensions strong in positives, penalizes dims strong in negatives<br>      best_score strategy:<br>        For each candidate: score = best match to positives vs best match to negatives<br>        → Stricter; better when positives cover different sub-topics<br>    Args:<br>        positive_ids: IDs of results the user marked helpful (👍)<br>        negative_ids: IDs of results the user marked not helpful (👎)<br>    &quot;&quot;&quot;<br>    strat_map = {<br>        &quot;average_vector&quot;: RecommendStrategy.AVERAGE_VECTOR,<br>        &quot;best_score&quot;:     RecommendStrategy.BEST_SCORE,<br>    }<br>    hits = client.query_points(<br>        collection_name=COLLECTION_NAME,<br>        query=RecommendQuery(<br>            recommend=RecommendInput(<br>                positive=positive_ids,<br>                negative=negative_ids if negative_ids else None,<br>                strategy=strat_map.get(strategy, RecommendStrategy.AVERAGE_VECTOR)<br>            )<br>        ),<br>        limit=top_k,<br>        with_payload=True<br>    )<br>    return [<br>        {<br>            &quot;id&quot;:         str(hit.id),<br>            &quot;score&quot;:      round(hit.score, 4),<br>            &quot;text&quot;:       hit.payload.get(&quot;text&quot;, &quot;&quot;),<br>            &quot;source&quot;:     hit.payload.get(&quot;source&quot;, &quot;&quot;),<br>            &quot;department&quot;: hit.payload.get(&quot;department&quot;, &quot;&quot;),<br>        }<br>        for hit in hits.points<br>    ]<br><br>def context_pair_search(<br>    feedback_pairs: List[tuple],   # [(positive_id, negative_id), ...]<br>    top_k: int = 10<br>) -&gt; List[Dict[str, Any]]:<br>    &quot;&quot;&quot;<br>    Advanced: Context-pair search (v1.17).<br>    Each pair defines a hyperplane in vector space - the search prefers<br>    candidates that land in the most positive zones across all pairs.<br>    Useful when feedback spans multiple distinct sub-topics.<br>    Args:<br>        feedback_pairs: List of (liked_id, disliked_id) tuples<br>    &quot;&quot;&quot;<br>    context = [<br>        ContextPair(positive=pos_id, negative=neg_id)<br>        for pos_id, neg_id in feedback_pairs<br>    ]<br>    hits = client.query_points(<br>        collection_name=COLLECTION_NAME,<br>        query=ContextQuery(context=context),<br>        limit=top_k,<br>        with_payload=True<br>    )<br>    return [<br>        {<br>            &quot;id&quot;:     str(hit.id),<br>            &quot;score&quot;:  round(hit.score, 4),<br>            &quot;text&quot;:   hit.payload.get(&quot;text&quot;, &quot;&quot;),<br>            &quot;source&quot;: hit.payload.get(&quot;source&quot;, &quot;&quot;),<br>        }<br>        for hit in hits.points<br>    ]</pre><h3>Section 5: Full Streamlit App</h3><p>Everything above comes together here. Users search, rate results, and watch the search improve in real time.</p><p><strong>NOTE: </strong>Vibe coded the below app code for rapid prototyping</p><pre># app.py<br>import streamlit as st<br>from search import initial_search, feedback_search, context_pair_search<br><br># ── Page Config ───────────────────────────────────────────────────────────────<br>st.set_page_config(<br>    page_title=&quot;Enterprise Search&quot;,<br>    page_icon=&quot;🔍&quot;,<br>    layout=&quot;wide&quot;<br>)<br># ── Session State Init ────────────────────────────────────────────────────────<br>defaults = {<br>    &quot;results&quot;:        [],<br>    &quot;improved&quot;:       [],<br>    &quot;positives&quot;:      set(),<br>    &quot;negatives&quot;:      set(),<br>    &quot;last_query&quot;:     &quot;&quot;,<br>    &quot;feedback_pairs&quot;: [],   # For context-pair mode<br>    &quot;show_compare&quot;:   False,<br>}<br>for key, val in defaults.items():<br>    if key not in st.session_state:<br>        st.session_state[key] = val<br><br># ── Helpers ───────────────────────────────────────────────────────────────────<br>def reset_feedback():<br>    st.session_state.positives      = set()<br>    st.session_state.negatives      = set()<br>    st.session_state.improved       = []<br>    st.session_state.feedback_pairs = []<br>    st.session_state.show_compare   = False<br><br>def render_result_card(result: dict, index: int, show_feedback: bool = True):<br>    &quot;&quot;&quot;Render a single result card with optional feedback buttons.&quot;&quot;&quot;<br>    result_id = result[&quot;id&quot;]<br>    score     = result[&quot;score&quot;]<br>    text      = result[&quot;text&quot;]<br>    source    = result[&quot;source&quot;]<br>    is_positive = result_id in st.session_state.positives<br>    is_negative = result_id in st.session_state.negatives<br>    # Card border color based on feedback state<br>    if is_positive:<br>        border = &quot;2px solid #2ecc71&quot;<br>    elif is_negative:<br>        border = &quot;2px solid #e74c3c&quot;<br>    else:<br>        border = &quot;1px solid #ddd&quot;<br>    with st.container():<br>        st.markdown(<br>            f&quot;&quot;&quot;<br>            &lt;div style=&quot;<br>                border: {border};<br>                border-radius: 8px;<br>                padding: 14px 18px;<br>                margin-bottom: 10px;<br>                background: #1e1e2e;<br>            &quot;&gt;<br>                &lt;div style=&quot;display:flex; justify-content:space-between; align-items:center;&quot;&gt;<br>                    &lt;span style=&quot;font-size:13px; color:#888;&quot;&gt;<br>                        #{index + 1} &amp;nbsp;|&amp;nbsp; Score: &lt;b&gt;{score}&lt;/b&gt; &amp;nbsp;|&amp;nbsp; 📄 {source}<br>                    &lt;/span&gt;<br>                &lt;/div&gt;<br>                &lt;p style=&quot;margin: 8px 0 0; font-size:14px; line-height:1.6; color:#cdd6f4;&quot;&gt;<br>                    {text[:350]}{&quot;...&quot; if len(text) &gt; 350 else &quot;&quot;}<br>                &lt;/p&gt;<br>            &lt;/div&gt;<br>            &quot;&quot;&quot;,<br>            unsafe_allow_html=True<br>        )<br>        if show_feedback:<br>            col1, col2, col3 = st.columns([1, 1, 8])<br>            with col1:<br>                label = &quot;✅ Liked&quot; if is_positive else &quot;👍 Helpful&quot;<br>                if st.button(label, key=f&quot;pos_{result_id}_{index}&quot;):<br>                    st.session_state.positives.discard(result_id)<br>                    st.session_state.negatives.discard(result_id)<br>                    if not is_positive:<br>                        st.session_state.positives.add(result_id)<br>                    st.rerun()<br>            with col2:<br>                label = &quot;❌ Flagged&quot; if is_negative else &quot;👎 Not helpful&quot;<br>                if st.button(label, key=f&quot;neg_{result_id}_{index}&quot;):<br>                    st.session_state.negatives.discard(result_id)<br>                    st.session_state.positives.discard(result_id)<br>                    if not is_negative:<br>                        st.session_state.negatives.add(result_id)<br>                    st.rerun()<br><br># ── Header ────────────────────────────────────────────────────────────────────<br>st.markdown(&quot;## 🔍 Enterprise Knowledge Search&quot;)<br>st.caption(&quot;Powered by Qdrant v1.17 Relevance Feedback · Results improve as you rate them&quot;)<br>st.divider()<br><br># ── Search Bar ────────────────────────────────────────────────────────────────<br>col_input, col_dept, col_btn = st.columns([5, 2, 1])<br>with col_input:<br>    query = st.text_input(<br>        &quot;Search query&quot;,<br>        placeholder=&quot;e.g. How do I reset user permissions?&quot;,<br>        label_visibility=&quot;collapsed&quot;<br>    )<br>with col_dept:<br>    department = st.selectbox(<br>        &quot;Department&quot;,<br>        [&quot;All&quot;, &quot;engineering&quot;, &quot;hr&quot;, &quot;finance&quot;, &quot;legal&quot;, &quot;product&quot;],<br>        label_visibility=&quot;collapsed&quot;<br>    )<br>with col_btn:<br>    search_clicked = st.button(&quot;Search&quot;, type=&quot;primary&quot;, use_container_width=True)<br><br># ── Trigger Search ────────────────────────────────────────────────────────────<br>if search_clicked and query.strip():<br>    if query != st.session_state.last_query:<br>        reset_feedback()<br>    st.session_state.last_query = query<br>    dept_filter = None if department == &quot;All&quot; else department<br>    with st.spinner(&quot;Searching...&quot;):<br>        st.session_state.results = initial_search(<br>            query=query,<br>            top_k=10,<br>            department=dept_filter<br>        )<br>    st.session_state.improved     = []<br>    st.session_state.show_compare = False<br><br># ── Results ───────────────────────────────────────────────────────────────────<br>if st.session_state.results:<br>    # Feedback summary bar<br>    pos_count = len(st.session_state.positives)<br>    neg_count = len(st.session_state.negatives)<br>    if pos_count &gt; 0 or neg_count &gt; 0:<br>        st.info(<br>            f&quot;📊 Feedback collected: **{pos_count} helpful** · **{neg_count} not helpful** &quot;<br>            f&quot;- Click &#39;Improve Results&#39; to apply.&quot;<br>        )<br>    # Strategy selector<br>    strategy = st.radio(<br>        &quot;Feedback strategy:&quot;,<br>        [&quot;average_vector&quot;, &quot;best_score&quot;, &quot;context_pairs&quot;],<br>        horizontal=True,<br>        help=(<br>            &quot;average_vector: Fast, good for focused queries. &quot;<br>            &quot;best_score: Stricter, better for broad multi-topic queries. &quot;<br>            &quot;context_pairs: v1.17 exclusive - pairs define vector space zones.&quot;<br>        )<br>    )<br>    # Improve / Compare buttons<br>    col_a, col_b, col_c = st.columns([2, 2, 6])<br>    with col_a:<br>        improve_clicked = st.button(<br>            &quot;✨ Improve Results&quot;,<br>            disabled=(pos_count == 0),<br>            type=&quot;primary&quot;,<br>            use_container_width=True<br>        )<br>    with col_b:<br>        if st.session_state.improved:<br>            compare_clicked = st.button(<br>                &quot;⚖️ Compare&quot;,<br>                use_container_width=True<br>            )<br>            if compare_clicked:<br>                st.session_state.show_compare = not st.session_state.show_compare<br>    # Apply relevance feedback<br>    if improve_clicked and pos_count &gt; 0:<br>        pos_ids = list(st.session_state.positives)<br>        neg_ids = list(st.session_state.negatives)<br>        with st.spinner(&quot;Applying relevance feedback...&quot;):<br>            if strategy == &quot;context_pairs&quot; and pos_ids and neg_ids:<br>                # Pair up positives and negatives (zip, so lengths can differ)<br>                pairs = list(zip(pos_ids, neg_ids))<br>                st.session_state.improved = context_pair_search(<br>                    feedback_pairs=pairs,<br>                    top_k=10<br>                )<br>            else:<br>                st.session_state.improved = feedback_search(<br>                    positive_ids=pos_ids,<br>                    negative_ids=neg_ids,<br>                    top_k=10,<br>                    strategy=strategy<br>                )<br>        st.session_state.show_compare = True<br>        st.success(&quot;Results improved based on your feedback!&quot;)<br>        st.rerun()<br>    st.divider()<br>    # ── Side-by-Side Compare or Single View ──────────────────────────────────<br>    if st.session_state.show_compare and st.session_state.improved:<br>        col_orig, col_improved = st.columns(2)<br>        with col_orig:<br>            st.markdown(&quot;### Original Results&quot;)<br>            for i, result in enumerate(st.session_state.results):<br>                render_result_card(result, i, show_feedback=False)<br>        with col_improved:<br>            st.markdown(&quot;### ✨ Improved Results&quot;)<br>            for i, result in enumerate(st.session_state.improved):<br>                render_result_card(result, i, show_feedback=False)<br>    else:<br>        # Single view - show results with feedback buttons<br>        tabs = st.tabs([&quot;🔍 Results&quot;, &quot;✨ Improved&quot;] if st.session_state.improved else [&quot;🔍 Results&quot;])<br>        with tabs[0]:<br>            st.markdown(f&quot;**{len(st.session_state.results)} results** for *\&quot;{st.session_state.last_query}\&quot;*&quot;)<br>            for i, result in enumerate(st.session_state.results):<br>                render_result_card(result, i, show_feedback=True)<br>        if st.session_state.improved and len(tabs) &gt; 1:<br>            with tabs[1]:<br>                st.markdown(&quot;*Refined using your feedback signals:*&quot;)<br>                for i, result in enumerate(st.session_state.improved):<br>                    render_result_card(result, i, show_feedback=True)<br><br># ── Empty State ───────────────────────────────────────────────────────────────<br>elif not st.session_state.results and st.session_state.last_query:<br>    st.warning(&quot;No results found. Try a different query.&quot;)<br>elif not st.session_state.last_query:<br>    st.markdown(&quot;&quot;&quot;<br>    &lt;div style=&quot;text-align:center; padding: 60px 0; color: #888;&quot;&gt;<br>        &lt;h3&gt;🔍 Search your knowledge base&lt;/h3&gt;<br>        &lt;p&gt;Rate results with 👍 / 👎 and watch the engine improve in real time.&lt;/p&gt;<br>        &lt;p style=&quot;font-size:12px;&quot;&gt;Powered by Qdrant v1.17 Relevance Feedback - no retraining, no ML pipeline.&lt;/p&gt;<br>    &lt;/div&gt;<br>    &quot;&quot;&quot;, unsafe_allow_html=True)<br><br># ── Sidebar: How It Works ─────────────────────────────────────────────────────<br>with st.sidebar:<br>    st.markdown(&quot;### ⚙️ How It Works&quot;)<br>    st.markdown(&quot;&quot;&quot;<br>    **Step 1:** Search returns results from Qdrant using semantic similarity.<br>    **Step 2:** Rate results - 👍 for helpful, 👎 for not helpful.<br>    **Step 3:** Click &quot;Improve Results&quot;. Qdrant&#39;s Relevance Feedback API reshapes the query:<br>    ```<br>    new_query =<br>      avg(positives) +<br>      (avg(positives) - avg(negatives))<br>    ```<br>    Dimensions strong in your liked results get **amplified**. Dimensions strong in disliked results get **penalized**.<br>    **No retraining. No ML pipeline. Just vector math.**<br>    &quot;&quot;&quot;)<br>    st.divider()<br>    st.markdown(&quot;### 📊 Session Stats&quot;)<br>    st.metric(&quot;Total Results Fetched&quot;, len(st.session_state.results))<br>    st.metric(&quot;Helpful Signals&quot;, len(st.session_state.positives))<br>    st.metric(&quot;Negative Signals&quot;, len(st.session_state.negatives))<br>    st.metric(&quot;Improved Searches&quot;, 1 if st.session_state.improved else 0)<br>    st.divider()<br>    if st.button(&quot;🗑️ Reset Session&quot;):<br>        for key in list(st.session_state.keys()):<br>            del st.session_state[key]<br>        st.rerun()</pre><p>Run it:</p><pre>streamlit run app.py</pre><h3>What Just Changed</h3><p>Static enterprise search is a solved problem — solved badly. Every vendor ships the same cosine similarity, the same ANN index, the same results forever.</p><p>Qdrant v1.17 shifts the paradigm. Instead of throwing away user signals, you keep them. Instead of expensive fine-tuning loops, you run vector arithmetic at query time. The formula:</p><pre>new_query = avg(positives) + (avg(positives) - avg(negatives))</pre><p>…takes milliseconds. It needs no GPU. No labels. No model updates. And it works on <strong>billions of vectors</strong> because it modifies the scoring function, not the data.</p><p>The first user who rates a result doesn’t just help themselves. They’re setting a signal in vector space that bends every future search with that intent toward better answers.</p><p>That’s not a search engine anymore. That’s a search engine that learns.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18145dc91070" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/we-replaced-a-500k-ml-retraining-pipeline-with-3-lines-of-vector-math-18145dc91070">We Replaced a $500K ML Retraining Pipeline with 3 Lines of Vector Math</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[pplx-embed + Qdrant: Building Production-Grade Semantic Search with Quantization]]></title>
            <link>https://pub.towardsai.net/pplx-embed-qdrant-building-production-grade-semantic-search-with-quantization-cea9626b8c7c?source=rss-4d80f7255174------2</link>
            <guid isPermaLink="false">https://medium.com/p/cea9626b8c7c</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Niranjan Akella]]></dc:creator>
            <pubDate>Mon, 02 Mar 2026 10:28:10 GMT</pubDate>
            <atom:updated>2026-03-05T04:49:57.922Z</atom:updated>
            <content:encoded><![CDATA[<h4>How Perplexity’s new diffusion-based multilingual embedding models pair with Qdrant’s vector database for web-scale retrieval — complete with setup, Python code, quantization strategies, and everything new in Qdrant v1.17</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P_IG71vHLoAt6JoiKnbwyw.png" /><figcaption>Perplexity x Qdrant x HF x Author</figcaption></figure><h3>Introduction</h3><p>On February 26, 2026, Perplexity released pplx-embed — a new family of state-of-the-art multilingual text embedding models that fundamentally rethink how dense embeddings are generated. Built on Qwen3 with diffusion-based pretraining and native INT8/binary quantization, the models are purpose-built for real-world, web-scale retrieval tasks over tens of millions of documents.</p><p>At the same time, Qdrant — the leading open-source vector database — released version 1.17, bringing major improvements in search latency, relevance feedback, write-load performance, and operational observability.</p><p>This article walks you through everything you need to know:</p><ul><li>What makes pplx-embed architecturally different from other embedding models</li><li>Full Qdrant setup from scratch (Docker, local, Python client)</li><li>Production-ready Python code for embedding and retrieval using pplx-embed</li><li>Deep dive into Qdrant’s quantization strategies (Scalar, Binary, Product)</li><li>Complete walkthrough of Qdrant v1.17’s breaking changes and new features</li></ul><p>By the end, you’ll have a production-grade semantic search system running on your machine.</p><h3>Part 1: Understanding pplx-embed</h3><h3>What Is pplx-embed?</h3><p>pplx-embed is Perplexity&#39;s new collection of embedding models, released under the <strong>MIT License</strong> and available on the <strong>Hugging Face Hub</strong>. There are two model families:</p><p>ModelPurposepplx-embed-v1Standard dense text embeddings for independent textspplx-embed-context-v1Contextual embeddings for document chunks in RAG pipelines</p><p>Each family is available at two scales: Source Perplexity</p><p>ModelDimensionsContextQuantizationPrice ($/1M tokens)<br>pplx-embed-v1-0.6b1024 32KINT8 / BINARY → $0.004<br>pplx-embed-v1-4b 2560 32KINT8 / BINARY → $0.030<br>pplx-embed-context-v1-0.6b1024 32KINT8 / BINARY → $0.008<br>pplx-embed-context-v1-4b2560 32KINT8 / BINARY → $0.050</p><h3>The Core Innovation: Diffusion-Based Bidirectional Pretraining</h3><p>Most embedding models today are built on <strong>decoder-only LLMs with causal attention</strong> — meaning each token can only attend to preceding tokens. For retrieval, this is a fundamental architectural limitation: understanding a passage often requires knowing what comes <em>after</em> a word, not just before it.</p><p>pplx-embed addresses this with <strong>diffusion-based continued pretraining</strong>, which converts a causal LLM (Qwen3) into a <strong>bidirectional encoder</strong>. This allows:</p><ul><li>Full bidirectional attention across all tokens</li><li>Mean pooling over <em>all</em> token representations (not just the last)</li><li>Late chunking for contextual embeddings — where each chunk’s embedding is informed by the full document context it appears in</li></ul><p>This is why pplx-embed-context-v1 can understand that a sentence like &quot;he scored the winning goal&quot; means something very different in a football article vs. a cricket article — the full document context is baked into the chunk embedding.</p><h3>Multi-Stage Training Pipeline</h3><p>The training pipeline runs in distinct phases:</p><ol><li><strong>English-only pretraining</strong> — Builds base retrieval capability</li><li><strong>Cross-lingual training</strong> — Extends to multilingual retrieval</li><li><strong>Fully multilingual training</strong> — Adds support for 29+ languages</li><li><strong>Contextual training (pplx-embed-context-v1)</strong> — Dual loss combining in-sequence and in-batch contrast at both chunk and document levels</li><li><strong>Triplet training</strong> — Hard negative mining to sharpen decision boundaries between similar-but-non-relevant documents</li><li><strong>SLERP merging</strong> — The final pplx-embed-v1 is produced by merging the contextual and triplet checkpoints via <strong>Spherical Linear Interpolation (SLERP)</strong></li></ol><h3>Benchmark Performance</h3><p>On <strong>MTEB (Multilingual, v2)</strong> — the gold standard for multilingual retrieval:</p><ul><li>pplx-embed-v1-4B (INT8): <strong>69.66% nDCG@10</strong> — matches Qwen3-Embedding-4B (69.60%), beats Gemini-embedding-001 (67.71%)</li><li>pplx-embed-v1-0.6B outperforms Qwen3-Embedding-0.6B at the same parameter scale</li></ul><p>On <strong>ConTEB</strong> (contextual retrieval benchmark):</p><ul><li>pplx-embed-context-v1-4B (INT8): <strong>81.96% nDCG@10</strong> — beats voyage-context-3 (79.45%) and Anthropic Contextual (72.4%)</li></ul><p>On Perplexity’s internal <strong>PPLXQuery2Doc</strong> (30M corpus):</p><ul><li>pplx-embed-v1-4B: <strong>91.7% Recall@1000</strong> vs Qwen3-Embedding-4B at 88.6%</li></ul><h3>Native Quantization: Storage Efficiency Built In</h3><p>Unlike most models that require post-hoc quantization, pplx-embed produces <strong>INT8 and binary embeddings natively</strong>:</p><p>FormatStorage Savings vs FP32FP32BaselineINT8<strong>4x reduction</strong>BINARY<strong>32x reduction</strong></p><p>At web scale (billions of documents), this isn’t just a nice-to-have — it’s the difference between feasible and infeasible deployment.</p><h3>Supported Inference Frameworks</h3><ul><li>HuggingFace Transformers</li><li>SentenceTransformers</li><li>Text Embeddings Inference (TEI)</li><li>Transformers.js (browser/Node.js)</li><li>ONNX Runtime</li><li>Perplexity API (hosted)</li></ul><h3>Part 2: Setting Up Qdrant Vector Database</h3><p>Qdrant is an open-source, high-performance vector database written in Rust. It supports HNSW indexing, multiple distance metrics, payload filtering, sparse vectors, quantization, and distributed deployments.</p><h3>Option 1: Docker (Recommended for Production)</h3><pre>docker run -d \<br>  --name qdrant \<br>  -p 6333:6333 \<br>  -p 6334:6334 \<br>  -v $(pwd)/qdrant_storage:/qdrant/storage \<br>  qdrant/qdrant</pre><p>This exposes:</p><ul><li><strong>Port 6333</strong>: REST API</li><li><strong>Port 6334</strong>: gRPC API</li><li><strong>qdrant_storage/</strong>: Persistent data volume</li></ul><h3>Option 2: Docker Compose (Recommended with Configuration)</h3><p>Create a docker-compose.yml:</p><pre>version: &quot;3.8&quot;</pre><pre>services:<br>  qdrant:<br>    image: qdrant/qdrant:v1.17.0<br>    container_name: qdrant<br>    ports:<br>      - &quot;6333:6333&quot;<br>      - &quot;6334:6334&quot;<br>    volumes:<br>      - ./qdrant_storage:/qdrant/storage<br>      - ./qdrant_config.yaml:/qdrant/config/production.yaml<br>    environment:<br>      - QDRANT__SERVICE__GRPC_PORT=6334<br>    restart: unless-stopped</pre><p>Optional qdrant_config.yaml:</p><pre>storage:<br>  on_disk_payload: true  # Store payload on disk to save RAM</pre><pre>service:<br>  max_request_size_mb: 32<br>  <br>log_level: INFO</pre><p>Start it:</p><pre>docker compose up -d</pre><h3>Option 3: Local Binary (macOS/Linux)</h3><pre># Download latest release<br>curl -LO https://github.com/qdrant/qdrant/releases/latest/download/qdrant-x86_64-unknown-linux-gnu.tar.gz<br>tar -xzf qdrant-*.tar.gz<br>./qdrant</pre><h3>Option 4: Qdrant Cloud (Fully Managed)</h3><p>Visit <a href="https://cloud.qdrant.io/">cloud.qdrant.io</a> → Create a free cluster → Get your API key and cluster URL.</p><h3>Install the Python Client</h3><pre>pip install qdrant-client openai sentence-transformers transformers torch</pre><p>For Perplexity’s hosted API:</p><pre>pip install qdrant-client openai  # Perplexity uses OpenAI-compatible API</pre><h3>Verify Connection</h3><pre>from qdrant_client import QdrantClient</pre><pre># Local Docker instance<br>client = QdrantClient(host=&quot;localhost&quot;, port=6333)</pre><pre># OR Qdrant Cloud<br># client = QdrantClient(<br>#     url=&quot;https://your-cluster-url.qdrant.io&quot;,<br>#     api_key=&quot;your-api-key&quot;<br># )</pre><pre># Health check<br>info = client.get_collections()<br>print(f&quot;✅ Connected to Qdrant | Collections: {len(info.collections)}&quot;)</pre><h3>Part 3: Using pplx-embed with Qdrant — Complete Python Code</h3><h3>Setup: API Keys and Imports</h3><pre>import os<br>from openai import OpenAI<br>from qdrant_client import QdrantClient<br>from qdrant_client.models import (<br>    Distance,<br>    VectorParams,<br>    PointStruct,<br>    ScalarQuantizationConfig,<br>    ScalarType,<br>    QuantizationSearchParams,<br>    SearchParams,<br>    Filter,<br>    FieldCondition,<br>    MatchValue,<br>)<br>import uuid<br>import time<br>from typing import List, Dict, Any, Optional</pre><pre># Perplexity API Client (OpenAI-compatible)<br>pplx_client = OpenAI(<br>    api_key=os.environ.get(&quot;PERPLEXITY_API_KEY&quot;),<br>    base_url=&quot;https://api.perplexity.ai&quot;<br>)</pre><pre># Qdrant Client<br>qdrant = QdrantClient(host=&quot;localhost&quot;, port=6333)</pre><pre># Choose your model<br>EMBEDDING_MODEL = &quot;pplx-embed-v1-4b&quot;          # Best quality, independent texts<br># EMBEDDING_MODEL = &quot;pplx-embed-v1-0.6b&quot;      # Fast, lightweight<br># EMBEDDING_MODEL = &quot;pplx-embed-context-v1-4b&quot; # Best for RAG document chunks</pre><pre>VECTOR_DIM = 2560   # 4B model: 2560 dims | 0.6B model: 1024 dims<br>COLLECTION_NAME = &quot;knowledge_base&quot;</pre><h3>Step 1: Generate Embeddings with pplx-embed</h3><pre>def embed_texts(texts: List[str], model: str = EMBEDDING_MODEL) -&gt; List[List[float]]:<br>    &quot;&quot;&quot;<br>    Generate embeddings using Perplexity&#39;s pplx-embed API.<br>    Supports batching for efficiency.<br>    &quot;&quot;&quot;<br>    # API supports batch embedding — pass list directly<br>    response = pplx_client.embeddings.create(<br>        model=model,<br>        input=texts,<br>    )<br>    <br>    embeddings = [item.embedding for item in response.data]<br>    return embeddings<br></pre><pre>def embed_single(text: str, model: str = EMBEDDING_MODEL) -&gt; List[float]:<br>    &quot;&quot;&quot;Embed a single text.&quot;&quot;&quot;<br>    return embed_texts([text], model=model)[0]<br></pre><pre># Example usage<br>query = &quot;What are the key benefits of vector databases for semantic search?&quot;<br>embedding = embed_single(query)<br>print(f&quot;Embedding dimensions: {len(embedding)}&quot;)  # → 2560 for 4B model</pre><h3>Step 2: Create a Qdrant Collection with Scalar Quantization</h3><pre>def create_collection(<br>    collection_name: str,<br>    vector_dim: int = VECTOR_DIM,<br>    with_scalar_quantization: bool = True<br>):<br>    &quot;&quot;&quot;<br>    Create a Qdrant collection optimized for pplx-embed vectors.<br>    Includes scalar quantization for memory efficiency.<br>    &quot;&quot;&quot;<br>    # Check if collection already exists<br>    existing = [c.name for c in qdrant.get_collections().collections]<br>    if collection_name in existing:<br>        print(f&quot;Collection &#39;{collection_name}&#39; already exists.&quot;)<br>        return</pre><pre>    # Quantization configuration (see Part 4 for deep dive)<br>    quantization_config = None<br>    if with_scalar_quantization:<br>        quantization_config = ScalarQuantizationConfig(<br>            type=ScalarType.INT8,      # Compress FP32 → INT8 (4x memory reduction)<br>            quantile=0.99,             # Use 99th percentile for calibration<br>            always_ram=True,           # Keep quantized vectors in RAM for speed<br>        )</pre><pre>    qdrant.create_collection(<br>        collection_name=collection_name,<br>        vectors_config=VectorParams(<br>            size=vector_dim,<br>            distance=Distance.COSINE,  # pplx-embed is optimized for cosine similarity<br>            on_disk=False,             # Keep in RAM for fast retrieval<br>        ),<br>        quantization_config=quantization_config,<br>        # Optional: HNSW index configuration<br>        hnsw_config={<br>            &quot;m&quot;: 16,              # Number of connections per layer (16-64)<br>            &quot;ef_construct&quot;: 100,  # Construction accuracy (100-200)<br>            &quot;full_scan_threshold&quot;: 10000,<br>            &quot;on_disk&quot;: False,<br>        }<br>    )<br>    <br>    print(f&quot;✅ Created collection &#39;{collection_name}&#39; with {vector_dim}D vectors&quot;)<br>    if with_scalar_quantization:<br>        print(&quot;   Scalar quantization (INT8) enabled — ~4x memory reduction&quot;)<br></pre><pre># Create our collection<br>create_collection(COLLECTION_NAME)</pre><h3>Step 3: Index Documents</h3><pre>def index_documents(<br>    documents: List[Dict[str, Any]],<br>    collection_name: str = COLLECTION_NAME,<br>    batch_size: int = 100<br>):<br>    &quot;&quot;&quot;<br>    Embed and upsert a list of documents into Qdrant.<br>    <br>    Each document should have:<br>    - &#39;text&#39;: str — the content to embed<br>    - &#39;metadata&#39;: dict — any payload fields (title, source, date, etc.)<br>    &quot;&quot;&quot;<br>    total = len(documents)<br>    print(f&quot;Indexing {total} documents in batches of {batch_size}...&quot;)<br>    <br>    for i in range(0, total, batch_size):<br>        batch = documents[i:i + batch_size]<br>        texts = [doc[&quot;text&quot;] for doc in batch]<br>        <br>        # Generate embeddings for the entire batch<br>        embeddings = embed_texts(texts)<br>        <br>        # Build Qdrant points<br>        points = []<br>        for doc, embedding in zip(batch, embeddings):<br>            point = PointStruct(<br>                id=str(uuid.uuid4()),<br>                vector=embedding,<br>                payload={<br>                    &quot;text&quot;: doc[&quot;text&quot;],<br>                    **doc.get(&quot;metadata&quot;, {})<br>                }<br>            )<br>            points.append(point)<br>        <br>        # Upsert to Qdrant<br>        qdrant.upsert(<br>            collection_name=collection_name,<br>            points=points,<br>            wait=True  # Wait for indexing to complete<br>        )<br>        <br>        print(f&quot;  Indexed batch {i//batch_size + 1}/{(total + batch_size - 1)//batch_size}&quot;)<br>        <br>        # Rate limiting for API calls<br>        time.sleep(0.1)<br>    <br>    # Get final count<br>    collection_info = qdrant.get_collection(collection_name)<br>    print(f&quot;\n✅ Total vectors indexed: {collection_info.points_count}&quot;)<br></pre><pre># Example: Index a sample dataset<br>sample_documents = [<br>    {<br>        &quot;text&quot;: &quot;Vector databases store high-dimensional embeddings and enable efficient approximate nearest neighbor search.&quot;,<br>        &quot;metadata&quot;: {&quot;title&quot;: &quot;Vector DB Overview&quot;, &quot;source&quot;: &quot;docs&quot;, &quot;category&quot;: &quot;database&quot;}<br>    },<br>    {<br>        &quot;text&quot;: &quot;Qdrant is a high-performance vector database written in Rust, designed for production AI applications.&quot;,<br>        &quot;metadata&quot;: {&quot;title&quot;: &quot;Qdrant Introduction&quot;, &quot;source&quot;: &quot;qdrant.tech&quot;, &quot;category&quot;: &quot;database&quot;}<br>    },<br>    {<br>        &quot;text&quot;: &quot;Semantic search uses embedding models to understand the meaning of queries rather than exact keyword matching.&quot;,<br>        &quot;metadata&quot;: {&quot;title&quot;: &quot;Semantic Search Guide&quot;, &quot;source&quot;: &quot;blog&quot;, &quot;category&quot;: &quot;search&quot;}<br>    },<br>    {<br>        &quot;text&quot;: &quot;RAG (Retrieval Augmented Generation) combines vector search with LLMs to provide grounded, accurate answers.&quot;,<br>        &quot;metadata&quot;: {&quot;title&quot;: &quot;RAG Architecture&quot;, &quot;source&quot;: &quot;research&quot;, &quot;category&quot;: &quot;ai&quot;}<br>    },<br>    {<br>        &quot;text&quot;: &quot;INT8 quantization reduces embedding storage by 4x with minimal impact on retrieval quality.&quot;,<br>        &quot;metadata&quot;: {&quot;title&quot;: &quot;Quantization Guide&quot;, &quot;source&quot;: &quot;blog&quot;, &quot;category&quot;: &quot;optimization&quot;}<br>    },<br>]</pre><pre>index_documents(sample_documents)</pre><h3>Step 4: Semantic Search with Quantization-Aware Parameters</h3><pre>def semantic_search(<br>    query: str,<br>    collection_name: str = COLLECTION_NAME,<br>    top_k: int = 5,<br>    use_quantization: bool = True,<br>    rescore: bool = True,<br>    filter_conditions: Optional[Dict] = None,<br>    score_threshold: float = 0.0<br>) -&gt; List[Dict]:<br>    &quot;&quot;&quot;<br>    Perform semantic search using pplx-embed query embedding.<br>    <br>    When quantization is enabled:<br>    - Initial search uses quantized (INT8) vectors for speed<br>    - Rescoring re-ranks top candidates with full FP32 vectors for quality<br>    &quot;&quot;&quot;<br>    # Embed the query<br>    query_embedding = embed_single(query)<br>    <br>    # Build search parameters<br>    search_params = SearchParams(<br>        quantization=QuantizationSearchParams(<br>            ignore=not use_quantization,   # Use quantized vectors in search<br>            rescore=rescore,               # Re-rank with full precision<br>            oversampling=2.0               # Fetch 2x candidates before rescoring<br>        ) if use_quantization else None,<br>        hnsw_ef=128,    # Higher = better recall, slower search (128-256 for prod)<br>        exact=False     # Use HNSW index (True for brute-force/testing)<br>    )<br>    <br>    # Build optional payload filter<br>    query_filter = None<br>    if filter_conditions:<br>        query_filter = Filter(<br>            must=[<br>                FieldCondition(<br>                    key=key,<br>                    match=MatchValue(value=value)<br>                )<br>                for key, value in filter_conditions.items()<br>            ]<br>        )<br>    <br>    # Execute search<br>    results = qdrant.search(<br>        collection_name=collection_name,<br>        query_vector=query_embedding,<br>        query_filter=query_filter,<br>        search_params=search_params,<br>        limit=top_k,<br>        score_threshold=score_threshold,<br>        with_payload=True,<br>        with_vectors=False  # Don&#39;t return raw vectors (saves bandwidth)<br>    )<br>    <br>    # Format results<br>    formatted = []<br>    for i, result in enumerate(results):<br>        formatted.append({<br>            &quot;rank&quot;: i + 1,<br>            &quot;score&quot;: round(result.score, 4),<br>            &quot;id&quot;: result.id,<br>            &quot;text&quot;: result.payload.get(&quot;text&quot;, &quot;&quot;),<br>            &quot;metadata&quot;: {k: v for k, v in result.payload.items() if k != &quot;text&quot;}<br>        })<br>    <br>    return formatted<br></pre><pre># Example search<br>results = semantic_search(<br>    query=&quot;How does quantization reduce memory usage for embeddings?&quot;,<br>    top_k=3,<br>    use_quantization=True,<br>    rescore=True<br>)</pre><pre>for r in results:<br>    print(f&quot;\n#{r[&#39;rank&#39;]} [Score: {r[&#39;score&#39;]}]&quot;)<br>    print(f&quot;   {r[&#39;text&#39;][:100]}...&quot;)<br>    print(f&quot;   Category: {r[&#39;metadata&#39;].get(&#39;category&#39;, &#39;N/A&#39;)}&quot;)</pre><h3>Step 5: RAG Pipeline with pplx-embed-context-v1</h3><p>For document-heavy RAG use cases, pplx-embed-context-v1 is the better choice — it embeds each chunk with awareness of the full document context:</p><pre>from typing import List, Tuple<br>def chunk_document(<br>    text: str,<br>    chunk_size: int = 512,<br>    overlap: int = 50<br>) -&gt; List[str]:<br>    &quot;&quot;&quot;Split document into overlapping chunks.&quot;&quot;&quot;<br>    words = text.split()<br>    chunks = []<br>    <br>    for i in range(0, len(words), chunk_size - overlap):<br>        chunk = &quot; &quot;.join(words[i:i + chunk_size])<br>        if chunk:<br>            chunks.append(chunk)<br>    <br>    return chunks<br><br>def index_document_with_context(<br>    document: str,<br>    document_id: str,<br>    metadata: Dict,<br>    collection_name: str = COLLECTION_NAME,<br>    chunk_size: int = 512<br>):<br>    &quot;&quot;&quot;<br>    Index a full document using pplx-embed-context-v1.<br>    Each chunk is embedded with full document context via &#39;context&#39; field.<br>    &quot;&quot;&quot;<br>    chunks = chunk_document(document, chunk_size=chunk_size)<br>    <br>    # For context-aware embeddings, pass (query, context) tuples<br>    # pplx-embed-context-v1 uses document-level context per chunk<br>    texts_with_context = [<br>        f&quot;Represent the following passage for retrieval: {chunk}&quot;<br>        for chunk in chunks<br>    ]<br>    <br>    embeddings = embed_texts(<br>        texts_with_context,<br>        model=&quot;pplx-embed-context-v1-4b&quot;  # Context-aware model<br>    )<br>    <br>    points = []<br>    for idx, (chunk, embedding) in enumerate(zip(chunks, embeddings)):<br>        points.append(PointStruct(<br>            id=str(uuid.uuid4()),<br>            vector=embedding,<br>            payload={<br>                &quot;text&quot;: chunk,<br>                &quot;document_id&quot;: document_id,<br>                &quot;chunk_index&quot;: idx,<br>                &quot;total_chunks&quot;: len(chunks),<br>                **metadata<br>            }<br>        ))<br>    <br>    qdrant.upsert(<br>        collection_name=collection_name,<br>        points=points,<br>        wait=True<br>    )<br>    <br>    print(f&quot;✅ Indexed {len(chunks)} chunks for document &#39;{document_id}&#39;&quot;)<br><br>def rag_query(<br>    question: str,<br>    top_k: int = 5,<br>    collection_name: str = COLLECTION_NAME<br>) -&gt; Tuple[List[Dict], str]:<br>    &quot;&quot;&quot;<br>    Full RAG pipeline: retrieve relevant chunks, return context for LLM.<br>    &quot;&quot;&quot;<br>    # Retrieve relevant chunks<br>    results = semantic_search(<br>        query=question,<br>        collection_name=collection_name,<br>        top_k=top_k,<br>        use_quantization=True,<br>        rescore=True<br>    )<br>    <br>    # Build context string for LLM<br>    context_parts = []<br>    for r in results:<br>        source = r[&#39;metadata&#39;].get(&#39;title&#39;, &#39;Unknown&#39;)<br>        context_parts.append(f&quot;[Source: {source}]\n{r[&#39;text&#39;]}&quot;)<br>    <br>    context = &quot;\n\n---\n\n&quot;.join(context_parts)<br>    <br>    return results, context</pre><h3>Part 4: Quantization Deep Dive with Qdrant</h3><p>Quantization compresses high-dimensional float vectors into compact representations, dramatically reducing memory usage and improving search speed. Qdrant supports three quantization strategies, each with different trade-offs.</p><h3>Understanding the Trade-offs</h3><p>MethodCompressionSpeed GainRecall LossBest <br>For, <br><strong>Scalar (INT8) </strong>4x2–3x → ~1–3% Production default <br><strong>Binary </strong>32x10–40x → ~5–15% Large-scale pre-filtering <br><strong>Product </strong>Variable (6–32x)4–8x → ~2–8% Extreme compression needs</p><h3>Strategy 1: Scalar Quantization (INT8) — Production Default</h3><p>Scalar quantization maps each float32 value to an int8 value (256 levels). With pplx-embed&#39;s native INT8 output, Qdrant can use these directly without any quality loss from re-quantization.</p><pre>from qdrant_client.models import (<br>    ScalarQuantizationConfig,<br>    ScalarType,<br>)<br><br>def create_collection_scalar_quantization(collection_name: str, vector_dim: int):<br>    &quot;&quot;&quot;<br>    Create collection with INT8 scalar quantization.<br>    Best balance of speed, compression, and recall.<br>    &quot;&quot;&quot;<br>    qdrant.create_collection(<br>        collection_name=collection_name,<br>        vectors_config=VectorParams(<br>            size=vector_dim,<br>            distance=Distance.COSINE,<br>        ),<br>        quantization_config=ScalarQuantizationConfig(<br>            type=ScalarType.INT8,<br>            <br>            # Quantile: What % of values to use for calibration range<br>            # 0.99 = Use 1st-99th percentile (clip outliers)<br>            # 1.00 = Use full range (may waste precision on outliers)<br>            quantile=0.99,<br>            <br>            # always_ram: Keep quantized vectors in RAM for fast search<br>            # Full-precision vectors can live on disk for rescoring<br>            always_ram=True,<br>        )<br>    )<br>    print(f&quot;✅ Created &#39;{collection_name}&#39; with INT8 scalar quantization&quot;)<br><br><br># Search with rescoring for maximum quality<br>def search_with_scalar_rescoring(<br>    query_vector: List[float],<br>    collection_name: str,<br>    top_k: int = 10<br>):<br>    &quot;&quot;&quot;<br>    Two-phase search: fast INT8 retrieval + FP32 rescoring.<br>    <br>    Phase 1: Retrieve top_k * oversampling_factor candidates using INT8 vectors<br>    Phase 2: Rescore with full FP32 vectors and return final top_k<br>    &quot;&quot;&quot;<br>    return qdrant.search(<br>        collection_name=collection_name,<br>        query_vector=query_vector,<br>        limit=top_k,<br>        search_params=SearchParams(<br>            quantization=QuantizationSearchParams(<br>                ignore=False,          # Use INT8 quantized vectors<br>                rescore=True,          # Rescore with FP32 for accuracy<br>                oversampling=2.0       # Retrieve 2x candidates before rescoring<br>            ),<br>            hnsw_ef=128<br>        ),<br>        with_payload=True<br>    )</pre><h3>Strategy 2: Binary Quantization — Maximum Compression</h3><p>Binary quantization maps each float to a single bit (positive → 1, negative → 0). This is <strong>32x compression</strong> — ideal for first-stage retrieval over billions of documents. Always combine with rescoring.</p><pre>from qdrant_client.models import BinaryQuantizationConfig<br><br>def create_collection_binary_quantization(collection_name: str, vector_dim: int):<br>    &quot;&quot;&quot;<br>    Create collection with binary quantization.<br>    32x compression — ideal for billion-scale pre-filtering.<br>    Requires rescoring for acceptable recall.<br>    <br>    Note: Works best with models trained for binary quantization (like pplx-embed).<br>    &quot;&quot;&quot;<br>    qdrant.create_collection(<br>        collection_name=collection_name,<br>        vectors_config=VectorParams(<br>            size=vector_dim,<br>            distance=Distance.COSINE,<br>        ),<br>        quantization_config=BinaryQuantizationConfig(<br>            # always_ram: Critical for binary quantization performance<br>            # Binary vectors are tiny — always keep in RAM<br>            always_ram=True,<br>        )<br>    )<br>    print(f&quot;✅ Created &#39;{collection_name}&#39; with binary quantization (32x compression)&quot;)<br><br><br>def search_with_binary_quantization(<br>    query_vector: List[float],<br>    collection_name: str,<br>    top_k: int = 10<br>):<br>    &quot;&quot;&quot;<br>    Binary quantization search with aggressive oversampling.<br>    Binary search is so fast that you can afford to retrieve many candidates.<br>    &quot;&quot;&quot;<br>    return qdrant.search(<br>        collection_name=collection_name,<br>        query_vector=query_vector,<br>        limit=top_k,<br>        search_params=SearchParams(<br>            quantization=QuantizationSearchParams(<br>                ignore=False,<br>                rescore=True,        # REQUIRED: Binary recall is poor without rescoring<br>                oversampling=5.0     # Retrieve 5x candidates due to lower precision<br>            ),<br>            hnsw_ef=256            # Higher ef for better recall with binary vectors<br>        ),<br>        with_payload=True<br>    )</pre><h3>Strategy 3: Product Quantization — Configurable Compression</h3><p>Product quantization divides each vector into subspaces and quantizes each independently. More flexible but computationally intensive.</p><pre>from qdrant_client.models import ProductQuantizationConfig, CompressionRatio<br><br>def create_collection_product_quantization(<br>    collection_name: str,<br>    vector_dim: int,<br>    compression: str = &quot;x16&quot;<br>):<br>    &quot;&quot;&quot;<br>    Create collection with product quantization.<br>    Compression options: x4, x8, x16, x32, x64<br>    <br>    x4  → 4x compression (best quality)<br>    x16 → 16x compression (balanced)<br>    x64 → 64x compression (most compact)<br>    &quot;&quot;&quot;<br>    compression_map = {<br>        &quot;x4&quot;: CompressionRatio.X4,<br>        &quot;x8&quot;: CompressionRatio.X8,<br>        &quot;x16&quot;: CompressionRatio.X16,<br>        &quot;x32&quot;: CompressionRatio.X32,<br>        &quot;x64&quot;: CompressionRatio.X64,<br>    }<br>    <br>    qdrant.create_collection(<br>        collection_name=collection_name,<br>        vectors_config=VectorParams(<br>            size=vector_dim,<br>            distance=Distance.COSINE,<br>        ),<br>        quantization_config=ProductQuantizationConfig(<br>            compression=compression_map.get(compression, CompressionRatio.X16),<br>            always_ram=True,<br>        )<br>    )<br>    print(f&quot;✅ Created &#39;{collection_name}&#39; with product quantization ({compression})&quot;)</pre><h3>Updating Quantization on Existing Collections</h3><pre>def update_quantization(collection_name: str, new_config):<br>    &quot;&quot;&quot;<br>    Update quantization config on an existing collection.<br>    Qdrant will re-quantize existing vectors automatically.<br>    &quot;&quot;&quot;<br>    qdrant.update_collection(<br>        collection_name=collection_name,<br>        quantization_config=new_config<br>    )<br>    <br>    # Wait for optimization to complete<br>    while True:<br>        info = qdrant.get_collection(collection_name)<br>        if info.status.value == &quot;green&quot;:<br>            print(f&quot;✅ Quantization updated and optimized for &#39;{collection_name}&#39;&quot;)<br>            break<br>        print(&quot;⏳ Optimizing...&quot;)<br>        time.sleep(2)<br><br># Upgrade from scalar to binary for a large collection<br>update_quantization(<br>    COLLECTION_NAME,<br>    BinaryQuantizationConfig(always_ram=True)<br>)</pre><h3>Choosing the Right Quantization Strategy</h3><pre>Use SCALAR (INT8) when:<br>✅ You want production-default quality with 4x memory savings<br>✅ Your dataset is &lt;100M vectors<br>✅ You need predictable, stable recall (~97-99%)<br>✅ Using pplx-embed with native INT8 output<br><br>Use BINARY when:<br>✅ Dataset is 100M+ vectors (the 32x compression becomes essential)<br>✅ Speed is more important than precision for initial retrieval<br>✅ You&#39;ll always rescore with full-precision vectors<br>✅ Using pplx-embed (trained for binary quantization)<br><br>Use PRODUCT QUANTIZATION when:<br>✅ You need very high compression (&gt;16x) with better quality than binary<br>✅ Running on severely memory-constrained hardware<br>✅ Willing to accept higher indexing time for better compression</pre><h3>Part 5: What’s New in Qdrant v1.17</h3><p>Released February 20, 2026, Qdrant 1.17 is a major update focused on production reliability under high write loads and search latency improvements.</p><h3>1. Relevance Feedback Query (New Feature)</h3><p>One of the headline features in v1.17 is <strong>Relevance Feedback</strong> — the ability to improve search quality by incorporating user feedback signals about result relevance.</p><pre># Relevance Feedback: Improve results based on known relevant/irrelevant examples<br>results = qdrant.search(<br>    collection_name=COLLECTION_NAME,<br>    query_vector=query_embedding,<br>    limit=10,<br>    with_vectors=True  # Required for relevance feedback<br>)<br><br># After showing results to user and collecting feedback:<br>positive_ids = [results[0].id, results[1].id]   # User marked as relevant<br>negative_ids = [results[4].id, results[7].id]    # User marked as irrelevant<br><br># Use recommend API for relevance feedback<br>refined_results = qdrant.recommend(<br>    collection_name=COLLECTION_NAME,<br>    positive=positive_ids,<br>    negative=negative_ids,<br>    limit=10,<br>    with_payload=True<br>)</pre><h3>2. Search Latency Improvements</h3><p><strong>Problem:</strong> Under high write loads, Qdrant clusters accumulate large unoptimized segments, causing slow query response times and unpredictable tail latency.</p><p><strong>v1.17 Solution: Two-pronged approach</strong></p><h4>2a. Avoiding Large Unoptimized Segments</h4><pre># qdrant_config.yaml — v1.17 new configuration<br>optimizer:<br>  # Maximum size of segments to avoid during heavy write loads<br>  # Prevents the accumulation of large unoptimized segments<br>  max_segment_size: 200000  # points per segment<br>  <br>  # Flush interval for WAL (Write-Ahead Log)<br>  flush_interval_sec: 5</pre><p>This ensures that during heavy ingestion (e.g., bulk document indexing), queries continue hitting already-optimized segments rather than large unoptimized ones.</p><h4>2b. Delayed Fan-Out for Tail Latency</h4><p>Delayed fan-out is a new replication strategy that eliminates P99/P999 tail latency spikes in distributed deployments:</p><pre># v1.17: Configure delayed fan-out per search request<br>results = qdrant.search(<br>    collection_name=COLLECTION_NAME,<br>    query_vector=query_embedding,<br>    limit=10,<br>    search_params=SearchParams(<br>        # New in v1.17: If first replica doesn&#39;t respond within timeout_ms,<br>        # send request to a second replica and use whoever responds first<br>        # Eliminates tail latency from slow/overloaded replicas<br>    )<br>)</pre><p>Configure the threshold in config.yaml:</p><pre># v1.17 — Delayed fan-out configuration<br>service:<br>  # If a replica doesn&#39;t respond within this threshold, <br>  # a second replica receives the read request<br>  read_rate_limit: null<br>  <br>cluster:<br>  # Delayed fan-out latency threshold (milliseconds)<br>  delayed_fanout_ms: 50   # Trigger fan-out if first replica &gt; 50ms</pre><h3>3. Weighted Reciprocal Rank Fusion (RRF)</h3><p>Standard RRF assigns equal weights to all ranked lists. v1.17 introduces <strong>Weighted RRF</strong>, allowing you to assign importance weights to individual queries when combining multiple ranking signals:</p><pre>from qdrant_client.models import Prefetch, FusionQuery, Fusion<br><br># v1.17: Weighted RRF — combine dense and sparse search with custom weights<br>results = qdrant.query_points(<br>    collection_name=COLLECTION_NAME,<br>    prefetch=[<br>        Prefetch(<br>            query=dense_embedding,<br>            using=&quot;dense&quot;,<br>            limit=50,<br>            # NEW in v1.17: weight parameter for RRF<br>        ),<br>        Prefetch(<br>            query=sparse_vector,<br>            using=&quot;sparse&quot;,<br>            limit=50,<br>        ),<br>    ],<br>    query=FusionQuery(<br>        fusion=Fusion.RRF,<br>        # Weighted RRF: dense retrieval contributes more than sparse<br>        # weights=[1.5, 0.5]  — dense gets 1.5x weight vs sparse 0.5x<br>    ),<br>    limit=10<br>)</pre><h3>4. Upsert with Update Modes (Insert-Only / Update-Only)</h3><p>v1.17 adds granular control over upsert operations:</p><pre>from qdrant_client.models import WriteOrdering<br><br># INSERT ONLY: Fail if point ID already exists (strict deduplication)<br>qdrant.upsert(<br>    collection_name=COLLECTION_NAME,<br>    points=points,<br>    wait=True,<br>    ordering=WriteOrdering.STRONG,<br>    # New in v1.17: update_mode parameter<br>    # &quot;insert&quot; → Only insert new points, skip existing<br>    # &quot;update&quot; → Only update existing points, skip new<br>    # None    → Default upsert (insert or update)<br>)</pre><h3>5. Cluster-Wide Telemetry API</h3><p>New in v1.17: A dedicated telemetry endpoint for cluster-wide observability:</p><pre>import requests<br><br># New v1.17 cluster telemetry endpoint<br>telemetry = requests.get(<br>    &quot;http://localhost:6333/telemetry&quot;,<br>    headers={&quot;api-key&quot;: &quot;your-api-key&quot;}  # If auth enabled<br>).json()<br><br>print(f&quot;Collections: {telemetry[&#39;collections&#39;][&#39;number_of_collections&#39;]}&quot;)<br>print(f&quot;Total vectors: {telemetry[&#39;collections&#39;][&#39;vectors&#39;]}&quot;)<br>print(f&quot;Memory usage: {telemetry[&#39;memory&#39;][&#39;rss_bytes&#39;] / 1e9:.2f} GB&quot;)</pre><h3>6. Segment Optimization Monitoring</h3><p>v1.17 exposes segment optimization status for diagnosing performance issues:</p><pre># Monitor segment optimization state<br>collection_info = qdrant.get_collection(COLLECTION_NAME)<br><br>for shard_id, shard_info in enumerate(collection_info.payload_schema or {}):<br>    print(f&quot;Shard {shard_id}: {shard_info}&quot;)<br><br># Detailed collection telemetry<br>telemetry = qdrant.get_collection(COLLECTION_NAME)<br>print(f&quot;Status: {telemetry.status}&quot;)<br>print(f&quot;Optimizer status: {telemetry.optimizer_status}&quot;)<br>print(f&quot;Segments: {telemetry.segments_count}&quot;)<br>print(f&quot;Points: {telemetry.points_count}&quot;)</pre><h3>7. Audit Logging</h3><p>v1.17 introduces audit logging for compliance and security:</p><pre># qdrant_config.yaml<br>security:<br>  # Enable audit logging for all authenticated API operations<br>  # Logs: operation type, user, timestamp, collection, result<br>  audit_log:<br>    enabled: true<br>    path: /qdrant/logs/audit.log<br>    format: json   # &quot;json&quot; or &quot;text&quot;</pre><h3>8. Improved WAL for Replica Recovery</h3><p>When a replica goes offline, v1.17 automatically increases the WAL (Write-Ahead Log) size to retain more operations, enabling faster catch-up when the replica comes back:</p><pre># v1.17: WAL auto-expansion for replica recovery<br>storage:<br>  wal:<br>    # Base WAL capacity<br>    wal_capacity_mb: 32<br>    <br>    # v1.17: Multiplier when remote replicas are unavailable<br>    # WAL expands to retain operations for offline replicas<br>    # (configured automatically — no manual setting needed)</pre><h3>9. Resharding via Web UI</h3><p>The Qdrant Cloud Web UI now supports <strong>resharding collections</strong> — adjusting the number of shards on a live collection without downtime. Previously this required manual API calls and careful coordination.</p><h3>10. External Provider API Keys in Request Headers</h3><pre># v1.17: Pass external inference API keys per-request (not just server config)<br># Useful for multi-tenant deployments where each user has their own API key<br>results = qdrant.search(<br>    collection_name=COLLECTION_NAME,<br>    query_vector=query_embedding,<br>    limit=10,<br>    # New: external_inference_key passed in header for per-request API key routing<br>)</pre><h3>Part 6: Production Architecture — End-to-End</h3><p>Here’s a complete production-ready pattern combining everything:</p><pre>class PPLXEmbedQdrantRetriever:<br>    &quot;&quot;&quot;<br>    Production semantic search system using pplx-embed + Qdrant v1.17.<br>    <br>    Features:<br>    - pplx-embed-v1 or pplx-embed-context-v1 embeddings<br>    - INT8 scalar quantization with FP32 rescoring<br>    - Payload filtering for metadata-based narrowing<br>    - Async-ready batch embedding<br>    &quot;&quot;&quot;<br>    <br>    def __init__(<br>        self,<br>        collection_name: str = &quot;production_kb&quot;,<br>        embedding_model: str = &quot;pplx-embed-v1-4b&quot;,<br>        vector_dim: int = 2560,<br>        qdrant_host: str = &quot;localhost&quot;,<br>        qdrant_port: int = 6333,<br>        pplx_api_key: str = None<br>    ):<br>        self.collection_name = collection_name<br>        self.embedding_model = embedding_model<br>        self.vector_dim = vector_dim<br>        <br>        self.qdrant = QdrantClient(host=qdrant_host, port=qdrant_port)<br>        self.pplx = OpenAI(<br>            api_key=pplx_api_key or os.environ[&quot;PERPLEXITY_API_KEY&quot;],<br>            base_url=&quot;https://api.perplexity.ai&quot;<br>        )<br>        <br>        self._ensure_collection()<br>    <br>    def _ensure_collection(self):<br>        &quot;&quot;&quot;Create collection with optimal settings if it doesn&#39;t exist.&quot;&quot;&quot;<br>        existing = [c.name for c in self.qdrant.get_collections().collections]<br>        if self.collection_name not in existing:<br>            self.qdrant.create_collection(<br>                collection_name=self.collection_name,<br>                vectors_config=VectorParams(<br>                    size=self.vector_dim,<br>                    distance=Distance.COSINE,<br>                ),<br>                quantization_config=ScalarQuantizationConfig(<br>                    type=ScalarType.INT8,<br>                    quantile=0.99,<br>                    always_ram=True,<br>                ),<br>                hnsw_config={&quot;m&quot;: 16, &quot;ef_construct&quot;: 100, &quot;on_disk&quot;: False}<br>            )<br>    <br>    def embed(self, texts: List[str]) -&gt; List[List[float]]:<br>        &quot;&quot;&quot;Generate embeddings via pplx-embed API.&quot;&quot;&quot;<br>        response = self.pplx.embeddings.create(<br>            model=self.embedding_model,<br>            input=texts<br>        )<br>        return [item.embedding for item in response.data]<br>    <br>    def add_documents(self, documents: List[Dict], batch_size: int = 100):<br>        &quot;&quot;&quot;Index documents with automatic batching.&quot;&quot;&quot;<br>        for i in range(0, len(documents), batch_size):<br>            batch = documents[i:i + batch_size]<br>            embeddings = self.embed([d[&quot;text&quot;] for d in batch])<br>            <br>            self.qdrant.upsert(<br>                collection_name=self.collection_name,<br>                points=[<br>                    PointStruct(<br>                        id=d.get(&quot;id&quot;, str(uuid.uuid4())),<br>                        vector=emb,<br>                        payload={&quot;text&quot;: d[&quot;text&quot;], **d.get(&quot;metadata&quot;, {})}<br>                    )<br>                    for d, emb in zip(batch, embeddings)<br>                ],<br>                wait=True<br>            )<br>    <br>    def search(<br>        self,<br>        query: str,<br>        top_k: int = 5,<br>        filters: Optional[Dict] = None,<br>        rescore: bool = True<br>    ) -&gt; List[Dict]:<br>        &quot;&quot;&quot;<br>        Search with quantization-aware parameters.<br>        Uses INT8 for speed + FP32 rescoring for quality.<br>        &quot;&quot;&quot;<br>        query_vector = self.embed([query])[0]<br>        <br>        query_filter = None<br>        if filters:<br>            query_filter = Filter(<br>                must=[<br>                    FieldCondition(key=k, match=MatchValue(value=v))<br>                    for k, v in filters.items()<br>                ]<br>            )<br>        <br>        results = self.qdrant.search(<br>            collection_name=self.collection_name,<br>            query_vector=query_vector,<br>            query_filter=query_filter,<br>            limit=top_k,<br>            search_params=SearchParams(<br>                quantization=QuantizationSearchParams(<br>                    ignore=False,<br>                    rescore=rescore,<br>                    oversampling=2.0<br>                ),<br>                hnsw_ef=128<br>            ),<br>            with_payload=True<br>        )<br>        <br>        return [<br>            {<br>                &quot;rank&quot;: i + 1,<br>                &quot;score&quot;: round(r.score, 4),<br>                &quot;text&quot;: r.payload.get(&quot;text&quot;, &quot;&quot;),<br>                &quot;metadata&quot;: {k: v for k, v in r.payload.items() if k != &quot;text&quot;}<br>            }<br>            for i, r in enumerate(results)<br>        ]<br><br><br># Usage<br>retriever = PPLXEmbedQdrantRetriever(<br>    collection_name=&quot;production_kb&quot;,<br>    embedding_model=&quot;pplx-embed-v1-4b&quot;,<br>    vector_dim=2560<br>)<br><br>retriever.add_documents(sample_documents)<br><br>results = retriever.search(<br>    &quot;What are the benefits of semantic search?&quot;,<br>    top_k=3,<br>    filters={&quot;category&quot;: &quot;search&quot;}<br>)<br><br>for r in results:<br>    print(f&quot;#{r[&#39;rank&#39;]} [{r[&#39;score&#39;]}] {r[&#39;text&#39;][:80]}...&quot;)</pre><h3>Summary: Why This Stack Works</h3><p>The combination of pplx-embed + Qdrant v1.17 represents a mature, production-grade semantic search stack:</p><p>LayerTechnologyWhy It Works<strong>Embedding</strong>pplx-embed-v1Bidirectional attention, native INT8, SOTA benchmarks<strong>Context-aware RAG</strong>pplx-embed-context-v1Document-level context in chunk embeddings<strong>Vector Storage</strong>Qdrant v1.17Rust performance, rich quantization, production reliability<strong>Memory Efficiency</strong>INT8 Scalar Quantization4x compression, ~1–3% recall loss, FP32 rescoring<strong>Extreme Scale</strong>Binary Quantization32x compression for billion-scale datasets<strong>Latency</strong>Delayed Fan-Out (v1.17)Eliminates tail latency in distributed deployments<strong>Relevance Tuning</strong>Relevance Feedback (v1.17)User feedback improves results over time<strong>Hybrid Search</strong>Weighted RRF (v1.17)Combine dense + sparse with custom weights</p><p>The key insight from Perplexity’s release: <strong>quantization is not a compromise — it’s a design choice</strong>. When your embedding model is trained from the ground up to produce high-quality INT8 and binary representations, quantization becomes a feature, not a trade-off.</p><p><em>That’s Wrap!! Guys…. Have questions? Drop a comment below.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cea9626b8c7c" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/pplx-embed-qdrant-building-production-grade-semantic-search-with-quantization-cea9626b8c7c">pplx-embed + Qdrant: Building Production-Grade Semantic Search with Quantization</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>