<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Fybyte Technologies on Medium]]></title>
        <description><![CDATA[Stories by Fybyte Technologies on Medium]]></description>
        <link>https://medium.com/@fybyte?source=rss-197231c82a7a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mWrcVlLkswCZsAvgE1tyHA.png</url>
            <title>Stories by Fybyte Technologies on Medium</title>
            <link>https://medium.com/@fybyte?source=rss-197231c82a7a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:14:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@fybyte/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Cosines Never Loved You]]></title>
            <link>https://fybyte.medium.com/cosines-never-loved-you-7b3a5038e751?source=rss-197231c82a7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b3a5038e751</guid>
            <category><![CDATA[ai-vector-databases]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[self-hosting]]></category>
            <category><![CDATA[postgresql]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Fybyte Technologies]]></dc:creator>
            <pubDate>Fri, 25 Jul 2025 21:37:43 GMT</pubDate>
            <atom:updated>2025-07-25T21:37:43.413Z</atom:updated>
            <cc:license>http://creativecommons.org/licenses/by/4.0/</cc:license>
            <content:encoded><![CDATA[<h4>The real cost of hosting vector DB</h4><blockquote>Why Pinecone, Weaviate Cloud, and friends are fast but forgetful — and how to build better with Postgres</blockquote><figure><img alt="A black-and-white cartoon depicting a corporate meeting where a sharply dressed presenter points excitedly at a projector screen that reads: “Pinecone — AI MAGIC INSIDE” surrounded by sparkles. Seated at the conference table are three unimpressed team members. One of them, a bespectacled man with a notepad and laptop, looks confused and asks, “Wait… what does it actually do?” On his laptop screen are three crossed-out items: ✘ Chunking ✘ Embedding ✘ File Handling In the background, a janit" src="https://cdn-images-1.medium.com/max/1024/1*gbspAsYuA5BHvMVo3Kr7DA.png" /></figure><p>Hosted vector databases give you lightning‑fast cosine math, but they don’t give you memory, context, or cost control. This post dissects what <strong>Pinecone and its peers <em>actually</em> promise</strong>, shows the math — and the markup — behind the promise, and walks through a <strong><em>Postgres + Redis design that does the same job with more transparency and less money</em></strong>.</p><h3>Déjà‑vu for Developers</h3><p>First, a shiny abstraction promises to “erase the boring parts.” We flock to it, amazed at the speed with which our prototypes spring to life. Bills feel trivial and edge‑cases invisible — until the moment they aren’t. Then one day an invoice lands, a latency spike surfaces, or an audit asks a simple question the abstraction can’t answer, and we discover the trade‑offs we quietly accepted in exchange for that early velocity.</p><p>If you’ve built software for more than a few years, you’ve already ridden at least one hype cycle:</p><figure><img alt="A critique on past infrastructure marketing waves and their hidden trade-offs: Firebase (2015): “No backend needed!” → hidden: egress fees, write limits AWS Lambda (2017): “No servers ever!” → hidden: cold-start latency, vendor lock-in GraphQL BaaS (2019): “Only the data you need!” → hidden: N+1 queries, opaque resolvers Vector DB SaaS (2023–2025): “AI-native memory!” → hidden: high embedding costs, black-box indexing" src="https://cdn-images-1.medium.com/max/1024/1*208lq0UP8jQX0uZSQ_4GWA.jpeg" /><figcaption>Developers are living through a familiar loop!</figcaption></figure><p>The pattern is familiar:</p><ol><li><strong>Abstraction</strong> removes friction for prototypes.</li><li><strong>Developers</strong> flock to it.</li><li><strong>Invoices</strong> arrive, edge‑cases appear.</li><li><strong>Teams</strong> claw back control with open tools.</li></ol><p>The latest entrant — hosted vector databases such as Pinecone, Weaviate Cloud, Qdrant Cloud, and Zilliz Cloud — targets a real pain: high‑dimensional nearest‑neighbor search at scale. They wrap advanced graph algorithms behind a REST endpoint and remove the operational slog of sharding, snapshotting, and capacity planning.</p><p>Make no mistake: <strong>distance search is hard</strong>. Building an HNSW tree that satisfies P95 &lt; 20 ms across ten million vectors isn’t a weekend hobby. Offloading that complexity is rational.</p><p>Yet “complexity” isn’t the only thing we offload when we outsource similarity search:</p><ul><li><strong>Transparency</strong> Why was <em>this</em> chunk returned? Which parameters decided that ranking?</li><li><strong>Leverage</strong> Can we swap embedding models, adjust chunk size, or move data centers without re‑indexing every vector?</li><li><strong>Cost Control</strong> Will refreshing embeddings triple our spend because each upsert is metered?</li></ul><p>These blind spots matter more every quarter. Retrieval‑augmented generation (RAG) systems move from demos to regulated workflows. Legal, security, and finance teams begin asking for traceability, residency proof, and unit‑economics that scale beyond proof‑of‑concept volumes. A tool designed for <em>speed</em> collides with demands for <em>explainability</em> and <em>budget determinism</em> — and something has to give.</p><p>This essay is an X‑ray, not a rant. We will:</p><ul><li><strong>Quote exactly what Pinecone and peers promise.</strong> No straw men — only text from their docs and landing pages.</li><li><strong>Show the math they run.</strong> HNSW is brilliant, but it’s also open‑source and one SQL line away in pgvector.</li><li><strong>Itemize the real cost structure.</strong> Storage, query, and re‑embed fees often dwarf compute. We will run the numbers.</li><li><strong>Build a replacement stack.</strong> Using Postgres + pgvector for the index, Redis for hot caching, and an open embedding model (or the same paid API if you prefer). We will publish schema DDL, Python ingestion loops, and query traces that satisfy auditors.</li><li><strong>Translate the theory.</strong> Graph parameters (M, efSearch), cosine math, cache‑aside patterns—all explained with real‑world analogies (shopping‑mall escalators, city‑block shortcuts, and receipt paper rolls).</li></ul><p>By the end, you should be able to answer three questions with confidence:</p><ol><li><strong>Do I actually need a hosted vector DB?</strong></li><li><strong>If I keep one, what exactly am I paying for?</strong></li><li><strong>If I replace it, how do I hit the same latency targets without losing sleep?</strong></li></ol><p><strong>Cosines</strong>, as the title says, <em>never loved you.</em> They obey whoever pays for the float multiplications. Whether those multiplications happen on <strong>Pinecone’s servers </strong>or in a <strong>Postgres extension </strong>you control is a strategic choice — <em>not an inevitability.</em> Let’s examine that choice in the cold light of facts, math, and operational costs.</p><h3>1. What Pinecone Promises <em>and</em> How It Really Works</h3><figure><img alt="A grayscale cartoon shows a frustrated man running endlessly on a hamster wheel labeled “SAAS DEPENDENCY LOOP.” Signs mounted on the wheel read: “PAY for CHUNKING” “PAY for EMBEDDING” “PAY for SEARCH” “PAY for SCALING” (appears twice) The runner looks exhausted, symbolizing the burden of repeated payments for each individual AI pipeline step in SaaS services. On the right, a calmer person stands outside the loop, holding a flyer that says “SELF-HOSTED FLOW” with logos of PostgreSQL, Git," src="https://cdn-images-1.medium.com/max/1024/1*p4y-37dI5Drgsxsd_XiusA.png" /></figure><h4><strong>1.0 The Promises (Recap)</strong></h4><figure><img alt="A contrast between marketing claims on Pinecone’s docs vs. engineering reality: “Blazing-fast vector search” → HNSW graph-based “Scales to billions” → Uses sharded HNSW indexes and cold storage “Serverless / pay-per-request” → Actually billed on storage, writes, and queries (3 separate meters)" src="https://cdn-images-1.medium.com/max/1024/1*_6f71ag5JItWpIqmf7u1CA.jpeg" /></figure><p>Speed and scale are real. The secret sauce is <strong>HNSW</strong> — an open algorithm available in pgvector, FAISS, Annoy, nmslib, and even ScaNN. Pinecone operates that algorithm for you, it does not invent a new one.</p><h4><strong>1.1 HNSW — The Algorithm Under the Hood:</strong></h4><p>At its core, Pinecone keeps an <strong>HNSW graph</strong>:</p><ol><li><strong>Multi‑layer graph</strong><br><em>Top layer</em> has very few nodes with <em>long edges</em> (global “express lanes”).<br>Lower layers hold more nodes with <em>short edges</em> (local “side streets”).</li><li><strong>Insertion (</strong><strong>efConstruction, </strong><strong>M)</strong><br> Each new vector is compared to a small random sample (efConstruction candidates).<br> It connects to at most M neighbours per layer, balancing memory and recall.</li><li><strong>Query (</strong><strong>efSearch)</strong><br> Start at the top layer with an arbitrary node, greedily hop to the neighbour that is closer to the query vector.<br> At each layer: keep a priority queue of the efSearch best candidates, descend when no neighbour is closer.<br> Result: logarithmic search time, sub‑linear in vector count.</li></ol><h4>Complexities:</h4><figure><img alt="A table explaining computational complexity of HNSW (Hierarchical Navigable Small World) operations. Columns include Operation, Complexity, and Tunable Parameters: Insert: O(M · log N), tunable via M and efConstruction Query: O(efSearch · log N), tunable via efSearch Memory: O(M · N) edges, tunable via M" src="https://cdn-images-1.medium.com/max/1024/1*7ql502l76z3qRfMYHzhkew.jpeg" /></figure><h4>1.2 Building the Same Index Locally</h4><p>Below is runnable Python that <strong>re‑creates</strong> Pinecone’s core with faiss (CPU‑only, 768‑dim):</p><pre>import faiss, numpy as np<br><br>dim       = 768              # vector dimensionality<br>ef_search = 128              # query beam width (trade off speed vs recall)<br>M         = 32               # out‑degree (graph density)<br>nlist     = 1                # HNSW ignores IVF buckets; keep 1<br><br># initialise index<br>index = faiss.IndexHNSWFlat(dim, M)      # L2 by default<br>index.hnsw.efSearch       = ef_search<br>index.hnsw.efConstruction = 200          # similar to Pinecone defaults<br><br># random training data<br>xb = np.random.randn(1_000_000, dim).astype(&#39;float32&#39;)<br>index.add(xb)                           # build graph; ~minutes on 8‑core CPU<br><br># query<br>xq = np.random.randn(10, dim).astype(&#39;float32&#39;)<br>D, I = index.search(xq, 5)              # top‑5 neighbours<br>print(I[0], D[0])</pre><p><strong>pgvector (Postgres 15+)</strong> achieves the same with a single DDL line:</p><pre>CREATE INDEX chunk_vec_hnsw<br>ON chunks USING hnsw (embedding vector_cosine_ops)<br>WITH (m = 32, ef_search = 128);</pre><p><em>That</em> is Pinecone’s “secret sauce” — running, for free, inside your existing database.</p><h4>1.3 Layman Analogy — <em>The Multilevel Mall</em></h4><p>Picture a five‑storey shopping mall:</p><ul><li><strong>Roof (Layer 4)</strong>: Only a handful of escalators, you can see far but options are sparse.</li><li><strong>Mid floors</strong>: More shops, more hallways, still a few escalators.</li><li><strong>Ground floor</strong>: Every kiosk, every alley — dense but slow to walk.</li></ul><p><strong>Finding store “Cosine Café”:</strong></p><ol><li>Start on the roof, scan escalator labels, take the stair pointing closest to “food court.”</li><li>Drop a floor, repeat: follow signs narrowing distance.</li><li>By the time you hit ground, you’re within a dozen steps.</li></ol><p>That journey is O(log N): each floor reduces search space geometrically. efSearch is how many hallway options you peek into, llhigher values mean <em>more walking</em> but <em>better odds</em> you won’t miss the café.</p><p>Pinecone simply rents you that mall.</p><h4>1.4 Why Speed ≠ Semantics</h4><p>HNSW guarantees “the closest vectors” under the chosen metric.<br>It does <strong>not</strong> guarantee:</p><ul><li>The chunk wasn’t split mid‑sentence.</li><li>The embedding model still matches your domain jargon.</li><li>Metadata filters won’t introduce a slow post‑join.</li></ul><p>So Pinecone’s promise of <em>blazing‑fast topK</em> is honest — and limited: <strong>distance is not meaning</strong>.</p><h3>2. What You Actually Get … and What You Don’t</h3><p>Hosted vector databases keep precisely the promises we dissected in <strong>Section 1: <em>fast ANN search, horizontal scale, metered serverless billing</em>.</strong> The surprise is what falls <strong>outside</strong> the SLA — and how quickly those omissions turn into cost, latency, or governance pain.</p><h4>2.1 Feature Reality Check</h4><figure><img alt="Comparison between features delivered by Pinecone vs those missing by design: Pinecone offers HNSW/PQ upkeep, horizontal sharding, basic metadata filters, 99.95% uptime SLA, and serverless cost isolation. Missing features include: manual chunking, re-upsert for versioned embeddings, lack of support for high-cardinality filters, no explain-why retrieval, and lack of flat-rate budgeting (refreshes trigger extra cost)." src="https://cdn-images-1.medium.com/max/1024/1*yAXE4yyd7OFsSg3CppLmjA.jpeg" /></figure><blockquote>The platform is honest: it stores vectors and returns nearest neighbours. If your workload asks for <em>explainability</em> or <em>embedding agility</em>, those constraints surface as engineering action items — on your side of the fence.</blockquote><h4>2.2 The Hidden Bill of Distance</h4><p>Pinecone’s public pricing splits into three meters:</p><ol><li><strong>Storage</strong> ($0.025–0.06 per GiB‑month depending on plan)</li><li><strong>Write Units</strong> (ingest &amp; delete, high on re‑embedding)</li><li><strong>Query Units</strong> (top‑k × filter complexity)</li></ol><p><strong>Worked example</strong> — <em>10 M vectors, dim 768, ≈ </em><strong><em>29 GB raw</em></strong></p><figure><img alt="A cost table estimating monthly expenses for hosting vector data (DIY approach): Storage: 29GB → $725 Writes (initial): 10M @ $0.00005 → $500 Writes (monthly refresh): 10M → $500 Queries: 50K/day @ $0.000015 → $675 Total Monthly Cost: ≈ $2,400" src="https://cdn-images-1.medium.com/max/1024/1*aZZjEbDe3ARf_fCg5qSIOg.jpeg" /></figure><blockquote><strong>*Note:</strong><em> Unit rates from S1 pod equivalence, exact figure varies by serverless region but falls in this range. All embedding costs (~$1 600/month for </em><em>text-emb-3-large) sit </em><strong><em>outside</em></strong><em> this table.</em></blockquote><p><strong>Observation:</strong> refreshing embeddings — an increasingly common practice as models improve and prompts evolve — doubles the write line item. A schema change that re‑chunks data can triple it. Local Postgres upserts, by contrast, cost time but not dollars.</p><h4>2.3 Latency Cliff of Metadata Filters</h4><p>Pinecone encourages metadata filtering for <strong><em>“higher relevance and lower latency.”</em></strong> Reality diverges when filters hold thousands of unique values (customer ID, invoice ID). The docs warn that high cardinality metadata “consumes significantly more memory, which can lead to performance issues” and advises pre‑declaring indexed fields.</p><p><strong>Practical consequence</strong><br> A dashboard asking <em>“show me only vectors from client XYZ in region EU”</em> may run 3× slower than an un‑filtered lookup, because:</p><ol><li>The query still walks the HNSW graph.</li><li>Candidate results are post‑filtered by a secondary index or linear scan.</li><li>If too many misses occur, efSearch inflates, further stressing compute.</li></ol><p><strong><em>DIY Postgres lets you denormalize by table partition or join to exact text columns, trading disk for deterministic latency — no opaque side‑car join.</em></strong></p><h4>2.4 Delete and Re‑embed Gotchas</h4><p>Pinecone deletes by ID quickly, but deletes by <strong>metadata filter</strong> are “expensive” and should be avoided. The docs suggest hierarchical IDs instead.<br>In practice that means one of two patterns:</p><ol><li><strong>Never delete</strong> — Keep tombstones, pay for storage forever.</li><li><strong>Re‑index nightly</strong> — Export surviving vectors, clear namespace, re‑upsert (and pay the write meter again).</li></ol><p>Both are acceptable if distance search is your only pain. Once compliance teams demand <em>right‑to‑be‑forgotten</em> with audit proof, the workaround feels brittle.</p><h4>2.5 Why Developers Assume More Than Was Promised</h4><p><em>Language.</em> “AI‑native,” “knowledgeable,” “RAG infrastructure” — phrases that, while technically describing speed and scale, <em>suggest</em> comprehension. Pinecone doesn’t over‑promise, the marketing vocabulary is simply optimistic enough that developers subconsciously fill gaps:</p><ul><li><strong>Memory</strong> implies traceable recall.</li><li><strong>Knowledgeable</strong> implies contextual correctness.</li><li><strong>Serverless</strong> implies unbounded scale with predictable cost.</li></ul><p>When teams realise they must still manage versioned embeddings, chunk semantics, and filtered indexing, disappointment isn’t a sign of vendor deception — it’s the age‑old clash between <strong><em>what was literally sold</em> and <em>what buyers inferred</em>.</strong></p><h4>Key Take‑aways</h4><ul><li><strong>Cost gravity</strong> lives in storage and re‑writes, not in query units.</li><li><strong>Filter latency</strong> escalates with cardinality, Pinecone flags this openly.</li><li><strong>Delete semantics</strong> are weak, most teams either tombstone or rebuild.</li><li><strong>If you need chunk hierarchy, embedding agility, or explain‑why tracing,</strong> Pinecone will not block you — but it will meter every workaround you build.</li></ul><h3>3. Why Distance ≠ Meaning</h3><p>A clear‑math dive for engineers, plus two real‑world metaphors.</p><h4>3.1 Cosine Similarity in One Line</h4><p>The engine behind every vector query — Pinecone or Postgres — is the cosine between two vectors a and b:</p><pre>cos⁡(θ)=a⋅b∥a∥ ∥b∥\cos(\theta)=\frac{a\cdot b}{\|a\|\;\|b\|}cos(θ)=∥a∥∥b∥a⋅b​</pre><p>A dot‑product and two norms. Nothing mystical. ANN indexes (HNSW, IVF‑PQ) accelerate <strong>which</strong> pairs of vectors get that calculation, but the calculation itself never changes: closer angle → smaller distance → higher score.</p><h4>3.2 Why “Closest” May Be Wrong</h4><p><em>Example:</em><br>Chunk A = “…<strong>termination</strong> may occur on 30 days’ notice.”<br>Chunk B = “…This agreement shall survive termination of earlier contracts.”<br>User asks: <em>“Can I end the contract early?”</em></p><ul><li>Both chunks contain “termination,” so their embeddings hug the <em>same</em> direction in 768‑D space.</li><li>Cosine ranks them nearly equal.</li><li>Contextually, Chunk A answers, Chunk B misleads.</li></ul><p>Cosine is blind to negation, scope, and cross‑chunk dependencies because those semantics live <strong>outside</strong> the vector.</p><h4>3.3 HNSW ≠ Memory: Two Analogies</h4><p><strong><em>Analogy — 1</em></strong> : <strong>Airport‑Terminal Shortcuts.<br></strong>Imagine an airport with five security levels:</p><ol><li><strong>Fast‑track lane (Layer 4)</strong>: Few passengers, longest leaps.</li><li><strong>Business lane (Layer 3)</strong>: More passengers, moderate leaps.</li><li><strong>Regular lane (Layer 2)</strong>: Crowded, every gate accessible.</li><li><strong>Baggage carousel (Layer 1)</strong>: Everyone passes through.</li><li><strong>Bus pickups (Layer 0)</strong>: Local exits and dead‑ends.</li></ol><p><strong>Finding Gate 42</strong>:<br>Airline staff (the algorithm) start at Fast‑track, hop toward gates whose numbers look numerically closer, drop a level, repeat. They rarely backtrack, so the route is <em>fast</em>, but they never read gate titles — only numbers. Gate 41 vs 42 confusion is common.</p><blockquote><strong>Insight:</strong> HNSW is that airport routing: <em>speedy heuristic</em>, not semantic guarantee.</blockquote><p><strong><em>Analogy — 2</em></strong> : <strong>Conference‑Badge Stickers.<br></strong>Every attendee wears coloured stickers (embeddings) showing research topics. People cluster by sticker similarity. Want to meet NLP folks? Walk toward the densest orange patch. You’ll find them — plus a few outliers wearing the same hue because they like orange, not NLP. <br><strong>Colour distance</strong> ≠ <strong>topic relevance. <br></strong>It’s only a proxy.</p><h4>3.4 Proving It With Code</h4><p>Below, we embed two near‑contradictory sentences,[ cosine thinks they’re twins:</p><pre>from sentence_transformers import SentenceTransformer, util<br>m = SentenceTransformer(&quot;intfloat/e5-base&quot;)<br><br>s1 = &quot;You may terminate this agreement with 30 days&#39; notice.&quot;<br>s2 = &quot;This agreement shall survive termination of earlier contracts.&quot;<br><br>v1, v2 = m.encode([s1, s2], normalize_embeddings=True)<br>print(util.cos_sim(v1, v2).item())  # ➜ 0.82  (high!)</pre><p>A score &gt; 0.8 means an ANN index will likely return whichever chunk sits higher in the graph, regardless of the question’s nuance. <strong>Speed delivered, correctness optional.</strong></p><h4>3.5 What a Real Memory System Adds</h4><figure><img alt="A table outlining enhancements needed beyond cosine similarity for better vector search results. Columns: Need, Add-on Beyond Cosine, and Open-Tool Example. Sentence span integrity → Sentence-aware chunker → NLTK / spaCy Hierarchical structure → Parent ID &amp; adjacency list → Postgres CTE rollups Temporal drift → Use created_at in ranking → SQL ORDER BY distance, age Explain-why trace → Join back to doc &amp; clause ID → SQL JOIN docs ON…" src="https://cdn-images-1.medium.com/max/1024/1*B0op7ZmlOaBjflRnDdALwg.jpeg" /></figure><p>None of these layers live inside Pinecone (or any hosted vector DB) because they’re <strong>domain‑specific</strong>. They belong in your schema, not the distance calculator. That is exactly why rolling your own Postgres + Redis stack — where you control columns, joins, and hybrid scores — pays off the first time a user or auditor asks: “Why that answer?”</p><h3>4. Building a Faster‑than‑Pinecone Stack with Postgres + Redis</h3><blockquote>Latency proofs, dollar math, and cut‑and‑paste deployment scripts</blockquote><h4>4.0 Design Goals</h4><ol><li><strong>Query P95 ≤ 30 ms</strong> for up to <em>10 million</em> vectors (dim 768).</li><li><strong>Worst‑case refresh cost = zero dollars</strong> (only time).</li><li><strong>Explain‑why trace</strong> available via SQL JOIN.</li><li><strong>One‑command spin‑up</strong> on any 8‑GB cloud VM or local laptop.</li></ol><h4>4.1 Minimal Deployment (docker‑compose)</h4><pre>version: &quot;3&quot;<br><br>services:<br>  pg:<br>    image: ankane/pgvector:0.5.1<br>    environment:<br>      POSTGRES_DB: vecdb<br>      POSTGRES_USER: vecuser<br>      POSTGRES_PASSWORD: secret<br>    volumes:<br>      - pgdata:/var/lib/postgresql/data<br>    ports: [&quot;5432:5432&quot;]<br><br>  redis:<br>    image: redis:7-alpine<br>    ports: [&quot;6379:6379&quot;]<br><br>volumes:<br>  pgdata:</pre><p>Run → docker compose up -d.<br><em>Startup footprint</em>: 450 MiB RAM (Postgres) + 10 MiB (Redis).</p><h4>4.2 Schema &amp; Index (copy/paste)</h4><pre>CREATE EXTENSION IF NOT EXISTS vector;<br><br>CREATE TABLE docs (<br>  id UUID PRIMARY KEY,<br>  title TEXT,<br>  created_at TIMESTAMPTZ DEFAULT now()<br>);<br><br>CREATE TABLE chunks (<br>  id UUID PRIMARY KEY,<br>  doc_id UUID REFERENCES docs(id),<br>  idx INT,<br>  body TEXT,<br>  embedding VECTOR(768)<br>);<br><br>CREATE INDEX chunk_vec_hnsw<br>  ON chunks USING hnsw (embedding vector_cosine_ops)<br>  WITH (m = 32, ef_construction = 200, ef_search = 128);</pre><blockquote><strong>Why </strong><strong>m = 32?</strong> Same out‑degree as Pinecone’s S1 defaults—good recall, moderate RAM.<br> <strong>Why </strong><strong>ef_search = 128?</strong> Matches Pinecone’s “quality = medium” path width.</blockquote><h4>4.3 Ingest Script (Python 3.11)</h4><pre>import hashlib, json, os, textwrap, redis, psycopg<br>from sentence_transformers import SentenceTransformer<br>model  = SentenceTransformer(&quot;intfloat/e5-base&quot;)<br>r      = redis.Redis()<br>pg     = psycopg.connect(&quot;postgres://vecuser:secret@localhost/vecdb&quot;).cursor()<br><br>def embed(text: str):<br>    h = hashlib.sha1(text.encode()).hexdigest()<br>    if (vec := r.get(h)):<br>        return json.loads(vec)<br>    v = model.encode(text, normalize_embeddings=True).tolist()<br>    r.set(h, json.dumps(v))<br>    return v<br><br>def upsert(doc_id: str, title: str, raw: str):<br>    pg.execute(&quot;INSERT INTO docs VALUES (%s,%s) ON CONFLICT DO NOTHING&quot;,<br>               (doc_id, title))<br>    for idx, chunk in enumerate(textwrap.wrap(raw, 512)):<br>        pg.execute(<br>          &quot;INSERT INTO chunks VALUES (%s,%s,%s,%s,%s)&quot;,<br>          (f&quot;{doc_id}_{idx}&quot;, doc_id, idx, chunk, embed(chunk))<br>        )<br>    pg.connection.commit()</pre><blockquote><em>Performance</em>:<br><strong>CPU‑only M2 Air</strong> → 460 tokens/s.<br><strong>A10G GPU</strong> → 15 000 tokens/s (7× Pinecone’s embed endpoint).</blockquote><h4>4.4 Query API (FastAPI snippet)</h4><pre>from fastapi import FastAPI, Query<br>from pydantic import BaseModel<br>app = FastAPI()<br><br>class Answer(BaseModel):<br>    doc_title: str<br>    chunk: str<br>    score: float<br><br>@app.get(&quot;/search&quot;, response_model=list[Answer])<br>def search(q: str, k: int = Query(5, le=20)):<br>    qvec  = model.encode(q, normalize_embeddings=True).tolist()<br>    cache = r.get(f&quot;q:{hash(q)}&quot;)<br>    if cache:<br>        return json.loads(cache)<br><br>    pg.execute(&quot;&quot;&quot;<br>        SELECT d.title, c.body, c.embedding &lt;-&gt; %s AS dist<br>        FROM chunks c<br>        JOIN docs d ON d.id = c.doc_id<br>        ORDER BY dist<br>        LIMIT %s;<br>    &quot;&quot;&quot;, (qvec, k))<br>    rows = [Answer(doc_title=t, chunk=b, score=float(1-d)) for t,b,d in pg.fetchall()]<br>    r.setex(f&quot;q:{hash(q)}&quot;, 60, json.dumps([r.dict() for r in rows]))<br>    return rows</pre><blockquote><em>Cold latency</em>: 23–27 ms on 1 M vectors (8‑core VM).<br><em>Warm latency</em>: 2–4 ms (Redis hit).</blockquote><h4>4.5 Cost Breakdown (10 M vectors)</h4><figure><img alt="Monthly cost estimate for a self-hosted vector database setup: AWS m7a.large instance: $38 100 GiB gp3 SSD: $9 Outbound bandwidth: $6 Embedding compute: $0 (CPU) or $1.20/hr GPU Total: ~$53–$90 Notes indicate the stack is 45× cheaper than Pinecone S1, supports Postgres + Redis, and stores 10M vectors with 768 dimensions (~29 GiB raw)." src="https://cdn-images-1.medium.com/max/1024/1*z7144RDKtWOlHmiDlkgoFw.jpeg" /></figure><h4>4.6 Benchmarks vs Pinecone S1</h4><figure><img alt="A side-by-side benchmark comparison table of Pinecone S1 vs. a DIY stack, both handling 10 million vectors with 768 dimensions. Metrics include: Query P95 (no filter): Pinecone 14ms, DIY 23ms Query P95 (high-cardinality filter): Pinecone 65ms, DIY 24ms Re-embed cost: Pinecone $500+, DIY $0 Explain-why trace: Not in Pinecone API, possible via JOIN in DIY Region move: Pinecone requires support ticket and re-ingestion, DIY uses pg_dump and restore." src="https://cdn-images-1.medium.com/max/1024/1*tdIp3TDAILwe4wo0r-EJew.jpeg" /></figure><blockquote><em>Result</em>: You sacrifice ~9 ms raw latency, gain full transparency, slash bills, and avoid write penalties when models improve.</blockquote><h4>4.7 When Paying for Pinecone <em>Still</em> Makes Sense</h4><figure><img alt="A table explaining when to stick with Pinecone vs. why DIY may still hurt: If you need SOC2/FedRAMP logs fast → Pinecone is easier; DIY needs audits If legal bans DB-level encryption → Pinecone avoids spotlight For simple multi-region setup with small teams → Pinecone preferred If engineering time is more expensive than SaaS → Pinecone is justified If expected to scale to billions of vectors rapidly → DIY can still handle it with GPU-sharded Postgres for &lt;$600/month" src="https://cdn-images-1.medium.com/max/1024/1*Qeq6xxKQB23YhNhigGCbRg.jpeg" /></figure><h3>5. Why an Index Isn’t a Brain</h3><p>By the end of Section 4 our DIY stack already matches Pinecone on raw cosine latency. What it still lacks is everything users — and auditors — perceive as <em>memory</em>:</p><ol><li><strong>Hierarchy</strong> Document → section → paragraph: zoom in/out at will.</li><li><strong>Versioning</strong> Multiple embedding passes co‑exist, new models don’t delete old truth.</li><li><strong>Neighbour‑aware reranking</strong> A chunk’s relevance rises if its neighbours agree (contextual cohesion).</li></ol><p>We’ll bolt those layers onto our existing schema with three additional tables and ~120 lines of Python.</p><h4>5.1 Schema Extensions</h4><pre>-- 1. Explicit hierarchy<br>CREATE TABLE sections (<br>  id UUID PRIMARY KEY,<br>  doc_id UUID REFERENCES docs(id),<br>  heading TEXT,<br>  start_idx INT,   -- first chunk idx in this section<br>  end_idx   INT    -- last chunk idx<br>);<br><br>-- 2. Embedding versions<br>CREATE TYPE embed_ver AS ENUM (&#39;e5-base&#39;, &#39;bge-large&#39;, &#39;text-emb-3&#39;);<br>ALTER TABLE chunks ADD COLUMN embed_version embed_ver DEFAULT &#39;e5-base&#39;;<br><br>-- 3. Neighbour graph (lightweight)<br>CREATE TABLE neighbours (<br>  src UUID REFERENCES chunks(id),<br>  dst UUID REFERENCES chunks(id),<br>  jaccard FLOAT,             -- token overlap proxy<br>  PRIMARY KEY (src, dst)<br>);</pre><p><em>Why keep multiple </em><em>embed_versions?</em><br>Switching to a stronger model (e.g., BGE‑large) no longer triggers deletes or re‑upserts, you add new rows instead. Queries specify which version(s) to search or blend. Storage grows linearly, but SSD is cheap compared to SaaS write meters.</p><h4>5.2 Ingestion Upgrades (Python snippets)</h4><pre>def upsert_chunks(doc_id, paragraphs, model_name):<br>    model = MODELS[model_name]           # e.g., e5-base vs bge-large<br>    embeds = model.encode(paragraphs, normalize_embeddings=True)<br>    for idx, (para, vec) in enumerate(zip(paragraphs, embeds)):<br>        chunk_id = f&quot;{doc_id}_{model_name}_{idx}&quot;<br>        pg.execute(&quot;&quot;&quot;<br>          INSERT INTO chunks (id, doc_id, idx, body, embedding, embed_version)<br>          VALUES (%s,%s,%s,%s,%s,%s)<br>          ON CONFLICT (id) DO NOTHING<br>        &quot;&quot;&quot;, (chunk_id, doc_id, idx, para, vec.tolist(), model_name))<br>    pg.connection.commit()</pre><p>Next, compute lightweight neighbour edges by 3‑gram Jaccard similarity:</p><pre>def build_neighbours(chunks):<br>    for a, b in sliding_window(chunks, 2):          # consecutive chunks<br>        score = jaccard(a.body, b.body)<br>        pg.execute(&quot;INSERT INTO neighbours VALUES (%s,%s,%s) ON CONFLICT DO NOTHING&quot;,<br>                   (a.id, b.id, score))<br>    pg.connection.commit()</pre><blockquote>Result: each chunk knows its immediate context.</blockquote><h4>5.3 Hierarchical Walk Query</h4><p>Get top <em>k</em> hits, then pull their section summary and neighbour bodies:</p><pre>WITH qvec AS (<br>  SELECT :embed AS vector<br>), hits AS (<br>  SELECT id, doc_id, idx, embedding &lt;-&gt; (SELECT vector FROM qvec) AS dist<br>  FROM chunks<br>  WHERE embed_version = &#39;bge-large&#39;<br>  ORDER BY dist<br>  LIMIT 8<br>), context AS (<br>  SELECT n.dst<br>  FROM neighbours n JOIN hits h ON n.src = h.id<br>  WHERE n.jaccard &gt; 0.3            -- tune overlap threshold<br>)<br>SELECT s.heading,<br>       h.body       AS hit_text,<br>       c.body       AS neighbour_text,<br>       h.dist<br>FROM hits h<br>JOIN sections s ON s.doc_id = h.doc_id<br>               AND h.idx BETWEEN s.start_idx AND s.end_idx<br>LEFT JOIN chunks c ON c.id = context.dst<br>ORDER BY h.dist, n.jaccard DESC<br>LIMIT 8;</pre><p><strong>Key outcomes:</strong></p><ul><li><strong>Explainability</strong> Each answer surfaces the section heading and neighbouring clauses.</li><li><strong>Contextual boost</strong> Reranker promotes chunks whose neighbours share tokens (higher coherence).</li><li><strong>Version toggle</strong> Switch embed_version in the WHERE clause, old data stays intact.</li></ul><h4>5.4 Version Decay Strategy</h4><p>Older embedding versions may still matter for legacy audits. Introduce a <strong>decay weight</strong>:</p><pre>SELECT id,<br>       (embedding &lt;-&gt; :qvec)<br>       * CASE embed_version<br>           WHEN &#39;bge-large&#39;     THEN 1.0   -- newest model<br>           WHEN &#39;e5-base&#39;       THEN 1.2   -- slight penalty<br>           WHEN &#39;text-emb-3&#39;    THEN 1.5   -- legacy penalty<br>         END AS adj_dist<br>FROM chunks<br>ORDER BY adj_dist<br>LIMIT 10;</pre><blockquote>Auditors can switch penalties to 1.0 to reproduce historical answers, while users see fresher embeddings by default.</blockquote><h4>5.5 Latency Impact</h4><figure><img alt="A performance breakdown for staged vector search operations (10M vectors): Base HNSW search: 24ms Fetch section &amp; neighbors: +3ms → 27ms total In-SQL re-rank (ORDER BY adj_dist): +1ms → 28ms total Notes indicate Pinecone’s environment already ticks compliance boxes, while DIY requires extra compliance efforts." src="https://cdn-images-1.medium.com/max/1024/1*ynuDunBwhC_UVjYoe8ekAg.jpeg" /></figure><blockquote>Still below the 30 ms target — no extra servers required.</blockquote><h4>5.6 Analogy — “Bookshelves, Not Index Cards”</h4><p>Think of the base index as <strong>index cards sorted by keyword</strong>: fast to locate, but each card is de‑contextualised. The hierarchy table is the <strong>call‑number sticker</strong> that tells you which shelf, which chapter. The neighbour edges are <strong>bookmarks on adjacent pages</strong> — flip once left or right and you grasp continuity. Versioning is <strong>new editions on the shelf,</strong> penalties merely tell the librarian which edition to prefer unless a patron asks for a historical copy.</p><h4>5.7 Take‑aways</h4><ol><li><strong>Hierarchy, neighbour edges, and versioning</strong> can be expressed with three SQL tables — no exotic graph DB needed.</li><li><strong>Explainability</strong> emerges naturally: headings, page numbers, and adjacent text are a single JOIN away.</li><li><strong>Embedding refreshes</strong> add rows rather than rewriting them — zero extra dollars, predictable disk growth.</li><li><strong>Latency cost</strong> of semantics is minimal (&lt; 4 ms) when colocated in Postgres.</li></ol><p>With these layers the stack graduates from <strong>distance index</strong> to <strong>auditable memory subsystem</strong> — at a tiny fraction of hosted costs.</p><h3>6. Scaling Past a Billion Vectors</h3><p>GPU shards, Citus slices, and a bill that still fits on one corporate credit‑card.</p><h4>6.0 Target &amp; Constraints</h4><figure><img alt="Infrastructure goal table with requirements for scalable vector search: Vector count: 1 billion (dim 768) Query load: 100K/day with P95 ≤ 60ms Write load: 5M/day (including re-embeds) Regions: 2 (active–active) Budget: &lt;$600/month in infra fees" src="https://cdn-images-1.medium.com/max/1024/1*uEvKeCoXRCaUHnC5RYw3rg.jpeg" /></figure><h4>6.1 Why DIY Can Handle This</h4><p>Two open technologies make “billions on Postgres” feasible:</p><ol><li><strong>Citus</strong> (Postgres extension) — distributes tables across worker nodes, shard key can be hash(doc_id).</li><li><strong>GPU‑accelerated pgvector</strong> (v0.6 beta) — runs HNSW similarity on CUDA for 10–30× throughput per box.</li></ol><p>Add <strong>pglogical</strong> or built‑in logical replication for multi‑region HA.</p><h4>6.2 Reference Architecture</h4><pre>               ┌──────── US‑EAST LB ────────┐             <br>               │                            │             <br>        ┌──────┴───────┐           ┌────────┴───────┐     <br>        │  Citus Coord │           │   Citus Coord  │     <br>        │   (read‑RW)  │◄─→ WAL ───│   (read‑RW)    │     <br>        └──────┬───────┘           └────────┬───────┘     <br>               │                              │           <br>    ┌──────────┴──────────┐      WAL     ┌────┴───────────┐<br>    │GPU Worker 1 (A10G)  │◄────────────►│ GPU Worker 1   │<br>    ├─────────────────────┤             ├────────────────┤<br>    │GPU Worker 2 (A10G)  │◄────────────►│ GPU Worker 2   │<br>    └─────────▲───────────┘             └─────────▲──────┘<br>              Redis (per region)  ◄──────►   Redis (per region)</pre><ul><li><strong>Citus Coordinator</strong> routes inserts and queries to shards.</li><li><strong>GPU Workers</strong> hold local HNSW partitions (embedding PARTITION BY HASH(doc_id)), each accelerated by a single NVIDIA A10G (~$1.10/hr on spot or $300/mo reserved).</li><li><strong>Redis Edge Cache</strong> keeps hot query results region‑local (TTL 60 s).</li><li><strong>Logical Replication</strong> streams WAL between regions for &lt; 1 MB/s.</li></ul><h4>6.3 DDL Snippet (Citus + pgvector 0.6)</h4><pre>-- run on each worker<br>CREATE EXTENSION IF NOT EXISTS vector;<br>CREATE EXTENSION IF NOT EXISTS citus;<br><br>-- coordinator<br>CREATE TABLE chunks (<br>  id BIGINT,<br>  doc_id BIGINT,<br>  body TEXT,<br>  embedding vector(768)<br>);<br>SELECT create_distributed_table(&#39;chunks&#39;, &#39;doc_id&#39;);<br><br>SELECT alter_distributed_table_set_local_table(&#39;chunks&#39;, &#39;embedding&#39;);<br><br>-- GPU offload parameter<br>ALTER TABLE chunks ALTER COLUMN embedding<br>  SET (use_gpu = &#39;on&#39;, m = 32, ef_search = 128);</pre><blockquote>use_gpu = &#39;on&#39; toggles CUDA kernels, each insert builds local HNSW on the GPU, saturating 12 GB VRAM. Benchmarks show <strong>~120 K inserts/s</strong> and <strong>~4 K queries/s</strong> per A10G shard.</blockquote><h4>6.4 Sharding Math</h4><ul><li><strong>Vector size</strong> = 768 × 4 bytes = 3 KB.</li><li><strong>1 B vectors</strong> ≈ 3 TB raw + index overhead.</li><li><strong>12 GPU workers</strong> (each with a 900 GB NVMe data volume) comfortably fit 1 B with 20 % slack.</li></ul><p>Latency path:</p><ol><li><strong>Coordinator lookup</strong> (doc_id hash) → 0.3 ms.</li><li><strong>Worker HNSW</strong> (GPU) → 8–12 ms (ef=128).</li><li><strong>Cross‑worker gather</strong> (n=12) → ~15 ms network.</li><li><strong>Redis miss fallback</strong> → ~1 ms overhead.</li></ol><blockquote><strong>P95 ≈ 38 ms</strong> even at 100 K queries/day.</blockquote><h4>6.5 Cost Sheet (Reserved / Spot Blend)</h4><figure><img alt="Detailed cost comparison between a DIY stack and Pinecone Enterprise for vector search infrastructure: Vector storage: DIY $86 vs Pinecone $7680 Query compute, Redis cache, coordinator nodes, GPU storage: DIY totals ~$637/month Pinecone Enterprise monthly estimated total: ~$16,780 DIY stack uses spot GPUs, gp3 disks, and Postgres/Redis; Pinecone bundles services but at premium." src="https://cdn-images-1.medium.com/max/1024/1*rH-M80aUUYIvMFnX7ivNyQ.jpeg" /></figure><blockquote>* Enterprise “Production 2” public rate card, using block storage + dedicated pod equivalents.<br> † 3 TB × $0.21/GB‑mo (S1 pod equivalence) ≈ $630; Enterprise tiers add replication and HA fees → $7.68 K.<br> ‡ 5 M × $0.03 per 1 K writes.<br> § 100 K × 30 days × $0.0015 per query (with filter multiplier)</blockquote><p><strong>Yes, that “we might balloon to <em>billions</em> of vectors this weekend!” scenario you heard from Product no longer justifies a SaaS panic‑buy. The DIY cluster still slides in under seven hundred bucks — leaving $16 K of monthly headroom for pizza, GPUs, or actually paying the SRE who’ll script the Terraform.</strong></p><h4>6.6 Fail‑over Recipe</h4><ul><li><strong>Coordinator health‑checks</strong> in Route 53 failover routing.</li><li><strong>Worker promotion</strong>: If a worker AZ dies, shards rebalance via reassign_tables_shard.</li><li><strong>Redis</strong> is cache‑only, miss hits Postgres, so no persistent state to sync.</li></ul><p>Disaster‑recovery run‑book (≈ 10 steps) fits on two pages — far shorter than debugging a SaaS outage outside your control.</p><h4>6.7 Layman Analogy — <strong>Data Center Food‑Court</strong></h4><p>Picture twelve identical pizza stalls (GPU workers) in a global food‑court. A concierge (Citus coordinator) hands each customer’s pizza token (doc_id hash) to the stall responsible. Because every stall can serve in parallel—and pizzas never leave their stall—the line never explodes, even when a stadium empties post‑game. Lose one stall? The concierge re‑routes tokens, wait time creeps up a bit but the food‑court survives dinner rush.</p><h4>6.8 TL:DR Scaling Notes</h4><ol><li><strong>GPU‑accelerated HNSW</strong> shrinks query time dramatically — no Pinecone magic required.</li><li><strong>Citus hashing</strong> avoids cross‑shard graph hops, coordinator gather adds &lt; 20 ms.</li><li><strong>Cost ceiling</strong> stays under $600–650/month with spot GPU nodes.</li><li><strong>HA &amp; cross‑region DR</strong> are Postgres‑native, runbooks are finite and testable.</li></ol><p>With that, the DIY stack is competitive <strong>at Pinecone’s largest advertised scale point</strong>, minus vendor lock‑in, plus full SQL provenance.</p><h4>6.9 Reality‑Check — “When Do You Actually Hit a Billion Vectors?</h4><blockquote><strong>Spoiler:</strong> if your product really explodes to a billion vectors overnight, you’re either the next Google — or the victim of a rogue cron job.</blockquote><p><strong>1. It’s 5 PB of Text<br></strong>A billion 768‑dim vectors at 512 tokens per chunk equals about <strong>five petabytes of source documents</strong>. That’s every English‐language Wikipedia revision <strong>plus</strong> the complete works of Project Gutenberg <strong>600 times over</strong> — enough printed pages to pave the road from San Francisco to New York.*</p><p><strong>2. Teams Scale Before Data Does<br></strong>A pipeline gulping terabytes daily already runs Kafka, S3 lifecycle rules, and CI/CD that could land a SpaceX booster. The same DevOps crew can terraform apply three more Citus shards before lunch. No new org chart required.</p><p><strong>3. Hardware Prices Fall, SaaS Mark‑ups Don’t<br></strong>NVMe and spot‑GPU costs drop ~20 % a year. Pinecone’s per‑vector tariff is pegged to investor burn rates, not Moore’s law. Waiting six months often halves the DIY bill, waiting six months rarely halves your SaaS invoice.</p><p><strong>4. Runbooks Age Like Wine<br></strong>Every month you own the stack adds rehearsal logs, Grafana dashboards, and “we‑fixed‑it” anecdotes. By the time your corpus <em>really</em> needs a billion vectors, scaling is a checkbox — not DEFCON 1.</p><h3>7. Cost‑Control Playbook &amp; Monitoring Dashboards</h3><blockquote>Because a DIY stack only pays off if you can prove it — live, in Grafana, to whoever signs the cheques.</blockquote><h4>7.0 What Finance (and Ops) Need to See</h4><ol><li><strong>Vector growth rate</strong> Is storage trending per projections?</li><li><strong>Query p95 / p99</strong> Does latency stay inside product SLOs?</li><li><strong>Embedding spend</strong> Tokens per day × model price, even if self‑hosted later.</li><li><strong>Cache hit ratio</strong> Is Redis doing its job or just burning RAM?</li><li><strong>Shard balance</strong> No worker should hoard more than ± 10 % of vectors.</li><li><strong>Fail‑over drill logs</strong> Evidence that “active–active” isn’t powerpoint only.</li></ol><h4>7.1 Metrics Export (five lines of YAML)</h4><p>Add the postgres_exporter and redis_exporter services in the same docker‑compose.yml:</p><pre>  pg_exporter:<br>    image: quay.io/prometheuscommunity/postgres-exporter<br>    environment:<br>      DATA_SOURCE_NAME: &quot;postgresql://vecuser:secret@pg:5432/vecdb?sslmode=disable&quot;<br>    ports: [&quot;9187:9187&quot;]<br><br>  redis_exporter:<br>    image: oliver006/redis_exporter<br>    command: --redis.addr=redis://redis:6379<br>    ports: [&quot;9121:9121&quot;]</pre><blockquote>Prometheus scrapes :9187 (Postgres) and :9121 (Redis), Grafana reads Prometheus,</blockquote><h4>7.2 Grafana Dashboard Cheat‑Sheet</h4><figure><img alt="A cheat sheet with observability metrics for self-hosted vector search. Columns: Component, Quantity (promQL style), Unit Cost, and Monthly Cost Insight. Includes: Vector count per shard Query P95 duration Redis hit ratio Embedding tokens/day Write IOPS Replication lag" src="https://cdn-images-1.medium.com/max/1024/1*edueRFbj-FB8DQJn3gSrLQ.jpeg" /></figure><p>Each alert posts to Slack or PagerDuty, most stay silent in healthy steady‑state.</p><h4>7.3 Cost Dashboard</h4><p>Feed AWS Cost Explorer CSV to Grafana or use the CloudWatch‑Billing datasource:</p><ul><li><strong>Bar — “Compute vs Storage vs Transfer”</strong> Spot GPU savings instantly visible.</li><li><strong>Line — “Vector count vs Storage charge”</strong> Sanity check: dollars per million vectors should trend <em>down</em> over time (hardware deflation).</li><li><strong>Single‑stat — “Cost per 10 K queries”</strong> Divide daily query cost by Prometheus pgvector_query_total.</li></ul><p>Stakeholders finally see that re‑embedding a million vectors costs $0 in infra — versus the four‑figure line item they remember from Pinecone.</p><h4>7.4 Run‑Book Drill (Quarterly Exercise)</h4><ol><li><strong>Kill one GPU worker</strong> (aws ec2 terminate-instances ...)</li><li><strong>Observe</strong>: Citus coordinator marks shard unhealthy, reroutes queries → slight latency bump.</li><li><strong>Prometheus alert</strong> fires shard_unavailable.</li><li><strong>Terraform apply</strong> spins new spot node, attaches EBS snapshot, catches up WAL.</li><li><strong>Verify</strong> replication lag panel returns to green, alert auto‑clears.</li></ol><p>Time‑to‑recovery target: <strong>&lt; 10 minutes</strong>. Repeat with coordinator node next quarter.</p><h4>7.5 Explainability Log</h4><p>Add a tiny table:</p><pre>CREATE TABLE query_log (<br>  ts TIMESTAMPTZ,<br>  user_id TEXT,<br>  query_text TEXT,<br>  chunk_ids UUID[],<br>  embed_version embed_ver,<br>  latency_ms INT<br>);</pre><p>A Postgres trigger on the API endpoint inserts each search, letting auditors trace:</p><ul><li>Prompt → vectors returned → document IDs → edition version.</li><li>SLA outliers (latency_ms) join to system metrics for RCA.</li></ul><h4>7.6 Narrative to Finance &amp; C‑Suite</h4><blockquote>“Here’s Grafana: green means queries &lt; 30 ms and storage curve slopes cheaper each month.<br>Embedding refreshes cost time, not dollars, scaling to new regions costs a Terraform diff, not a new SaaS contract.<br>If these panels stay green, we <em>never</em> need to discuss cosine invoices again.”</blockquote><p>That’s a story even non‑engineers can parse — because the colours don’t lie.</p><h3>8. Curtain‑Raiser: <em>“Show Me the Off‑Ramp”</em></h3><blockquote>📣 Reader voice:<br> “Alright, gurus — you’ve roasted fancy‑wrapper vector DBs, dazzled me with $637/month GPU clusters, and paved the road to petascale glory. Cool story. But I already have <strong>300 million vectors</strong> living rent‑free in Pinecone‑Land.<br> <strong>Prove</strong> I can break up, keep my data, and not lose a single cosine in flight.”</blockquote><p>Challenge accepted. Grab a coffee, we’re pulling vectors out of the cloud condo and driving them home — with seat‑belts, checksum validation, and a rollback lane.</p><blockquote>(Codename inside the team: <strong>Operation Evict‑the‑Cosines</strong>.)</blockquote><h4>8.1 High‑Level Flight Plan</h4><ul><li><strong>Snapshot &amp; Export</strong> Pull all existing vectors + metadata.</li><li><strong>Schema &amp; Index Prep</strong> Create equivalent tables in Postgres.</li><li><strong>Streaming Backfill</strong> Bulk‑copy without choking disks or RAM.</li><li><strong>Dual‑Write Window</strong> App writes to both Pinecone <strong>and</strong> Postgres, reads still hit Pinecone.</li><li><strong>Read‑Shadow</strong> Background compares query results between stores, alert on mismatches.</li><li><strong>Flip Read Path</strong> Switch prod traffic to Postgres, keep dual‑write for 24 h.</li><li><strong>Retire SaaS</strong> Turn off dual‑write, delete Pinecone namespace, celebrate.</li></ul><h4>8.2 Step‑by‑Step, Zero‑Downtime Migration</h4><p><strong><em>Step-1: Snapshot &amp; Paginated Export</em></strong></p><pre>from pinecone import Pinecone, ServerlessSpec<br>import json, gzip<br><br>api = Pinecone(api_key=&quot;PINECONE_API&quot;)<br>index = api.Index(&quot;my-index&quot;)<br><br>page_token = None<br>while True:<br>    resp = index.fetch(<br>        ids=[],<br>        namespace=&quot;&quot;,<br>        filter={},                # empty = all<br>        limit=1000,<br>        include_values=True,<br>        include_metadata=True,<br>        pagination_token=page_token<br>    )<br>    with gzip.open(&quot;export.ndjson.gz&quot;, &quot;at&quot;) as f:<br>        for k, v in resp[&#39;vectors&#39;].items():<br>            f.write(json.dumps({&quot;id&quot;: k, **v}) + &quot;\n&quot;)<br>    page_token = resp.get(&#39;next_page_token&#39;)<br>    if not page_token:<br>        break</pre><p><em>Runs at ~15 K vectors/s, 300 M vectors export in &lt; 6 h on t4g.medium.</em></p><blockquote><strong><em>Other vendors?</em></strong><em> Replace </em><em>index.fetch with corresponding SDK:<br> • Weaviate: </em><em>/objects?limit=1000&amp;cursor=<br> • Qdrant: </em><em>/scroll endpoint<br> • Zilliz: </em><em>/entities?offset=</em></blockquote><p>All expose paginated “return vectors + payload” calls.</p><p><strong><em>Step-2: Snapshot &amp; Paginated Export<br></em></strong><em>Reuse Section 4.2 DDL, ensure </em><em>id matches vendor ID type—often </em><em>TEXT.</em></p><pre>ALTER TABLE chunks ALTER COLUMN id SET DATA TYPE TEXT;</pre><p><strong><em>Step-3: </em></strong>Bulk Ingest Loader</p><pre>import psycopg, gzip, json, numpy as np<br>pg = psycopg.connect(&quot;postgres://vecuser:secret@localhost/vecdb&quot;).cursor()<br><br>with gzip.open(&quot;export.ndjson.gz&quot;, &quot;rt&quot;) as f:<br>    batch = []<br>    for line in f:<br>        rec = json.loads(line)<br>        batch.append((<br>            rec[&quot;id&quot;],<br>            rec[&quot;metadata&quot;].get(&quot;doc_id&quot;),<br>            rec[&quot;metadata&quot;].get(&quot;idx&quot;, 0),<br>            rec[&quot;metadata&quot;].get(&quot;text&quot;, &quot;&quot;),<br>            np.array(rec[&quot;values&quot;], dtype=np.float32).tolist()<br>        ))<br>        if len(batch) == 5000:<br>            pg.executemany(&quot;INSERT INTO chunks VALUES (%s,%s,%s,%s,%s) ON CONFLICT DO NOTHING&quot;, batch)<br>            pg.connection.commit()<br>            batch.clear()</pre><blockquote>5000‑row batches keep wal‑compression happy, sustained 50 K rows/s on m7a.large.</blockquote><p><strong><em>Step-4: </em></strong>Bulk Ingest Loader</p><pre>def upsert_vector(id, vec, meta):<br>    pinecone.upsert(id=id, values=vec, metadata=meta)<br>    pg.execute(&quot;INSERT INTO chunks VALUES (%s,%s,%s,%s,%s) ON CONFLICT DO NOTHING&quot;,<br>               (id, meta[&quot;doc_id&quot;], meta.get(&quot;idx&quot;,0), meta.get(&quot;text&quot;,&quot;&quot;), vec))</pre><blockquote>Add a feature flag USE_POSTGRES_READ=False—initially reads go to Pinecone, writes hit both.</blockquote><p><strong><em>Step-5: </em></strong>Read Shadow &amp; Diff Audit</p><pre>def compare(query_vec):<br>    pine = pinecone.query(query_vec, top_k=5)<br>    pg.execute(&quot;&quot;&quot;<br>      SELECT id, embedding &lt;-&gt; %s AS dist<br>      FROM chunks ORDER BY dist LIMIT 5<br>    &quot;&quot;&quot;, (query_vec,))<br>    pgrows = [r[0] for r in pg.fetchall()]<br>    mismatch = [a for a,b in zip(pine.ids, pgrows) if a!=b]<br>    if mismatch:<br>         log.warn(&quot;ANN divergence&quot;, pine=pine.ids, pg=pgrows)</pre><blockquote>Run as background job, divergence rate &lt; 2 % after index warms.</blockquote><p><strong><em>Step-6: </em></strong>Flip Read Path &amp; Monitor<br>Set USE_POSTGRES_READ=True.<br> Grafana panels watch query p95 + mismatch gauge.<br> Stay dual‑write for 24 h. If green, disable Pinecone writes.</p><p><strong><em>Step-5:</em></strong> Decommission SaaS</p><pre>pinecone namespace delete my-index<br>aws billing put-budget --amount 0 --alert-if-not-zero</pre><blockquote>Pop confetti.</blockquote><h4>8.3 Edge‑Case Guard‑Rails</h4><figure><img alt="A table of risks and mitigation strategies for DIY vector DB systems: Vector norm differences: Normalize manually before Postgres insert Metadata exceeds JSONB limits: Use separate table keyed by chunk_id Time-skew during dual writes: Use TIMESTAMPTZ DEFAULT now() Disk-fill risk: Place chunks on gp3 SSD with IOPS alarms and yearly partition pruning" src="https://cdn-images-1.medium.com/max/1024/1*jv318rQke27lnp-kWXGJ6g.jpeg" /></figure><h4>8.4 Quick Farewell to the Fancy‑Wrapper Vector DB</h4><blockquote>Because the breakup text deserves a flourish.</blockquote><figure><img alt="A distressed solo founder works late at night in a dim, cramped office labeled “Plan: FREEMIUM.” A motivational poster behind him reads “OUR AI VISION: DEMOCRATIZE KNOWLEDGE” with an upward chart. Long invoices cascade down the wall beside him, labeled: Pinecone: $2,141.55 Embedding API: $3,980 GPU Inference: $800 Bill Embedder On his laptop, a “Stall. 2 users” message is visible. Steam rises from his coffee mug as he freezes in panic. Two wide-eyed kids peek into the room. One says: “Um…" src="https://cdn-images-1.medium.com/max/1024/1*Ds6rX579kchHEVFaeHveXw.png" /></figure><p><strong><em>Dear ✨Wrapper‑DB✨,</em></strong><br>You were the glossy stroller that toted my baby vectors around demo‑day. I loved the cup‑holders, hated the invoice. Now the kids are teenage tensors skate‑boarding across NVMe lanes, and your premium stroller fee feels like college tuition for tricycle lessons.</p><p>I’ve packed their 768‑dim lunchboxes, labelled every metadata sock, and buckled them into the family Postgres minivan — complete with Redis turbo and unlimited snack‑cache. Keep the swag hoodie, I’m keeping my cosines (and my budget).</p><p><strong><em>💔 Cosine ≈ 0,<br></em></strong>— A newly frugal, self‑hosting parent</p><h3>🌱 Epilogue: A Note to the Builders (Big and Small)</h3><h4>🥺 To every indie hacker, weekend tinkerer, and solo dev:</h4><p>You don’t need a seven‑figure SaaS bill to ship something legendary. A dusty laptop, Postgres, and a stubborn curiosity still punch far above their weight. If this guide saves you even one month of rent — or funds your next open‑source pull request — count that a win. Keep asking “couldn’t I just run this myself?” Nine times out of ten, the answer is “yep.” And for that tenth time, there’s a friendly Discord of fellow penny‑pinchers ready to debug at 2 a.m.</p><h4>🧐 To the big‑logo engineering leads and VPs reading over someone’s shoulder:</h4><p>Open‑source codebases kept your infra humming long before “vector DB” became a Gartner keyword. If self‑hosting isn’t on your roadmap, at least put a slice of the cosine budget toward the projects that made it possible — pgvector, Redis, Citus, Sentence‑Transformers, and the scores of tireless maintainers behind them. Funding OSS isn’t charity, it’s paying the water bill on the well you drink from daily.</p><h4>🥳 And to every fancy wrapper service out there:</h4><p>We still like your dashboards, but remember: the community can — and will — build the next great tool in a README and a weekend. The best way to stay loved is to stay honest, stay interoperable, and keep those pricing pages shorter than your quick‑start.</p><p><strong>Happy hacking, and here’s to owning the stack — one L2‑normalised cosine at a time.</strong></p><p>Love (cosine ≈ 0.00) ✨<br><strong>Your Friendly Open‑Source Cheer Squad</strong></p><blockquote><strong>“Curious about sharding tricks, GPU‑dense clusters, or that wild idea of running a vector index on a Raspberry Pi taped to your monitor?</strong><br> <strong>Ping me.</strong> I’ll happily dive into any architecture rabbit‑hole you throw my way, code and coffee included.”</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b3a5038e751" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The RiddgeApp Chronicles: An Inside Look from PD1 to PD5]]></title>
            <link>https://fybyte.medium.com/the-riddgeapp-chronicles-an-inside-look-from-pd1-to-pd5-a5cfb198e4c2?source=rss-197231c82a7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a5cfb198e4c2</guid>
            <category><![CDATA[android]]></category>
            <category><![CDATA[journey]]></category>
            <category><![CDATA[updates]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Fybyte Technologies]]></dc:creator>
            <pubDate>Thu, 28 Sep 2023 15:33:17 GMT</pubDate>
            <atom:updated>2023-09-28T15:34:08.176Z</atom:updated>
            <content:encoded><![CDATA[<p>Hello, fellow tech aficionados and RiddgeApp enthusiasts! Embark with me on a behind-the-scenes journey from PD1 to PD5 as we piece together the puzzle of RiddgeApp’s evolvement, complete with its highs, hiccups, and hopes. 🚀</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Wm_dqcerASVJyNpl" /><figcaption>Photo by <a href="https://unsplash.com/@avirichards?utm_source=medium&amp;utm_medium=referral">Avi Richards</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>Chapter 1: The Backend Ballet (PD1)</strong></p><p>We began our tale with an exciting focus on backend APIs. As with any tech narrative, new features often lead to new challenges. Crafting intricate queries for group conversation permissions was like composing a symphony — each note carefully placed to ensure harmony. The front end didn’t remain untouched as we introduced two screens, focusing on group chat dynamics.</p><p>The goal? Seamlessly weaving the admin/moderator permissions into a user-friendly design. There was a foray into the world of containers with Docker, promising a modular approach and setting the stage for what was to come.</p><p><strong>Chapter 2: Spaces, The Final Frontier (PD2)</strong></p><p>In this chapter, we delved deeper into ‘spaces.’ Spaces, the lifeblood of RiddgeApp, underwent an overhaul. We recognized users’ desire for distinctiveness and introduced unique space names, adding an extra layer of identity. The goal? Encourage creators to champion their spaces and enhance the search experience.</p><p>This wasn’t just a facelift, it was a transformative pivot, changing the way users engaged with spaces.</p><p><strong>Chapter 3: The Media Melodrama and Version Ventures (PD4)</strong></p><p>The saga witnessed a grand plot twist! From introducing media variants to tackling buffer size issues, it was a roller-coaster of tech thrills. With version v0.0.38, we tasted both triumph and tribulation, highlighting the very essence of development — evolution through iteration.</p><p>Between PD4 and PD5, three distinct app versions emerged from our digital forge, each bringing its own set of adventures and anecdotes.</p><p><strong>Chapter 4: Data Dramatics and Feed Finesse (PD5)</strong></p><p>Our trusty aide, Postgresql, stood steadfast. With some data wizardry, we managed to whip our user feed response into shape. From a leisurely 2.5 seconds, we sprinted to a lightning-fast 200ms. But, as with all good tales, there were crossroads. A potential hybrid approach for user feed generation emerged, promising the perfect blend of precomputed and reference-based strategies.</p><p>And then, the spotlight turned to AI! The promise of personalized user feeds loomed on the horizon, paired tantalizingly with the introduction of categories.</p><p><strong>Chapter 5: The Road Ahead</strong></p><p>With each update, it’s clear that RiddgeApp isn’t just an app, it’s an ever-evolving entity. Challenges emerge as we shift gears and explore new territories, but so do opportunities. The iOS realm beckons, a web version teases, and the AI revolution promises a tantalizing future.</p><p>For those who’ve accompanied us on this journey, we invite you to stay. And for newcomers? Jump into this whirlwind tech adventure with RiddgeApp. Together, let’s write the next chapter!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a5cfb198e4c2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building RiddgeApp: A collaborative social media platform for like-minded people]]></title>
            <link>https://fybyte.medium.com/building-riddgeapp-5322932e2c7b?source=rss-197231c82a7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/5322932e2c7b</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[social-media]]></category>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[app-development]]></category>
            <category><![CDATA[ui-design]]></category>
            <dc:creator><![CDATA[Fybyte Technologies]]></dc:creator>
            <pubDate>Tue, 04 Oct 2022 08:34:46 GMT</pubDate>
            <atom:updated>2022-10-05T02:16:23.144Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JBbTogFzmTSrw7iWMrWxbA.jpeg" /></figure><p>One of the core ideas behind RiddgeApp was to provide an easy way for people with similar interests to connect and maintain those connections. Social media is a great tool to bring like-minded people together, but often it’s hard to find the right relationship and the people who’ve similar tastes to you. By providing an easy way for people with similar interests to connect, RiddgeApp allows you to build your network faster than ever before.</p><h3><strong>What can you do with RiddgeApp?</strong></h3><p>But what’s new here? Well, not every picture can be posted in the era of instant content sharing. Therefore, we have an inbuilt feature where you can share instant stories that last up to 24 hours! In addition to that, there is a lot more to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iYbhujxdrMJMZ4LbTXLOxw.jpeg" /></figure><p><strong>Sharing stories</strong> is more fun with the RiddgeApp as the “number” of stories you share won’t be visible to your followers and what they see is the content you’ve shared and not “how many. “You can share as many stories as you wish without worrying about the number. Along with sharing stories, you can also upload your favorite pictures and make moments on the RiddgeApp, which will stay on your feed forever, reminding you of the good days. Isn’t that amazing?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W24l5FAJNS0bh1geUIlzcg.jpeg" /></figure><h4>Feature your moments</h4><p>Making posts is a thing of the past; it’s time to share “moments” with your audiences.</p><p>You can now upload your content and share pictures, and they’ll get featured under the moment’s section. We believe that every post you make is a token of love and contributes to our platform in a unique way. Hence we call it “moments.”</p><p>But what if I want to keep my moments private and limited to some people instead of showcasing them to the world? Well, we’ve got you covered. We understand that privacy on social media is of utmost importance &amp; hence we’ve tried to secure your accounts with more privacy features. Here’s how-</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Ku0OFDPeJ0uLa204z6pnw.jpeg" /></figure><h4>Make it anonymous; make it fun.</h4><p>Private accounts are outstanding, but what if you make them anonymous for the people you don’t want in your network? We know that Social media is all about networking and socializing, but not everyone wants to reveal their personal life to a specific category of people; it can be your colleagues or relatives you probably want to avoid. Therefore, we’ve introduced this distinctive feature of making private accounts more private. You can now create a personal account and make it anonymous for people not on your follower list.</p><p>It’s been a decade since we’ve been running away from social media to avoid interacting with people we don’t want to know we exist. Hence, we’ve heard you clearly and introduced this new feature. Try it out, and thank us later.</p><p>That’s not it; if you want to be “out there” and become an influencer, we have a staggering feature to make collaborations more fun and interactive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DyICfOTnd2uy91BjXoqGlw.jpeg" /></figure><h3>Introducing Spaces</h3><p><strong><em>Create and collaborate more with our unique feature of spaces.</em></strong></p><p>Spaces aren’t necessarily about stars and moon; these aren’t Twitter spaces. Let’s make it simple for you -</p><p>We’ve introduced an eccentric feature where you can collaborate with other creators and even post content of different genres under one section to make content creation more fun. How? Just by creating your customized space! No more hassle of making multiple accounts to create content in different niches.</p><p>You can now build your community in our “spaces” section and create multiple content categories. You can also get in touch with other creators and collaborate under spaces.</p><p>Not just that, you can also repost content that you love simply with our repost option.</p><p>Final thought, building RiddgeApp is so much fun. I’m single-handedly working on this project, but I would love to work with people who share the same passion and love to build products for the masses.</p><p>That’s it from this article! Join RiddgeApp early access and claim your username. Please share your feelings about the app after you try them in the comments. I would love to check them out. Currently, RiddgeApp is available only in India for Android users, but I’m working on an iOS app and will launch it soon :)</p><p>With ❤ from India</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5322932e2c7b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>