<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Pallavi Chandrashekar on Medium]]></title>
        <description><![CDATA[Stories by Pallavi Chandrashekar on Medium]]></description>
        <link>https://medium.com/@pchandrashekar?source=rss-40c8597445e3------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*QM7q1R4CbvQr7UNllnMIog.jpeg</url>
            <title>Stories by Pallavi Chandrashekar on Medium</title>
            <link>https://medium.com/@pchandrashekar?source=rss-40c8597445e3------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 20:30:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@pchandrashekar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Here’s What I Learned About the Gap Between “I Have Data” and "I Have Answers"]]></title>
            <link>https://pchandrashekar.medium.com/heres-what-i-learned-about-the-gap-between-i-have-data-and-i-have-answers-f482095f618b?source=rss-40c8597445e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/f482095f618b</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Pallavi Chandrashekar]]></dc:creator>
            <pubDate>Fri, 13 Feb 2026 00:00:01 GMT</pubDate>
            <atom:updated>2026-02-13T00:00:17.786Z</atom:updated>
            <content:encoded><![CDATA[<h3>I Built an AI Data Analysis Platform. Here’s What I Learned About the Gap Between “I Have Data” and “I Have Answers”</h3><p>Every data team I’ve worked with has the same bottleneck.</p><p>Someone needs answers from data. They can’t write SQL. So they file a ticket, wait three days, and get a CSV they can’t use.</p><p>The analyst rewrites the question. The engineer rewrites the query. Three rounds later, the original question has mutated into something nobody asked.</p><p>I’ve watched this cycle repeat for 9 years across enterprise teams. And I kept thinking: this shouldn’t require an engineer in the middle.</p><p>So I built InsightForge — an open-source, multi-user data analysis platform where you upload any dataset, ask questions in plain English, and get answers and visualizations instantly.</p><p>No SQL required. No ticket. No three-day wait.</p><p>This article is the story of what I built, why I made the architectural decisions I did, and what I learned along the way.</p><h3>The Problem Is Worse Than It Looks</h3><p>On the surface, “non-technical people can’t query data” sounds like a tooling problem.</p><p>It’s not. It’s a systems problem.</p><p>Here’s what actually happens in most orgs:</p><ol><li>Data lives in silos. Sales data is in one system, product data in another, and finance in a third. Before anyone can ask a question, someone has to export, clean, and join.</li><li>The people with questions aren’t the people with query skills. Product managers, ops leads, executives — they know <em>what</em> they want to know. They just can’t express it in SQL.</li><li>Every ad-hoc request becomes engineering work. A “quick question” turns into a Jira ticket, a data pull, a review, and a Slack thread about whether the numbers look right.</li><li>By the time the answer arrives, the question has changed. Business moves fast. A three-day turnaround on a data question means the decision already been made on gut feel.</li></ol><p>The real cost isn’t the engineer’s time. It’s the decisions that never get made because the data was too hard to access.</p><h3>What InsightForge Actually Does</h3><p>The idea is simple:</p><p>Upload any data → Ask questions in English → Get answers + charts.</p><p>The engineering is not.</p><p>Here’s what happens when a user interacts with InsightForge:</p><ol><li>Upload — Drop a CSV, JSON, Excel, or Parquet file. The platform detects the schema, infers column types, and profiles the data automatically. You can also paste a URL — from Kaggle, GitHub, or any public source — and InsightForge downloads both the data and the context automatically. No manual setup. The dataset description, column definitions, and metadata from the source page become your context file for free.</li><li>Ask — Type a question like <em>“What are the top 5 products by revenue?”</em> or <em>“Show me how sales trend month over month.”</em> The AI translates your question into an optimized SQL or Pandas query, executes it against your dataset, and returns the results.</li><li>Visualize — The AI analyzes your data types and distribution, then recommends the chart that actually tells the story. Bar chart for comparisons. Line chart for trends. Heatmap for correlations. You can also request specific chart types or export to Tableau.</li><li>Add Context — Upload a context file alongside your data. This file describes what the data is about, what the columns mean, business rules, relationships between fields — anything that helps the AI understand your data the way <em>you</em> understand it. InsightForge uses this context to generate better queries and more accurate answers.</li><li>Iterate — Every query is saved. Every visualization is reusable. You can refine, re-ask, and dig deeper without starting over.</li></ol><p>It’s not a chatbot wrapper around an LLM. It’s a data platform that happens to use AI as its query interface.</p><h3>The Tech Stack (And Why I Chose Each Piece)</h3><p>Every technology choice in InsightForge was driven by a specific tradeoff. Here’s the stack and the reasoning:</p><h3>Backend: FastAPI + SQLAlchemy + Pydantic</h3><p>I considered Django. FastAPI won for three reasons:</p><ol><li>Async-native. Data processing and LLM calls are I/O-heavy. FastAPI’s async support means the server doesn’t block while waiting for Claude to respond.</li><li>Auto-generated OpenAPI docs. Every endpoint gets interactive documentation at /docs for free. When you’re building an API-first platform, this saves weeks of documentation work.</li><li>Pydantic schemas end-to-end. Request validation, response serialization, and database models all share the same type system. A schema mismatch gets caught at startup, not in production.</li></ol><h3>Frontend: React 18 + TypeScript + Vite + Tailwind</h3><p>TypeScript wasn’t optional. When your frontend handles file uploads, query results with dynamic schemas, and interactive charts, type safety is the difference between “it works” and “it works reliably.”</p><p>Vite’s hot module replacement makes the feedback loop tight enough that UI iteration doesn’t slow down backend development.</p><h3>Data Processing: Pandas + NumPy + pandasql</h3><p>This was a deliberate choice over something heavier like Spark.</p><p>InsightForge processes user-uploaded datasets — typically thousands to millions of rows, not billions. Pandas handles this range efficiently without the operational overhead of a distributed compute engine.</p><p>pandasql lets me execute SQL queries directly against DataFrames. This means the AI can generate either SQL or Pandas code, and both run against the same data layer.</p><h3>AI/LLM: Anthropic Claude + OpenAI (BYOK)</h3><p>More on this in the BYOK section below. The key decision was building a provider-agnostic abstraction from day one, rather than coupling to a single model.</p><h3>Storage: PostgreSQL + Redis</h3><p>PostgreSQL stores users, datasets, queries, and visualizations. Redis caches query results — identical natural language questions return cached results in under 10ms.</p><p>Why not a vector database? InsightForge isn’t doing semantic search over documents. It’s translating natural language to structured queries. The retrieval problem is different from RAG. PostgreSQL handles it cleanly.</p><h3>Infrastructure: Docker Compose (dev + prod)</h3><p>Two compose files. One for development (hot reload, debug logs). One for production (optimized builds, health checks). Same architecture, different configurations.</p><h3>Three Architecture Decisions I’d Make Again</h3><h3>1. User-Scoped Tenant Isolation at Every Layer</h3><p>This is the decision I’m most proud of.</p><p>Every dataset, every query, every visualization, and every cache key is scoped to the user who created it. There is no path — accidental or intentional — for User A to see User B’s data.</p><p>If you’ve read my Enterprise RAG series, you know I think about tenant isolation obsessively. The same principles apply here:</p><ul><li>At the data layer: Every database query includes a user filter. No query can return cross-tenant results.</li><li>At the cache layer: Redis keys include the user ID. One user’s cached results never leak to another.</li><li>At the API layer: JWT authentication scopes every request before it touches any data.</li></ul><p>This isn’t paranoia. It’s the foundation that makes multi-user platforms trustworthy.</p><p>I’ve seen what happens when tenant isolation is bolted on after the fact. It’s always incomplete, always fragile, and always the thing that breaks during a security review.</p><p>Building it in from Day 1 costs almost nothing. Retrofitting it later costs everything.</p><h3>2. Three Query Modes in One Interface</h3><p>InsightForge supports three ways to query data:</p><ol><li>Natural language — <em>“What are the top 5 products by revenue?”</em></li><li>SQL — SELECT product, SUM(revenue) FROM data GROUP BY product ORDER BY 2 DESC LIMIT 5</li><li>Pandas-style filtering — programmatic DataFrame operations</li></ol><p>All three run against the same engine and produce the same result format.</p><p>Why? Because different users have different comfort levels. A product manager wants natural language. A data analyst might prefer SQL. An engineer might want Pandas.</p><p>Forcing everyone through one interface creates friction for someone. Offering three entry points to the same engine means the tool meets users where they are.</p><p>The query engine doesn’t care how the query was expressed. It cares about producing correct, scoped, cacheable results.</p><h3>3. AI-Suggested Visualizations Based on Data Profiling</h3><p>Most BI tools make you choose the chart type first and hope you picked right.</p><p>InsightForge inverts this. The AI analyzes your column types, cardinality, and data distribution, then recommends the visualization that best represents your data.</p><ul><li>Categorical × numeric? Bar chart.</li><li>Time series? Line chart.</li><li>Two numeric columns? Scatter plot.</li><li>High-dimensional data? Heatmap.</li></ul><p>The user can override the suggestion. But in practice, the AI recommendation is right about 80% of the time — because chart selection is fundamentally a pattern-matching problem, and LLMs are excellent at pattern matching when you give them structured context.</p><h3>Bring Your Own Context</h3><p>This is the feature that made the biggest difference in answer quality.</p><h3>The Problem</h3><p>LLMs are smart. But they’re not clairvoyant.</p><p>When you upload a CSV with columns like cust_seg, arr_mrr, and nps_q3. The AI has to guess what those mean. Sometimes it guesses right. Often it doesn’t.</p><p>And the guessing gets worse with domain-specific data.</p><p>A column called status could mean order status, employee status, project status, or ticket status — each with completely different valid values and business logic.</p><p>I watched the AI confidently misinterpret a churn_flag column as a boolean (0/1) when it was actually a categorical with five levels. The query ran. The results looked plausible. They were wrong.</p><p>No error. No crash. Just a quietly incorrect answer that would have driven a bad decision.</p><p>This is the same pattern I keep seeing in enterprise AI: the failure mode isn’t “I don’t know.” It’s “I’m confident and wrong.”</p><h3>The Solution</h3><p>InsightForge lets users upload a context file alongside their data.</p><p>A context file is a plain text or markdown document that describes:</p><ul><li>What the data represents — “This is monthly SaaS revenue data for Q1-Q4 2025.”</li><li>What columns mean — “arr_mrr is Annual Recurring Revenue divided by 12. cust_seg is one of: Enterprise, Mid-Market, SMB, Self-Serve.”</li><li>Business rules — “Churned customers have status = 5. Active customers have status = 1 or 2”</li><li>Relationships between fields — “region_id maps to the regions table. US = 1, EMEA = 2, APAC = 3”</li><li>Known data quirks — “Revenue for Jan 2025 includes a one-time adjustment. Exclude for trend analysis.”</li></ul><p>When a context file is provided, InsightForge injects it into every prompt sent to the AI. The model doesn’t have to guess. It <em>knows</em> what it cust_seg means, what values it status can take, and which months have anomalies.</p><h3>Why This Changes Everything</h3><p>Without context:</p><blockquote>“Show me churn by segment”<em> AI guesses churn_flag = 0 means churned. Returns inverted results.</em></blockquote><p>With context:</p><blockquote>“Show me churn by segment”<em> AI knows status = 5 means churned, groups by cust_seg, returns correct breakdown.</em></blockquote><p>Same question. Same data. Completely different (and correct) answer.</p><p>The context file is optional. InsightForge works without one — the AI falls back to schema inference and column name heuristics. But with context, the accuracy improvement is dramatic.</p><p>Think of it this way: the context file is to InsightForge what a data dictionary is to a new engineer joining your team. It’s the tribal knowledge that turns raw columns into meaningful business data.</p><h3>Context From URLs — Zero Effort</h3><p>Here’s where it gets interesting.</p><p>When you import a dataset by pasting a URL — say, a Kaggle dataset page — InsightForge doesn’t just download the CSV.</p><p>It scrapes the page for context too. The dataset description, column definitions, usage notes, tags — all of it gets pulled in automatically and becomes your context file.</p><p>You paste one link. The platform gets the data <em>and</em> the domain knowledge.</p><p>This means you can go from “I found an interesting dataset on Kaggle” to “I’m asking it questions with full context” in about 10 seconds. No manual context file. No column-by-column descriptions. The source already did that work — InsightForge just uses it.</p><p>This also works with GitHub-hosted CSVs, public data portals, and any URL where the surrounding page describes the data.</p><h3>What I Learned</h3><p>The best AI features aren’t always about the model. Sometimes they’re about giving the model better input.</p><p>Prompt engineering gets all the attention. But context engineering — structuring and delivering the right domain knowledge to the model at the right time — is what actually moves accuracy in production.</p><h3>Bring Your Own API Key (BYOK)</h3><p>This is the most recent addition, and it’s the one that turned InsightForge from a personal tool into something teams can actually adopt.</p><h3>The Problem</h3><p>The fastest way to kill the adoption of an internal AI tool is to share one API key across the whole team.</p><p>I’ve seen this pattern play out three times now.</p><p>First, on a platform team, where a shared service key meant one team’s batch job would eat the rate limit right before another team’s demo. Then, on a side project, I couldn’t tell if the $400 bill was from testing or actual usage. And again, when a security review flagged a single secret shared across dev, staging, and prod.</p><p>Every time, the fix wasn’t technical. It was architectural.</p><h3>The Solution</h3><p>InsightForge now supports Bring Your Own API Key with multi-provider support.</p><ul><li>Users enter their own Anthropic Claude or OpenAI API key directly in the UI</li><li>No key? No problem. The platform falls back to a server-configured default</li><li>Switch between providers based on preference or use case</li><li>Secure key handling — keys live in the session, never logged, never persisted</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*erLWW78UDpH6POpmgLiHSg.png" /><figcaption>Figure 1: Architecture diagram of InsightForge</figcaption></figure><h3>The Architecture</h3><p>I built an LLM Provider Router — a clean abstraction that sits between the query engine and the model.</p><p>The query engine doesn’t know or care whether it’s talking to Claude or GPT. It sends a prompt with schema context. It gets structured results back.</p><p>Adding a new provider means implementing one interface. The router handles:</p><ul><li>Key resolution: user key → server default → error</li><li>Provider selection based on user preference</li><li>Response normalization across different model output formats</li></ul><p>That’s it.</p><h3>Why This Matters</h3><ol><li>Cost transparency — each user tracks their own API spend</li><li>Compliance — teams use org-approved API contracts and rate limits</li><li>Model comparison — run the same query through Claude and OpenAI, compare results</li><li>Zero-friction onboarding — new users get started immediately with the server default. Power users bring their own key and control their own experience.</li></ol><p>The line between a side project and a platform is in small decisions like this.</p><p>Side project: one key in .env. Platform: Every user controls their own AI provider.</p><h3>What I Learned Building This</h3><h3>1. The AI Is the Easy Part</h3><p>The LLM integration took maybe 10% of the total development time. The other 90% was everything around it: file parsing, schema detection, error handling, authentication, caching, tenant isolation, and making the UI feel responsive while waiting for model responses.</p><p>This tracks with what I’ve seen in enterprise AI systems. The model is never the bottleneck. The data pipeline, the infrastructure, and the trust layer are.</p><h3>2. File Parsing Is Harder Than It Looks</h3><p>CSV sounds simple until you encounter: mixed encodings, inconsistent delimiters, headers that span multiple rows, numeric columns that contain one string value, dates in seventeen different formats, and files that claim to be CSV but are actually TSV.</p><p>I spent more time on the file parsing engine than on the entire LLM integration.</p><h3>3. Caching Natural Language Queries Is Tricky</h3><p>“What are the top products?” and “Show me the best-selling products” should return cached results for the same dataset. But they’re different strings.</p><p>The current implementation caches on exact string match, which is simple, reliable, and good enough. Semantic deduplication is on the roadmap, but I’d rather ship something correct than something clever.</p><h3>4. Users Don’t Read Docs. They Click Buttons.</h3><p>The API is well-documented with auto-generated OpenAPI specs. Nobody reads them. Users upload a file, type a question, and expect it to work.</p><p>This completely changed how I prioritized features. Instead of building more query options, I focused on making the default path — upload, ask, visualize — work flawlessly with zero configuration.</p><h3>The Full Stack</h3><p>LayerTechnologyFrontendReact 18 + TypeScript + Vite + TailwindCSSBackendFastAPI + SQLAlchemy + PydanticDatabasePostgreSQL + RedisData ProcessingPandas + NumPy + pandasqlVisualizationPlotly + MatplotlibAI/LLMAnthropic Claude + OpenAI (BYOK)AuthJWT + bcryptInfrastructureDocker Compose (dev + prod)</p><h3>Try It</h3><p>InsightForge is live. No setup required.</p><p>→ <a href="https://insight-forge-ui-production.up.railway.app/">Try it now</a> — upload a dataset and start asking questions → <a href="https://github.com/pallavi-chandrashekar/insight-forge">Source code</a> — clone it, fork it, extend it</p><p>If you’re building at the intersection of data infrastructure and AI, I’d love to hear what you’re working on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f482095f618b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enterprise RAG: A Production Guide from Architecture to Multi-Tenant Security]]></title>
            <link>https://pchandrashekar.medium.com/enterprise-rag-a-production-guide-from-architecture-to-multi-tenant-security-f415c47ad36c?source=rss-40c8597445e3------2</link>
            <guid isPermaLink="false">https://medium.com/p/f415c47ad36c</guid>
            <category><![CDATA[retrieval-augmented-gen]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Pallavi Chandrashekar]]></dc:creator>
            <pubDate>Thu, 05 Feb 2026 17:12:17 GMT</pubDate>
            <atom:updated>2026-02-05T17:12:17.622Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Most RAG systems look impressive in demos. Most fail quietly in production. Here’s how to build one that doesn’t.</strong></p><p>If you’ve built a Retrieval-Augmented Generation (RAG) system, you’ve probably experienced the same arc I have. The prototype works beautifully — you upload a few documents, ask a question, and the LLM returns a perfect answer. You demo it to stakeholders. Everyone’s excited.</p><p>Then you ship it to real users.</p><p>Queries get vague. Data goes stale. One customer accidentally sees another customer’s data. The LLM confidently answers with outdated information, and nobody notices for weeks.</p><p>That’s not because the LLM is flawed. It’s because <strong>enterprise RAG is a distributed system, not a script.</strong> And distributed systems need engineering — not just prompt tuning.</p><p>Over the past year, I’ve been building multi-tenant RAG platforms at enterprise scale. This article distills what I’ve learned into five areas that matter most: architecture, ingestion, search, security, and evaluation. Each section addresses a failure mode I’ve seen firsthand.</p><h3>Why Most RAG Systems Fail in Production</h3><p>Before diving into solutions, let’s name the problems. In my experience, PoCs hit the same four bottlenecks when moving from local to cloud.</p><p><strong>The Retrieval Trap.</strong> Simple cosine similarity works when queries are short and well-formed. Real users ask vague, multi-intent questions. Without hybrid retrieval and reranking, systems confidently retrieve chunks that are semantically adjacent but contextually wrong. The result isn’t obvious failure — it’s plausible wrong answers, which are far more dangerous.</p><p><strong>The Data Lag.</strong> RAG assumes the retrieved context reflects reality. In production, source data changes constantly — policies update, schemas evolve, documents get deprecated. Without change data capture (CDC) or deterministic re-embedding pipelines, your system gradually begins to answer with yesterday’s truth. This is one of the hardest bugs to detect because nothing crashes.</p><p><strong>The Security Gap.</strong> Many systems generate answers first and filter later. That works in demos, but fails security reviews. Enterprise RAG must enforce access control at retrieval time— across tenant boundaries, document-level permissions, and row-level security. Once restricted data enters the prompt, it’s already too late.</p><p><strong>The “Vibes” Evaluation.</strong> If quality is judged by spot-checking responses, regressions ship silently. Production RAG needs automated retrieval quality checks, answer faithfulness scoring, and regression detection before users notice. Without evaluation, teams lose trust — and adoption stalls.</p><p>These aren’t theoretical concerns. They’re the reason most enterprise RAG projects stall between “impressive demo” and “production deployment.”</p><h3>The Architecture That Actually Scales</h3><p>In tutorials, RAG looks simple: User → VectorDB → LLM. In production, it looks like a distributed system. And the single most important architectural decision is <strong>separating the slow, heavy data processing from fast, real-time query serving.</strong></p><p>Here’s the high-level architecture I use:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pv53OoUxCC3xpW29z8CmZw.png" /><figcaption>Figure 1: Production RAG Architecture — async ingestion separated from real-time serving</figcaption></figure><p>This separation is everything. Let me walk through the three components that matter most.</p><h3>Ingestion Engine (Async Workers)</h3><p>Ingestion is too heavy for a web request, so it runs in background workers. The hard-earned lessons here are about reliability, not speed. <strong>Idempotency is non-negotiable</strong> — hash your content and avoid duplicate vectors. Hierarchy-aware chunking beats fixed sizes every time. And running OCR plus cleaning <em>before</em> embedding saves both tokens and money.</p><h3>Hybrid Storage (Postgres + pgvector)</h3><p>Vectors alone don’t cut it for enterprise systems. Combining pgvector with PostgreSQL gives you vector search and SQL filters in a single query, HNSW indexing for low-latency retrieval at scale, and ACID-compliant deletes for compliance. That last point matters more than most teams realize — when a customer asks you to delete their data, “eventually consistent” isn’t an acceptable answer.</p><h3>Orchestration API (FastAPI)</h3><p>The serving layer isn’t a proxy to an LLM. It’s a high-performance router that runs parallel vector and keyword search using asyncio.gather, applies cross-encoder reranking (e.g., BGE Reranker), and streams responses via SSE so users aren&#39;t staring at a spinner.</p><p><strong>Why this architecture wins:</strong> because it separates concerns. Slow embeddings don’t block queries. Re-indexing doesn’t break serving. Model swaps don’t collapse the system. That’s how RAG systems survive real users.</p><h3>Ingestion and Intelligent Chunking</h3><p>If your RAG system gives bad answers, don’t blame the LLM. Blame your chunking strategy.</p><p>Most teams spend 90% of their time tuning prompts and 10% thinking about the data pipeline. In reality, it should be the other way around. The ingestion service is the quiet workhorse that turns messy human knowledge — PDFs, PowerPoints, SharePoint docs — into something a model can actually reason over. Get this part wrong, and even the smartest LLM won’t save you.</p><p>I see three levels of ingestion maturity in practice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wI6P2zzoaqxjBzSe-1ToEQ.png" /><figcaption>Figure 2: Enriched semantic chunking vs. naive fixed-size splitting</figcaption></figure><h3>Level 1: Naive Splitting</h3><p>Take a document, split it every 500 characters, and embed it. That’s it. What goes wrong is predictable: sentences get cut in half, context disappears, authorship and timing are lost. The model hallucinates because it’s literally reading half a thought.</p><h3>Level 2: Structure-Aware Chunking</h3><p>Now you start respecting the document’s layout — headers, sections, paragraphs. A header in a PDF isn’t just formatting; it’s the context for everything underneath it. Chunk a paragraph without its parent header, and you’ve already lost the subject. Tools like Unstructured.io and LlamaParse help here, or you can build custom parsing with BeautifulSoup for HTML. This is a big step forward, but still not enough for enterprise use.</p><h3>Level 3: Enriched Chunking (Enterprise-Ready)</h3><p>This is what actually works. Before embedding a chunk, you inject metadata directly into the text. Instead of embedding <em>“The revenue was $5M”</em>, you embed <em>“{Source: Q3_Report.pdf, Section: Executive Summary, Date: 2024–01} The revenue was $5M.”</em></p><p>This changes everything. When someone asks about 2024 revenue, the vector search naturally favors chunks stamped with that context. No prompt hacks. No guessing. Just better data.</p><h3>The Ingestion Checklist</h3><p>Three checks that catch more bugs than any prompt tweak:</p><ul><li><strong>Idempotency:</strong> Can you upload the same file twice without duplicating vectors? Hash your content.</li><li><strong>Staleness handling:</strong> If a file is deleted at the source, does it disappear from the vector DB immediately?</li><li><strong>Visibility:</strong> Do you track failed ingestions? PDF parsing fails far more often than people think.</li></ul><p>The mental model is simple: building a RAG system without a real ingestion pipeline is like building a Ferrari engine and feeding it dirty fuel. The engine isn’t the problem. The fuel is.</p><h3>Hybrid Search: Why Vectors Alone Aren’t Enough</h3><p>Vector databases are powerful, but they’re terrible at exact matches. If you build a RAG system using only vector search, you’ll eventually hit what I call the <strong>“Project Alpha” problem.</strong></p><p>A user asks: <em>“What is the Q3 budget for Project Alpha?”</em> Vector search happily returns documents about Project Beta and Project Gamma. Why? Because they’re semantically similar — projects, budgets, quarters, finance language. What it misses is the one thing that matters: the exact keyword “Alpha.”</p><p>Vectors compress words into abstract numbers. Great for meaning. Bad for precision. The result: the LLM confidently answers using Project Beta’s numbers. Wrong answer, high confidence — the worst possible combination.</p><p>This is why enterprise RAG needs hybrid search. Not a vector <em>or</em> keyword search. Both.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kmoLPYMh_APSmITQXQMnsA.png" /><figcaption><em>Figure 3: The hybrid search pipeline — from parallel retrieval through reranking</em></figcaption></figure><h3><strong>Step 1: Sparse Retrieval (Keyword-First)</strong></h3><p>This step answers: <em>“Which documents are even allowed to participate?”</em> Using BM25 or Postgres tsvector, this catches exact project names, IDs, and acronyms. No “Alpha” in the document? No result. Simple and strict.</p><h3>Step 2: Dense Retrieval (Semantic)</h3><p>Among the candidates, which ones <em>mean</em> the same thing? Vector search handles synonyms, paraphrases, and fuzzy phrasing. This improves recall without sacrificing the precision from Step 1.</p><h3>Step 3: Rank Fusion (RRF)</h3><p>Two retrieval methods produce two ranked lists. Reciprocal Rank Fusion merges them into one stable ordering. It rewards documents that perform well in both searches, doesn’t require tuning, and degrades gracefully when one signal is weak.</p><h3>Step 4: Reranking (Deep Reading)</h3><p>Finally, a cross-encoder reads the full query and the full document together. This step answers: <em>“Does this actually answer the question?”</em> It’s slower and more expensive, but it fixes mistakes that nothing else can.</p><p>In production, this runs on Postgres with pgvector + tsvector in the same query, RRF to merge signals, and bge-reranker-v2 to finalize relevance.</p><p><strong>The takeaway:</strong> Users don’t ask vector queries or keyword queries. They ask human questions. If your system only supports one type of search, your LLM will confidently answer the wrong thing. Hybrid search isn’t an optimization — it’s a requirement for correctness.</p><h3>Multi-Tenant Security: Preventing the “Tenant Leak”</h3><p>The scariest bug in enterprise RAG isn’t a wrong answer. It’s the <em>right</em> answer from the <em>wrong customer’s data.</em></p><p>In a traditional API, tenant isolation is straightforward — every query has atenant_id, every database call filters by it. RAG adds a dangerous new surface: the vector search. Here&#39;s what goes wrong: a user from Tenant A asks a question, vector search returns the 10 most similar chunks, three of those belong to Tenant B, and the LLM weaves them into a confident answer. Tenant A now sees Tenant B&#39;s confidential data. No error, no crash — just a compliance nightmare.</p><h3>The 3 Levels of Tenant Isolation</h3><p><strong>Level 1: Filter after retrieval.</strong> Search all vectors, then filter results by tenant_id. This is broken. If your top-k is 10 and 8 results belong to other tenants, you’re left with 2 relevant chunks. Worse, you’re still <em>reading</em> other tenants’ vectors in memory. Never do this.</p><p><strong>Level 2: Filter at retrieval.</strong> Apply WHERE tenant_id = X inside the vector query itself. This is the minimum viable approach, and most vector databases support it — pgvector with SQL WHERE clauses, Pinecone with metadata filters, Weaviate with where clauses. It works, but all tenants share the same index. At scale, one tenant&#39;s noisy data can affect another&#39;s search quality.</p><pre>SELECT content, embedding &lt;-&gt; $query_embedding AS distance<br>FROM chunks<br>WHERE tenant_id = $tenant_id<br>ORDER BY distance<br>LIMIT 10;</pre><p><strong>Level 3: Physical isolation.</strong> Each tenant gets their own index or schema — separate Postgres schemas, separate Pinecone namespaces, or separate vector collections. Zero chance of cross-tenant leakage. The tradeoff is more infrastructure overhead, but for enterprise customers with compliance requirements, this is what they expect.</p><p>My rule of thumb: sensitive data with fewer than 50 tenants gets physical isolation. Fifty-plus tenants with shared workloads use metadata filtering with strict testing.</p><h3>The 4-Layer Security Checklist</h3><p>Tenant isolation isn’t just about vectors — it’s about every layer of the stack.</p><ol><li><strong>API Gateway:</strong> Every request must carry an X-Tenant-ID header. No header, reject the request. Don&#39;t infer tenant from query content.</li><li><strong>Retrieval:</strong> Filtering happens inside the vector query, not after, and not in application code. In the database.</li><li><strong>Prompt Assembly:</strong> Log every chunk ID that enters the prompt. If a chunk from Tenant B ever appears in Tenant A’s prompt, your alerting should catch it before production does.</li><li><strong>Response:</strong> The LLM shouldn’t mention document names or metadata that could leak tenant information. Sanitize source references.</li></ol><h3>The Test That Matters Most</h3><p>Write a test that queries as Tenant A and asserts that no Tenant B chunks appear in the results. If that test ever fails, you catch it before your customer does.</p><h3>Putting It All Together</h3><p>Enterprise RAG isn’t an LLM problem. It’s a systems problem. Once you internalize that, you stop tuning prompts and start engineering pipelines.</p><p>Here’s the checklist I use before calling any RAG system production-ready:</p><ul><li><strong>Architecture:</strong> Ingestion and serving are decoupled. Re-indexing doesn’t break queries.</li><li><strong>Ingestion:</strong> Chunks are structure-aware and metadata-enriched. Idempotency is enforced. Staleness is handled.</li><li><strong>Search:</strong> Hybrid retrieval (keyword + vector) with rank fusion and cross-encoder reranking.</li><li><strong>Security:</strong> Tenant isolation at the database level, not the application level. Automated cross-tenant leak tests.</li><li><strong>Evaluation:</strong> Automated retrieval quality checks and answer faithfulness scoring. Regressions are caught before users notice.</li></ul><p>If you’re building something similar, I’ve open-sourced a reference implementation that covers multi-tenant isolation, agentic routing, hybrid storage, and session management:</p><p><strong>GitHub:</strong> <a href="https://github.com/pallavi-chandrashekar/enterprise-rag-reference-architecture">enterprise-rag-reference-architecture</a></p><p>I also wrote about each of these topics individually in my <a href="https://www.linkedin.com/in/pchandrashekar/">LinkedIn series on Enterprise RAG</a>, where I break down each component with shorter, focused takes.</p><p>If you’ve shipped RAG in production — or struggled with it — I’d love to hear what patterns worked for you.</p><p><em>Pallavi Chandrashekar is a software engineer building AI-ready data platforms. She writes about enterprise RAG, distributed systems, and the infrastructure behind production AI.</em></p><p><strong>Tags:</strong> RAG LLM System Design Software Architecture Python FastAPI Vector Database pgvector Multi-Tenancy AI Engineering</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f415c47ad36c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>