<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vipul Shah on Medium]]></title>
        <description><![CDATA[Stories by Vipul Shah on Medium]]></description>
        <link>https://medium.com/@vipulmshah?source=rss-5489e5bfa115------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*tiWwfPdBi3Qk3O-w.</url>
            <title>Stories by Vipul Shah on Medium</title>
            <link>https://medium.com/@vipulmshah?source=rss-5489e5bfa115------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 04:11:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@vipulmshah/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AI Agent Reliability in Production: A Fraud-Detection Mindset]]></title>
            <link>https://vipulmshah.medium.com/ai-agent-reliability-in-production-a-fraud-detection-mindset-c0294bba8dbe?source=rss-5489e5bfa115------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0294bba8dbe</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[observability]]></category>
            <dc:creator><![CDATA[Vipul Shah]]></dc:creator>
            <pubDate>Wed, 24 Jun 2026 23:52:38 GMT</pubDate>
            <atom:updated>2026-06-26T21:12:24.452Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Find production failures the way fraud teams find fraud.</em></p><p>First in a series.</p><p>Before launch, a support agent handles refunds perfectly. In production, for one product line — and only when a customer mentions the “holiday bundle” promo - it quietly applies last year’s refund rules and approves refunds it should have denied. Every tool call returns success. Every eval passes. It runs for weeks before anyone connects the margin dip to the agent.</p><p>Nobody wrote an eval for that, because nobody imagined it — that exact product line, that exact phrasing, a stale policy version, one combination out of thousands. No crash, no error, no red dashboard: just a silent, semantic failure quietly eroding reliability.</p><p>And that failure is only one aspect of a much bigger problem.</p><p><strong>What are Unknowns</strong></p><p>Every day, production serves up what you never planned for. <strong>Unknowns</strong> are the patterns in production you didn’t anticipate, and hence you aren’t watching for them.</p><p>The most common sources of Unknowns are:</p><ul><li><strong>New Users &amp; Cohorts</strong> who behave differently than you anticipated</li><li><strong>New Use Cases</strong> that you didn’t build for</li><li><strong>Upstream Changes</strong> due to pricing, policy or API shifts</li><li><strong>Model Changes</strong> done silently by their providers</li><li><strong>Agent Capability Gaps</strong> in reasoning, planning, acting &amp; reflection</li><li><strong>Agent Coordination Gaps</strong> in orchestration, termination &amp; concurrency</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ccXrFqbPqpbw0rofgy2ZUQ.png" /><figcaption>Sources of Unknowns in Production</figcaption></figure><blockquote><strong>So how do you make an AI agent reliable when so much of what it meets in production is something nobody anticipated?</strong></blockquote><p><strong>Applying a Fraud-detection Mindset</strong></p><p>You can’t write an eval for what you never imagined, so discovery has to happen in production, continuously. And it’s genuinely hard:</p><ul><li>every piece of telemetry needs to be analyzed, near real-time</li><li>there are no labels for failures nobody has named</li><li>the patterns are rare against a flood of normal traffic, and</li><li>they’re compound - spread across sessions, turns, and agents rather than sitting on a single axis.</li></ul><p>The right mental model isn’t “more dashboards” - it’s <strong>fraud detection</strong>.</p><p>A fraud system doesn’t just look for known behavior patterns, it:</p><ul><li>analyzes all telemetry and events</li><li>flags novel fingerprints - those rare, compound patterns</li><li>triages and isolates them, and</li><li>confirms by recurrence and root cause</li></ul><p>These steps mirror the established fraud-management lifecycle.</p><p>Apply the same mindset to agents: take the refund-promo failure from the top - no single trace looks wrong, but cluster the traffic and the pattern jumps out: same intent, same product line, same promo phrasing, same stale-policy citation, a few times this week. That cluster <em>is</em> the unknown.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YGX55K7waUrt298OorDIcQ.png" /><figcaption>The Fraud-detection Mindset</figcaption></figure><blockquote><strong>Discovering unknowns needs a fraud-detection mindset that enables you to hunt for the anomalous patterns across many different axes.</strong></blockquote><p><strong>From Sources to Unknowns</strong></p><p>Earlier, we listed six broad sources of Unknowns. There’s no single detector that catches them all - each source hides differently, so discovery means matching the right anomaly-detection lens to each.</p><p>New behavior surfaces as emerging clusters; drift surfaces as a shift against a baseline; silent failures surface only as compound, sub-threshold combinations. Apply the relevant set of techniques to each source, and the corresponding Unknowns surface.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i3RQv95b7ysZhPLv01TXnA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Jzwpj6vvSiwvUphyvvraA.png" /></figure><blockquote><strong>Let’s look at that last kind, the Silent Failures.</strong></blockquote><p><strong>Silent Failures Compound</strong></p><p>Silent failures don’t just lurk <em>*inside*</em> a single agent. Within one agent, a wrong reasoning step, a bad tool call, or a forgotten instruction can each seed a failure. But the moment you compose more agents, a second gap opens <em>*between*</em> them - and it’s the harder kind.</p><p>Take a concrete case: a planner routes a sub-task to the wrong specialist; that specialist does its job correctly on the wrong input; a verifier signs off because its own checks pass. Every agent is individually “green,” yet the order ships wrong. The failure isn’t in any one agent — it’s in the coordination. One agent’s early, confident mistake becomes the next agent’s trusted input, laundered through layer after layer of plausible reasoning until a human finally sees the damage at the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9pn7-lQErEQyX7vko0imRA.png" /><figcaption>Why Silent Failures Compound</figcaption></figure><p>So the gap doesn’t add up as you scale - it multiplies. Each agent you add expands the ways the system can interact, and the share of failures nobody anticipated grows with it.</p><p><strong>Within a single agent (Intra-Agent)</strong></p><p>Recent failure taxonomies converge on the same picture — agents break along their own modules:</p><ul><li><strong>Reasoning &amp; planning:</strong> ungrounded hallucinations, flawed or incomplete plans, goal drift, reasoning-action loops</li><li><strong>Memory &amp; context:</strong> retrieval misses and context loss across turns (amnesia) — empirically the single largest bucket</li><li><strong>Reflection:</strong> the agent misjudges its own progress and never self-corrects</li><li><strong>Tool &amp; action:</strong> wrong tool, malformed arguments, or misreading what a tool returned</li><li><strong>System / execution:</strong> environment and API errors, plus runaway cost from loops and retries</li></ul><p><strong>Between the agents (Inter-Agent)</strong></p><p>Once agents are composed, a second set appears:</p><ul><li><strong>Orchestration:</strong> mis-routed or missing sub-tasks</li><li><strong>Communication:</strong> message storms</li><li><strong>Cascading hallucinations</strong></li><li><strong>Silent delegation</strong> failures</li><li><strong>Termination:</strong> premature stops</li></ul><p>These play out across axes of scope: <strong>Turns, Sessions, and Long-running Tasks.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aJ7MXlLU_MctqzZiWbSxeg.png" /></figure><p>As the failure surface compounds, the relative share of silent failures grows, and they become the majority of the total failure space.</p><p>Writing evals for known failures is necessary but not sufficient for reliability. Hence, discovering the <em>*silent failures*</em> is critical. But most teams only handle the known bucket — evals, tracing, dashboards — which is exactly why the expensive failures slip past.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3HWQkSDMAIrN4Ef72CBisA.png" /><figcaption>Relative Share of Silent Failures Grows with More Agents</figcaption></figure><p><strong>Closing the Loop</strong></p><p>Surfacing a pattern isn’t the finish line — it’s the start of a loop, and each turn makes the next ship more reliable. The steps:</p><ol><li><strong>Discover</strong> - a candidate pattern surfaces from production traffic.</li><li><strong>Confirm</strong> - is it a real, recurring failure or just noise? Enough evidence turns a candidate into a confirmed pattern.</li><li><strong>Root-cause</strong> - <em>why</em> is it happening? In the refund example: the agent pulled a stale policy document for that one product line.</li><li><strong>Fix + New Eval</strong> - resolving the anomaly produces two things together: the <strong>fix</strong> that clears it now (point the agent at the current policy), and a <strong>new eval</strong> distilled from it (assert that refunds on that product line cite the current policy). The fix stops today’s failure; the eval makes sure it can’t quietly come back.</li></ol><p>The result is a ship that’s <em>regression-proofed</em> - more reliable not only because you fixed the issue, but because you’ve guaranteed it won’t silently return.</p><p>And every confirmed unknown becomes a known, so your coverage improves over time, making the system more reliable. It’s the same loop modern fraud systems run - unsupervised detection surfaces the novel pattern, and confirmed cases are fed back to harden the known-pattern model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5B-zHoNDIssPUxve36l5Tw.png" /><figcaption>Feedback loop that improves reliability of AI Agents</figcaption></figure><p><strong>Where this leaves us</strong></p><p>Observability gives you passive telemetry. Evals tell you agents passed the criteria you wrote. Neither tells you about the failures you didn’t think to look for — and in production, that’s most of them, and rising as you add agents.</p><p>Operating agents reliably takes the whole loop: discover the unknowns, root-cause them, fix them, and fold each one back into what you watch for next time. That’s reliability beyond observability and evals.</p><p>If you’re shipping agents and getting surprised in production despite doing everything right, you’re not doing it wrong — you haven’t applied the right tooling for that problem.</p><p><strong>References</strong></p><ul><li><a href="https://arxiv.org/abs/2505.08638">TRAIL: Trace Reasoning and Agentic Issue Localization</a> (arXiv:2505.08638)</li><li><a href="https://arxiv.org/abs/2509.25370">Where LLM Agents Fail and How They Can Learn From Failures (AgentErrorTaxonomy)</a> (arXiv:2509.25370)</li><li><a href="https://arxiv.org/abs/2508.13143">Exploring Why Autonomous Agents Fail</a> (arXiv:2508.13143)</li><li><a href="https://arxiv.org/abs/2606.01581">Agent System Operations: Categorization, Challenges, and Future Directions</a> (arXiv:2606.01581)</li><li><a href="https://www.sciencedirect.com/science/article/abs/pii/S0020025519304451">Carcillo et al., Combining Unsupervised and Supervised Learning in Credit Card Fraud Detection (Information Sciences, 2019)</a></li><li><a href="https://www.utica.edu/academic/institutes/ecii/publications/articles/BA309CD2-01B6-DA6B-5F1DD7850BF6EE22.pdf">Wilhelm, The Fraud Management Lifecycle Theory (Journal of Economic Crime Management, 2004)</a></li></ul><p><em>Vipul Shah is co-founder of </em><a href="https://lumoz.ai/"><em>Lumoz AI </em></a><em>- a Production Reliability Ops solution for AI Agents, helping teams find and fix the production failures no one thought to test for. Previously CPO at AppDynamics (Cisco), with leadership roles at VMware, Microsoft, and Oracle.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0294bba8dbe" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Layout-Aware RAG with Evidence Pins: Building Clickable Citations for PDFs Using Docling, Neo4j]]></title>
            <link>https://vipulmshah.medium.com/layout-aware-rag-with-evidence-pins-building-clickable-citations-for-pdfs-using-docling-neo4j-5305769759f0?source=rss-5489e5bfa115------2</link>
            <guid isPermaLink="false">https://medium.com/p/5305769759f0</guid>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Vipul Shah]]></dc:creator>
            <pubDate>Thu, 18 Sep 2025 03:45:00 GMT</pubDate>
            <atom:updated>2025-09-18T03:45:00.094Z</atom:updated>
            <content:encoded><![CDATA[<p>Layout-aware RAG with clickable Evidence Pins. Faithful answers, inline citations, and exact PDF highlights using Docling &amp; Neo4j.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ApCYYzuPrZArzpBKoiRXw.png" /></figure><h3>Introduction</h3><p>This is Part 1 of our series on building a complete RAG pipeline with LLMs and Neo4j.</p><p>We built a layout‑aware RAG system that turns document regions into clickable evidence pins. We built this for an AI-native Supply-Chain Risk Automation system in the AEC (Architecture, Engineering &amp; Construction) industry. However, our approach is domain-agnostic.</p><p>We chose Docling for PDF parsing because it preserves structure + bounding boxes out‑of‑the‑box, stored chunks + spatial metadata directly in Neo4j with native vector indexes, and built a lightweight agent for contextually precise retrieval.</p><blockquote><strong>The result:</strong> answers with inline citations (links) that open the documents with exact regions highlighted. This transforms the output response from “trust me” to “show me exactly where.”</blockquote><blockquote>🔗 <strong>Demo available on GitHub</strong>: <a href="https://github.com/lumozai/layout-aware-rag-demo">lumozai/layout-aware-rag-demo</a> — includes a Chainlit frontend and FastAPI backend with Docling + Neo4j, showing clickable evidence highlighting.</blockquote><p><strong>Why We Built This</strong></p><p>AEC projects deal with complex multi-modal documents in PDF formats, such as, Design specifications, Materials specifications, Building codes, ADA codes, Submittals, RFIs, and Change orders. These documents contain critical text, tables, diagrams, and regulatory requirements where precision and provenance are non‑negotiable. <em>“Pretty sure the ramp slope is 1:12”</em> isn’t good enough — you need <em>the clause</em>, <em>the page</em>, and <em>the rectangle on the page</em> that justified the answer.</p><p>We wanted:</p><blockquote><strong>Faithful answers</strong> with sentence‑level citations</blockquote><blockquote><strong>Contextually precise evidence: </strong>open the PDF and highlight the exact region used</blockquote><blockquote><strong>Efficient iteration</strong> on complex document types without committing to heavyweight orchestration.</blockquote><h3><strong>1. Architecture (High‑Level)</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FRhM8nUqvaFOxdDBGVKbsA.png" /><figcaption>Figure 1: Architecture (High-Level)</figcaption></figure><h3><strong>2. Agentic Framework</strong></h3><p><strong>Why We Built Lightweight First</strong></p><p>We developed a <strong>lightweight agent</strong> that executes a deterministic tool loop so we could understand its underpinnings deeply:</p><ol><li>Build query → generate embeddings → vector query in Neo4j</li><li>Expand with structural context (headings/sections/neighbor chunks)</li><li>Prompt the LLM with retrieved snippets</li><li>Parse inline citations and post‑process to evidence links.</li></ol><p><strong>This article demonstrates the foundational direct processing approach — </strong>the first installment that implements the core data pipeline (PDF parsing → chunking → vector storage → retrieval → citation linking) without LLM integration. This foundation provides the infrastructure for the single agent approach described in our architecture, which will be covered in future installments.</p><p><strong>Why lightweight first</strong></p><ul><li><strong>Simplicity &amp; latency:</strong> Fewer hops, direct execution path</li><li><strong>Deep understanding:</strong> Complete visibility into each step of the process</li><li><strong>Determinism:</strong> Easy to bisect failures and reproduce runs</li><li><strong>No-friction observability: </strong>Out-of-box tracing for quick RCA and performance analysis</li><li><strong>Upgrade path:</strong> Foundation for adopting orchestration patterns <em>later</em> without rewriting the core data pipeline</li></ul><h3><strong>3. PDF Parsing: Docling and Comparisons</strong></h3><p><strong>Our requirements</strong></p><ul><li>Preserve <strong>reading order</strong> and <strong>hierarchy</strong> (headings, lists, tables).</li><li>Emit <strong>bounding boxes</strong> (page + `[x0, y0, x1, y1]`) for every text fragment/chunk.</li><li>Support <strong>local/offline</strong> processing for performance and control.</li><li>Provide <strong>chunkers</strong> that respect document structure and token limits.</li></ul><p><strong>Quick Comparisons</strong> (practitioner’s view)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h2lyJikzvaxCu0W3Iae96g.png" /><figcaption>Figure 2: Docling and Comparisons</figcaption></figure><p><strong>Why we chose Docling for this implementation</strong></p><ul><li><strong>Clean document model</strong> with <strong>bboxes</strong> and a <strong>hybrid/hierarchical chunker</strong>.</li><li><strong>Local</strong> execution and Python API/CLI that fit our pipeline.</li><li>Minimal integration overhead compared to assembling multiple tools.</li></ul><h3>4. Data Model in Neo4J</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*C67X6Aej1DQHHmr_GU3fJA.png" /><figcaption>Figure 3: Data Model in Neo4J</figcaption></figure><p>We keep <strong>`bounding box (bbox)`</strong> in PDF points, with coordinate system matching the PDF parser. For tables, we allow multiple bounding boxes (bboxes) per chunk (array) or store child <strong>Cell</strong> nodes if we need per‑cell highlights.</p><h3><strong>5. Ingestion: From PDFs → Chunks with BBoxes</strong></h3><p>You can interact with Docling via Python or its CLI. The CLI route is stable and keeps the code sample lightweight, so that’s what we show here.</p><p><strong>Environment Variables</strong></p><pre># Configure with your Neo4j credentials<br>NEO4J_URI = &quot;bolt://localhost:7687&quot;<br>NEO4J_USER = &quot;neo4j&quot; <br>NEO4J_PASSWORD = &quot;your_secure_password&quot;</pre><p><strong>1) Parse with Docling CLI</strong></p><pre># Example: parse a PDF into Docling JSON<br># (adjust to your docling install path / flags)<br>docling ./ada/2010-ada-standards.pdf \<br>  --output ./out \<br>  --ocr --to json</pre><p>This produces a JSON containing items (paragraphs, headings, tables) with page numbers, reading order, and <strong>**bounding boxes**</strong>.</p><p><strong>2) Structure‑aware chunking with Docling HybridChunker</strong></p><pre>import hashlib<br>from docling.document_converter import DocumentConverter<br>from docling.chunking import HybridChunker<br>from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer<br>from transformers import AutoTokenizer<br><br># Initialize converter and tokenizer-aware chunker<br>converter = DocumentConverter()<br>tokenizer = HuggingFaceTokenizer(<br>    tokenizer=AutoTokenizer.from_pretrained(&#39;sentence-transformers/all-MiniLM-L6-v2&#39;)<br>)<br>chunker = HybridChunker(tokenizer=tokenizer)<br><br>def make_id(text, page):<br>    return hashlib.sha1(f&quot;{page}:{text[:160]}&quot;.encode(&quot;utf-8&quot;)).hexdigest()<br><br>def structure_aware_chunking(pdf_path):<br>    # Convert PDF using Docling&#39;s native converter<br>    result = converter.convert(pdf_path)<br>    doc = result.document<br><br>    chunks = []<br>    for chunk in chunker.chunk(doc):<br>        # Extract bounding boxes from all doc_items and merge them<br>        all_bboxes = []<br>        page_num = 1<br>        headings = []<br><br>        if hasattr(chunk.meta, &#39;doc_items&#39;) and chunk.meta.doc_items:<br>            for doc_item in chunk.meta.doc_items:<br>                if hasattr(doc_item, &#39;prov&#39;) and doc_item.prov:<br>                    prov = doc_item.prov[0]<br>                    page_num = getattr(prov, &#39;page_no&#39;, 1)<br>                    if hasattr(prov, &#39;bbox&#39;):<br>                        bbox_obj = prov.bbox<br>                        all_bboxes.append([bbox_obj.l, bbox_obj.t, bbox_obj.r, bbox_obj.b])<br><br>        # Merge all bounding boxes into comprehensive bbox<br>        if all_bboxes:<br>            min_l = min(b[0] for b in all_bboxes)<br>            max_t = max(b[1] for b in all_bboxes)<br>            max_r = max(b[2] for b in all_bboxes)<br>            min_b = min(b[3] for b in all_bboxes)<br>            bbox = [min_l, max_t, max_r, min_b]<br>        else:<br>            bbox = [0, 0, 0, 0]<br><br>        # Extract headings from chunk metadata<br>        if hasattr(chunk.meta, &#39;headings&#39;) and chunk.meta.headings:<br>            headings = [h.text if hasattr(h, &#39;text&#39;) else str(h) for h in chunk.meta.headings]<br><br>        chunk_text = chunk.text.strip()<br>        if chunk_text:<br>            chunks.append({<br>                &quot;id&quot;: make_id(chunk_text, page_num),<br>                &quot;text&quot;: chunk_text,<br>                &quot;page_num&quot;: page_num,<br>                &quot;bbox&quot;: bbox,<br>                &quot;headings&quot;: headings,<br>            })<br><br>    return chunks<br><br>chunks = structure_aware_chunking(&quot;./ada/2010-ada-standards.pdf&quot;)<br>print(f&quot;Built {len(chunks)} chunks&quot;)</pre><p><strong>3) Compute embeddings</strong></p><p>Use any text embedding model (open or proprietary). Store the vector on the chunk node. Here we use SentenceTransformers for actual embeddings.</p><pre>from sentence_transformers import SentenceTransformer<br><br># Initialize embedding model (384 dimensions)<br>embedding_model = SentenceTransformer(&#39;all-MiniLM-L6-v2&#39;)<br><br>def get_embedding(text: str) -&gt; list[float]:<br>    &quot;&quot;&quot;Generate 384-dimensional embedding vector&quot;&quot;&quot;<br>    embedding = embedding_model.encode(text)<br>    return embedding.tolist()<br><br>for ch in chunks:<br>    ch[&quot;embedding&quot;] = get_embedding(ch[&quot;text&quot;])  # 384 dims</pre><p><strong>4) Upsert into Neo4j</strong></p><pre>// Constraints<br>CREATE CONSTRAINT doc_id IF NOT EXISTS<br>FOR (d:Document) REQUIRE d.id IS UNIQUE;<br><br>CREATE CONSTRAINT page_key IF NOT EXISTS<br>FOR (p:Page) REQUIRE (p.docId, p.page_num) IS UNIQUE;<br><br>CREATE CONSTRAINT chunk_id IF NOT EXISTS<br>FOR (c:Chunk) REQUIRE c.id IS UNIQUE;<br><br>// Vector index (Neo4j 5.23+)<br>CREATE VECTOR INDEX chunk_vec IF NOT EXISTS<br>FOR (c:Chunk) ON (c.embedding)<br>OPTIONS { indexConfig: {<br>  `vector.dimensions`: 384,<br>  `vector.similarity_function`: &#39;cosine&#39;<br>}};</pre><pre>from neo4j import GraphDatabase<br><br>driver = GraphDatabase.driver(NEO4J_URI, auth=(NEO4J_USER, NEO4J_PASSWORD))<br><br>doc_id = &quot;ada-2010&quot;<br><br>def upsert(tx, doc_id, title, source_uri, pages):<br>    tx.run(<br>        &quot;MERGE (d:Document {id:$id}) SET d.title=$title, d.source_uri=$src, d.family=&#39;ADA&#39;&quot;,<br>        id=doc_id, title=title, src=source_uri)<br>    for p in pages:<br>        tx.run(<br>        &quot;MERGE (p:Page {docId:$docId, page_num:$n})\n&quot;<br>        &quot;SET p.width=$w, p.height=$h\n&quot;<br>        &quot;WITH p\n&quot;<br>        &quot;MATCH (d:Document {id:$docId}) MERGE (p)-[:OF]-&gt;(d)&quot;,<br>        docId=doc_id, n=p[&quot;page&quot;], w=p[&quot;width&quot;], h=p[&quot;height&quot;])<br><br>with driver.session() as s:<br>    # Insert document + pages (width/height available from parser or PDF metadata)<br>    pages_meta = [{&quot;page&quot;: 1, &quot;width&quot;: 612, &quot;height&quot;: 792}]  # example<br>    s.execute_write(upsert, doc_id, &quot;2010 ADA Standards&quot;, &quot;file:///ada/2010.pdf&quot;, pages_meta)<br><br>    # Insert chunks<br>    for ch in chunks:<br>        s.run(<br>        &quot;MERGE (c:Chunk {id:$id})\n&quot;<br>        &quot;SET c.text=$text, c.page_num=$page, c.bbox=$bbox, c.headings=$headings, c.embedding=$emb\n&quot;<br>        &quot;WITH c\n&quot;<br>        &quot;MATCH (p:Page {docId:$docId, page_num:$page}) MERGE (c)-[:IN_PAGE]-&gt;(p)&quot;,<br>        id=ch[&quot;id&quot;], text=ch[&quot;text&quot;], page=ch[&quot;page_num&quot;],<br>        bbox=ch[&quot;bbox&quot;], headings=ch[&quot;headings&quot;], emb=ch[&quot;embedding&quot;],<br>        docId=doc_id)</pre><h3>6. <strong>Retrieval: Vector + Structure Filters</strong></h3><p><strong>Parameters:</strong></p><ul><li><strong><em>$queryEmbedding</em></strong> — Vector representation of the user query</li><li><strong><em>$k</em></strong> — Number of candidate chunks to retrieve from vector index</li><li><strong><em>$limit</em></strong> — Final number of results to return</li><li><strong><em>$docType</em></strong> — Document category filter (e.g., ‘ADA’, ‘regulatory’)</li></ul><pre>// Given parameters above, filter by document type and sections<br>CALL db.index.vector.queryNodes(&#39;chunk_vec&#39;, $k, $queryEmbedding)<br>YIELD node AS c, score<br>WHERE c.family = $docType OR EXISTS((c)-[:IN_PAGE]-&gt;(:Page)-[:OF]-&gt;(:Document {family: $docType}))<br>MATCH (c)-[:IN_PAGE]-&gt;(p:Page)-[:OF]-&gt;(d:Document)<br>RETURN c { .id, .text, .bbox, .page_num, .headings } AS chunk,<br>       d { .id, .title } AS doc,<br>       p.page_num AS page,<br>       score<br>ORDER BY score DESC<br>LIMIT $limit;</pre><p>We often expand results by including neighboring chunks in the same section (follow a stored `:NEXT` chain or share `headings`).</p><h3><strong>7. Prompting &amp; Post‑Processing for Inline Citations</strong></h3><p><strong>Note:</strong> This section describes the future LLM-powered approach. The current demo implementation uses simple keyword-based answer generation and citation linking as the foundation for this architecture.</p><p><strong>RAG prompt policy (snippet)</strong></p><pre>You are producing document-based answers with inline citations. After every factual claim, add a bracketed list of supporting chunk IDs, e.g. [c123] or [c123, c371].<br>Only cite chunks provided in the context. Do not fabricate IDs. Prefer concise quotes.</pre><p><strong>Turning `[c123]` into links</strong></p><pre>import re<br><br>def linkify(text, id_to_payload):<br>    # id_to_payload[cid] -&gt; {docId, page, bbox: [x0,y0,x1,y1]}<br>    def repl(m):<br>        ids = [i.strip() for i in m.group(1).split(&#39;,&#39;)]<br>        links = []<br>        for cid in ids:<br>            p = id_to_payload.get(cid)<br>            if not p:<br>                links.append(cid)  # fallback<br>                continue<br>            url = (<br>                f&quot;/viewer?doc={p[&#39;docId&#39;]}&amp;page={p[&#39;page&#39;]}&quot;<br>                f&quot;&amp;bbox={p[&#39;bbox&#39;][0]},{p[&#39;bbox&#39;][1]},{p[&#39;bbox&#39;][2]},{p[&#39;bbox&#39;][3]}&quot;<br>            )<br>            links.append(f&quot;&lt;a href=&#39;{url}&#39; target=&#39;_blank&#39; rel=&#39;noopener&#39;&gt;{cid}&lt;/a&gt;&quot;)<br>        return &#39;[&#39; + &#39;, &#39;.join(links) + &#39;]&#39;<br>    return re.sub(r&quot;\[(c[0-9a-f]+(?:\s*,\s*c[0-9a-f]+)*)\]&quot;, repl, text)</pre><p><strong>PDF Viewer: Highlighting the Exact Region</strong></p><p>We use <strong>PDF.js</strong> and draw an overlay rectangle after the page renders. The key is converting <strong>PDF points</strong> (72 dpi, origin depends on pipeline) to viewport pixels.</p><pre>&lt;canvas id=&quot;pdf&quot;&gt;&lt;/canvas&gt;<br>&lt;div id=&quot;overlay&quot; style=&quot;position:absolute; pointer-events:none;&quot;&gt;&lt;/div&gt;</pre><pre>// Assumes URL like /viewer?doc=...&amp;page=...&amp;bbox=x0,y0,x1,y1<br>const params = new URLSearchParams(window.location.search);<br>const pageNum = parseInt(params.get(&#39;page&#39;), 10);<br>const bbox = params.get(&#39;bbox&#39;).split(&#39;,&#39;).map(parseFloat);<br>const [x0, y0, x1, y1] = bbox;<br><br>const url = `/documents/${params.get(&#39;doc&#39;)}.pdf`;<br><br>pdfjsLib.getDocument(url).promise.then(async (pdf) =&gt; {<br>  const page = await pdf.getPage(pageNum);<br>  const scale = 1.5; // fit to container as needed<br>  const viewport = page.getViewport({ scale });<br><br>  const canvas = document.getElementById(&#39;pdf&#39;);<br>  const ctx = canvas.getContext(&#39;2d&#39;);<br>  canvas.width = viewport.width;<br>  canvas.height = viewport.height;<br>  await page.render({ canvasContext: ctx, viewport }).promise;<br><br>  // PDF points (origin bottom-left) → canvas pixels (origin top-left)<br>  const toCanvas = (x, y) =&gt; {<br>    const px = x * scale;<br>    const py = viewport.height - (y * scale);<br>    return [px, py];<br>  };<br><br>  const [ax, ay] = toCanvas(x0, y0);<br>  const [bx, by] = toCanvas(x1, y1);<br>  const w = bx - ax, h = by - ay;<br><br>  const overlay = document.getElementById(&#39;overlay&#39;);<br>  overlay.style.left = canvas.offsetLeft + &#39;px&#39;;<br>  overlay.style.top = canvas.offsetTop + &#39;px&#39;;<br>  overlay.style.width = canvas.width + &#39;px&#39;;<br>  overlay.style.height = canvas.height + &#39;px&#39;;<br><br>  const rect = document.createElement(&#39;div&#39;);<br>  rect.style.position = &#39;absolute&#39;;<br>  rect.style.left = ax + &#39;px&#39;;<br>  rect.style.top = ay + &#39;px&#39;;<br>  rect.style.width = w + &#39;px&#39;;<br>  rect.style.height = h + &#39;px&#39;;<br>  rect.style.border = &#39;2px solid #007acc&#39;;<br>  rect.style.background = &#39;rgba(0, 122, 204, 0.15)&#39;;<br>  overlay.appendChild(rect);<br>});</pre><p><strong>Notes</strong></p><ul><li>If Docling uses a different coordinate origin (some pipelines use top‑left), invert accordingly.</li><li>For multi‑box highlights (tables), draw one `div` per bbox.</li><li>For zoom/resize, recompute rectangles after re‑render.</li></ul><h3><strong>8. Evaluation &amp; Observability</strong></h3><ul><li><strong>Structure‑first chunking pays off.</strong> Headings, list items, and table captions dramatically improved retrieval quality and reduced hallucinations versus naive fixed‑size chunking.</li><li><strong>Graph + vectors = a good combo.</strong> Vectors for recall; the graph for precision and context expansion (neighbors, sections, definitions).</li><li><strong>Deterministic agent loop.</strong> Every step logs inputs/outputs and selected chunk IDs; we can replay a session given only the user query and model settings.</li><li><strong>Latency &amp; cost.</strong> With local parsing and graph‑resident vectors, the hot path is (embedding → ANN query → prompt). We eliminated cross‑store joins.</li></ul><h3><strong>9. Security &amp; Governance</strong></h3><ul><li><strong>Local parsing</strong> avoids shipping sensitive PDFs to cloud OCR.</li><li><strong>Provenance</strong> is first‑class: every sentence links to a rectangle of source text.</li><li><strong>Audit trail</strong>: we persist the exact chunk IDs and prompt template used for each answer. This enables downstream compliance and traceability for regulated domains or audit-heavy workflows.</li></ul><h3><strong>10. What We’d Change</strong></h3><ul><li><strong>Graduate to frameworked orchestration</strong> for human‑in‑the‑loop reviews and policy gates.</li><li><strong>Table‑aware chunking</strong>: store cell‑level nodes and render multi‑cell highlights.</li><li><strong>Versioning</strong>: detect upstream doc changes and re‑chunk or re‑embed incrementally.</li><li><strong>Evaluation harness</strong>: small labeled QA set with cite‑checking, and automated regression tests on retrieval.</li></ul><h3><strong>Appendix: Quickstart (Dev Box)</strong></h3><blockquote><em>👉 Explore the full working code on GitHub: </em><a href="https://github.com/lumozai/layout-aware-rag-demo"><em>lumozai/layout-aware-rag-demo</em></a><em>.</em></blockquote><p><strong>Dependencies</strong></p><ul><li>Python 3.11+</li><li>Docling (CLI or Python)</li><li>Neo4j 5.23+ with vector indexes enabled</li><li>pdfjs‑dist (web viewer)</li></ul><p><strong>Neo4j config snippet</strong> (neo4j.conf)</p><pre>```properties<br>server.memory.pagecache.size=2G<br>server.memory.heap.initial_size=2G<br>server.memory.heap.max_size=4G<br># vector indexes are on by default in 5.23+, but confirm build</pre><p><strong>Docling install</strong></p><pre>pip install docling  # or follow project instructions</pre><p><strong>Run the pipeline</strong></p><p>1. Parse PDFs with Docling → JSON.</p><p>2. Run the chunking/embedding script.</p><p>3. Upsert nodes into Neo4j; create vector index.</p><p>4. Serve a small web app that:</p><ul><li>Accepts a query, computes an embedding, runs the Cypher.</li><li>Calls the LLM with retrieved chunks.</li><li>Post‑processes `[c123]` into `/viewer?…` links.</li><li>Displays PDF.js with highlights.</li></ul><h3><strong>Closing Thoughts</strong></h3><p>Clickable evidence pins changed how users trust AI answers. They shifted the conversation from “do I believe this?” to “show me exactly where.” By keeping the agent simple, choosing a parser that preserves layout, and storing vectors next to structure in Neo4j, we created evidence pins that are fast, explainable, and practical for real workflows. This contextually precise approach with faithful citations establishes measurable quality signals that can scale beyond individual use cases.</p><p><strong>About the Authors</strong></p><p><em>Vipul Shah</em> and <em>Dinesh Nair </em>work at a stealth-mode AI company focused on AI agent reliability and security. The techniques described here emerged from earlier work on supply chain document processing and now inform broader research into trustworthy AI systems.</p><p><strong>Disclaimer</strong></p><p><strong><em>Legal Notice:</em></strong><em> The 2010 ADA Standards for Accessible Design referenced and parsed in this article are published by the U.S. Department of Justice and are in the public domain, as works of the United States government. This article is intended for educational and technical demonstration purposes only. It does not modify, interpret, or substitute for any legal obligations under the Americans with Disabilities Act (ADA). The official version of the 2010 ADA Standards can be accessed here: https://www.ada.gov/regs2010/2010ADAStandards/2010ADAStandards.pdf</em></p><p><strong><em>Technical Disclaimer:</em></strong><em> Code examples and architectural patterns in this article are provided for educational and demonstration purposes only. They may not be production-ready and should be thoroughly reviewed, tested, and adapted before implementation. No warranty is provided regarding fitness for any particular purpose.</em></p><p><strong><em>Data Privacy:</em></strong><em> All examples use synthetic or publicly available data. Readers should implement appropriate data privacy, security, and compliance measures for their specific use cases and jurisdictions.</em></p><p><strong><em>Licenses:</em></strong><em> Readers are advised to check licensing terms for Docling and Neo4j before deploying these tools in their environments.</em></p><p><strong><em>Third-Party Tools:</em></strong><em> References to third-party tools, libraries, and services are for informational purposes only and do not constitute endorsement. Users should evaluate tools independently and verify current documentation.</em></p><p><strong><em>Views Expressed:</em></strong><em> Opinions and technical approaches described represent the authors’ perspectives and do not necessarily reflect the views of any employer or organization.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5305769759f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>