<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dami Gupta on Medium]]></title>
        <description><![CDATA[Stories by Dami Gupta on Medium]]></description>
        <link>https://medium.com/@dami.gupta?source=rss-4687bd7ed107------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*4cV-kCSTVbQ6HVy9.jpg</url>
            <title>Stories by Dami Gupta on Medium</title>
            <link>https://medium.com/@dami.gupta?source=rss-4687bd7ed107------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 00:20:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dami.gupta/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Learning LangGraph : A Journey Through Agents, Blackboards, and Bottlenecks]]></title>
            <link>https://pub.towardsai.net/learning-langgraph-the-hard-way-a-journey-through-agents-blackboards-and-bottlenecks-7a480fca3eb9?source=rss-4687bd7ed107------2</link>
            <guid isPermaLink="false">https://medium.com/p/7a480fca3eb9</guid>
            <category><![CDATA[langgraph-tutorial]]></category>
            <category><![CDATA[langgraph]]></category>
            <category><![CDATA[langgraph-agents]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Dami Gupta]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 16:58:07 GMT</pubDate>
            <atom:updated>2026-07-24T03:59:35.500Z</atom:updated>
            <content:encoded><![CDATA[<p>I set out to learn LangChain/LangGraph agents by building something small — a two-agent pipeline where one agent researches a topic and another writes it up. What follows is the path that took, in the order I actually hit each wall, not a cleaned-up curriculum. If you’re starting the same climb, the confusion points below are probably yours too.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*jvq4mqum76dujZx3KPqm6Q.png" /></figure><h3>Step 1: The minimum agent</h3><p>Before any of the interesting problems, I needed the smallest thing that could be called an “agent.” It turns out to be four pieces: a model, some tools, a system prompt, and create_agent to wire them together.</p><pre>from langchain.agents import create_agent          # langchain 1.x (NOT langgraph.prebuilt)<br>from langchain.tools import tool<br>from langchain_anthropic import ChatAnthropic<br><br># 1. Model<br>llm = ChatAnthropic(model=&quot;claude-haiku-4-5&quot;, max_tokens=1024)<br><br># 2. Tool — a plain function + @tool.<br>#    - type hints (drug_name: str) become the input schema<br>#    - the docstring is what the LLM reads to decide WHEN to call it<br>#    The model only &quot;sees&quot; the signature + docstring, so write the docstring<br>#    like an API description.<br>@tool<br>def get_drug_targets(drug_name: str) -&gt; str:<br>    &quot;&quot;&quot;Return the protein targets for a given drug name.&quot;&quot;&quot;<br>    return &quot;EGFR, BRAF&quot;   # real data-source call goes here<br><br># 3. System prompt — a plain string<br>SYSTEM_PROMPT = &quot;You are a research assistant. Use the tools to gather evidence.&quot;<br><br># 4. Build the agent<br>agent = create_agent(model=llm, tools=[get_drug_targets], system_prompt=SYSTEM_PROMPT)<br><br># 5. Run it — pass in {&quot;messages&quot;: [...]}, get the full message list back<br>result = agent.invoke({&quot;messages&quot;: [{&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;What does metformin target?&quot;}]})<br>print(result[&quot;messages&quot;][-1].content)   # last message = final answer</pre><p>LangChain runs the think → call tool → observe → repeat loop for you — you never write that loop yourself. The mental model that stuck: <strong>you write tools, LangChain runs the loop.</strong> Input and output are both a {&quot;messages&quot;: [...]} dict; the agent appends its tool calls, tool results, and final answer to that list as it works.</p><ul><li><strong>Model</strong> — ChatAnthropic(...). Newer Claude models (Opus 4.7+) reject sampling parameters like temperature; omit them for those models.</li><li><strong>Tools</strong> — any function with @tool. Type hints → input schema. Docstring → the tool&#39;s instructions to the LLM. Both matter. A string return is simplest; other types get serialized.</li><li><strong>Prompt</strong> — the system message. A plain string or a SystemMessage. Dynamic prompts go through middleware (the @dynamic_prompt decorator), not a callable passed to system_prompt.</li><li><strong>Invoke</strong> — agent.invoke({&quot;messages&quot;: [...]}). You hand it messages; the agent loops tool calls until done and returns the full message list. The last message is the answer.</li></ul><p>A few things bit me early, in case they bite you too:</p><ul><li>The import is from langchain.agents import create_agent on langchain 1.x. Older tutorials use create_react_agent from langgraph.prebuilt — still works, but it&#39;s the legacy path, and its argument is prompt= not system_prompt=.</li><li>Every tool needs a clear docstring, type-hinted args, and a string return — skip any of those and the model either won’t call it or won’t know what to do with the result.</li><li>.invoke() returns {&quot;messages&quot;: [...]}, not a plain string. You have to pull result[&quot;messages&quot;][-1].content yourself.</li></ul><p>To prove to myself I actually understood the four pieces — not just copy-pasted them — I wrote a throwaway calculator tool and forced the agent to use it:</p><pre>@tool<br>def calculator(expression: str) -&gt; str:<br>    &quot;&quot;&quot;Evaluate a basic arithmetic expression like &#39;2 + 2 * 3&#39;.&quot;&quot;&quot;<br>    return str(eval(expression))   # fine for learning; never eval untrusted input in prod</pre><p>That exercises all four pieces without touching the loop — a good gut check before moving on.</p><h3>Step 2: Getting two agents to talk</h3><p>One agent was fine, but my actual goal was a pipeline: a supervisor that coordinates a research agent and a writing agent. The moment I had two agents, a new question showed up that nothing in the “minimum agent” tutorial prepared me for: <strong>how does the research result actually reach the writer?</strong></p><p>There turned out to be two genuinely different answers.</p><h3>Option 1: Messages</h3><p>The simplest option keeps the supervisor as a single ReAct agent, with the sub-agents exposed as @tool functions. The research tool returns a <strong>string</strong>, which becomes a ToolMessage in the supervisor&#39;s message list. The supervisor LLM then <em>reads that text</em> and re-types it into the write tool&#39;s input.</p><p>So the data crosses agents <em>through the model</em> — it gets re-typed, and potentially paraphrased, along the way.</p><ul><li><strong>Pros:</strong> simple, it’s just create_agent; the LLM decides the flow (routing) for you.</li><li><strong>Cons:</strong> the handoff is lossy and non-deterministic — a smaller model may reword or drop content. The model sits in the middle of every handoff.</li></ul><h3>Option 2: Blackboard</h3><p>The alternative drops the supervisor down from an agent to a <strong>graph</strong>. I define a typed state object — the “blackboard” — and each node reads and writes named fields on it directly. research_node writes state[&quot;research&quot;]; write_node reads state[&quot;research&quot;] straight out of state. The data never passes through the LLM at all — it&#39;s a plain Python value in shared state, deterministic and exact.</p><ul><li><strong>Pros:</strong> typed, exact, no LLM paraphrasing of the data.</li><li><strong>Cons:</strong> more machinery — you have to define state, nodes, and edges — and since you’re wiring the flow yourself, keeping LLM-driven routing requires an explicit router node.</li></ul><p>The reason StateGraph is <em>required</em> for a blackboard, and not just a nicer way to write one, is that create_agent only gives you one kind of memory: the message list. It doesn&#39;t let you define your own state fields. A blackboard is my own typed state that steps read and write directly, and only StateGraph exposes that.</p><p>The trade-off, compressed to one line: <strong>Messages</strong> = the LLM routes and carries the data (simple, lossy). <strong>Blackboard</strong> = I wire the flow, state carries the data (more setup, exact).</p><p>I ended up building both versions side by side to actually feel the difference (agents/messages/ and agents/blackboard/ in the repo).</p><h3>Step 3: Getting typed data out of one agent</h3><p>The blackboard version raised a new problem almost immediately. An agent returns <em>one</em> thing — its final message. But I wanted the research agent’s result split across several blackboard fields (research, city, built_year, and so on). One string can&#39;t cleanly become several fields.</p><p>The fix is structured output: make the agent return a typed Pydantic object instead of a string, then let the node unpack it into state.</p><pre>from pydantic import BaseModel, Field<br><br>class ResearchOutput(BaseModel):<br>    attraction: str = Field(description=&quot;2 sentences about the attraction&quot;)<br>    city: str = Field(description=&quot;1 sentence about the city it is in&quot;)<br><br>agent = create_agent(<br>    model=llm,<br>    tools=[attraction_info, city_info],<br>    system_prompt=SYSTEM_PROMPT,<br>    response_format=ResearchOutput,   # &lt;- agent emits this typed object<br>)</pre><p>The agent still calls its tools normally; only its <em>final</em> answer is forced to match the schema. That result doesn’t show up in messages — it lands in a separate key:</p><pre>result = agent.invoke({&quot;messages&quot;: [...]})<br>out = result[&quot;structured_response&quot;]   # a ResearchOutput instance<br>out.attraction   # str<br>out.city         # str</pre><p>Only two things actually make this work, and it’s worth being precise about them because it’s easy to think there’s more magic involved than there is:</p><ol><li><strong>response_format=ResearchOutput</strong> tells the agent that, after it&#39;s done using tools, its final answer must match this schema. Internally it adds a step that forces the model to emit JSON matching the schema, then validates and parses it into the Pydantic object. Each field&#39;s Field(description=...) tells the model what belongs in it.</li><li><strong>Reading </strong><strong>result[&quot;structured_response&quot;].</strong> Setting response_format is what adds this key to the result in the first place. Without it, that key doesn&#39;t exist — you only get result[&quot;messages&quot;] and a plain text answer.</li></ol><p>So response_format declares the shape going in; structured_response is where the validated object comes out, with the agent&#39;s normal tool loop running in between. The node itself stays dumb plumbing:</p><pre>def research_node(state):<br>    out = research(state[&quot;topic&quot;])           # ResearchOutput<br>    return {&quot;research&quot;: out.attraction, &quot;city&quot;: out.city}</pre><p>One agent, one run, two typed fields out — no string parsing, no second agent, no guessing. The layering, in short: <strong>tool → agent → (structured object) → node → blackboard.</strong> The tool gathers, the agent produces a typed object, the node files it into state.</p><h3>The coupling this creates</h3><p>Once research was typed, adding or removing a tool suddenly touched four places instead of one: the @tool definition, the tools=[...] list, the system-prompt prose describing the tools, and the ResearchOutput schema field (plus whatever downstream code reads that field).</p><p>Two of those four are removable coupling. A TOOLS registry lets the prompt be <em>generated</em> from each tool&#39;s name and docstring instead of hand-written:</p><pre>SYSTEM_PROMPT = &quot;...\n&quot; + &quot;\n&quot;.join(f&quot;- {t.name}: {t.description}&quot; for t in TOOLS)</pre><p>The schema is different — it’s inherent coupling if you want typed output at all. Any code reading built_year has to change if that field stops being produced. You can soften it with Optional[...] + a default so a missing tool degrades to None instead of erroring (which also matches a rule I try to hold to: never fabricate a default for data you don&#39;t actually have), but you can&#39;t make that coupling free.</p><p>That gave me a rule of thumb I now apply by default: <strong>type the fields your code reasons about; leave everything else as text.</strong> A field only earns a schema slot if some code branches on it — a routing if, a DB write, a filter or sort. Fields that only get concatenated into a prompt or shown to a user can stay as text and collapse into one summary field. Then removing a tool touches its definition, the registry line, and <em>maybe</em> one schema field — not four.</p><p>I built an unstructured variant to feel this directly: one research: str blob, no Pydantic, prompt derived from the tool registry. Dropping a tool there was just deleting its definition and its registry entry — zero downstream edits, at the cost of losing individually addressable fields.</p><h3>Four variants, one dial each</h3><p>By this point I had enough variations floating around that it was worth naming them as points on a small set of axes, all solving the same research → write pipeline:</p><ul><li><strong>Baseline</strong> — static graph edges, a ReAct research agent, typed ResearchOutput. Predictable and fully addressable, but the schema and the prompt template are coupled, so changing the flow or the fields is expensive.</li><li><strong>Dynamic routing</strong> — the supervisor is itself an LLM that decides research | write | done at runtime via a next field, instead of hard-wired edges. The flow is decided by state, not code; the cost is that correctness now depends on the supervisor&#39;s prompt reliably reading the state.</li><li><strong>Fan-out</strong> — research drops the ReAct loop entirely for several concurrent direct LLM calls merged at a join. Faster and deterministic, but only works when the research dimensions are fixed in advance — there’s no agent left to pick tools.</li><li><strong>Unstructured</strong> — no Pydantic, one text blob, prompt derived from the tool registry. Cheapest to maintain when the tool set churns, but downstream code can’t read individual fields anymore.</li></ul><h3>Step 4: Why is this so slow — and what actually fixes it</h3><p>With the pipeline working, the next problem was speed. My first fan-out experiment ran research in about 3.7 seconds; the ReAct version of the same work took 13–15 seconds on the same model. That gap was big enough to be worth understanding properly rather than shrugging off.</p><p>To dig in, I added two deliberately slow tools to the ReAct research agent: weather_info, which just awaits a 3-second sleep (I/O-bound), and crowd_score, a blocking busy loop of about 60 million iterations (CPU-bound). Then I tried every lever I could think of and measured what actually moved the needle.</p><ul><li><strong>async / </strong><strong>ainvoke</strong> — helps I/O-bound: yes. Helps CPU-bound: <strong>no</strong>. Async overlaps <em>waiting</em>, not <em>computing</em>.</li><li><strong>LangGraph fan-out / </strong><strong>asyncio.gather</strong> — helps I/O-bound: yes. Helps CPU-bound: <strong>no</strong>. Same reason — concurrent awaits, but the GIL serialises CPU.</li><li><strong>Threads (</strong><strong>run_in_executor)</strong> — helps I/O-bound: yes. Helps CPU-bound: sometimes. Threads help I/O always; CPU only for GIL-releasing libraries (numpy etc.).</li><li><strong>Processes (</strong><strong>ProcessPoolExecutor)</strong> — helps I/O-bound: yes. Helps CPU-bound: <strong>yes</strong>. Separate interpreters = real CPU parallelism.</li></ul><p>The trap that caught me: marking the busy-loop tool async def changed nothing. A blocking CPU loop inside an async tool freezes the <em>entire</em> event loop — every other async tool sits and waits for it, because Python&#39;s GIL means only one thread runs Python at a time and a busy loop never hits an await to yield control. Async is only good for I/O — waiting on a network call, a sleep, an LLM response — not for raw computation.</p><p>The fix for the CPU-bound tool was asyncio.to_thread, which moves the blocking function onto a worker thread, off the event loop:</p><pre>def _crowd_score_sync(attraction: str) -&gt; str:   # plain blocking function<br>    total = 0<br>    for i in range(60_000_000):<br>        total += i<br>    return f&quot;... {total % 100} ...&quot;<br><br>@tool<br>async def crowd_score(attraction: str) -&gt; str:<br>    return await asyncio.to_thread(_crowd_score_sync, attraction)</pre><p>Measured effect on the research node: about 24 seconds down to about 13. Same agent, same tools, same output — just no longer blocking the loop.</p><p>The nuance worth sitting with: the CPU thread still holds the GIL while it computes, but the async I/O legs (network calls, sleeps) <em>release</em> the GIL while they wait, so they make progress alongside the thread. to_thread stops a busy loop from freezing async I/O — it does not give you multi-core CPU parallelism. For genuine parallel CPU work across cores, you need processes, not threads.</p><p>Timings across the whole progression, same model (Sonnet), same research node:</p><ul><li>ReAct, serial: ~15s</li><li>ReAct, sync + a “call these in parallel” prompt nudge: ~13s</li><li>ReAct, async + the same nudge: ~10.5s</li><li>Fan-out workflow (no agent at all): ~3.7s</li><li>ReAct async with the two slow tools added (3s I/O + CPU loop): ~29s — the new baseline before to_thread</li><li>Same, after moving the CPU loop to to_thread: ~13s</li></ul><h3>Why the ReAct agent stays slow regardless</h3><p>Even with every async lever pulled, the ReAct agent still pays for its <strong>decide-turns</strong>: think → batch tool calls → think again → structured-output step. Async only parallelizes the tool-<em>execution</em> slice of that; the fan-out workflow was roughly four times faster mainly because it has no decide-turns at all, not purely because of async.</p><p>That’s really the big lesson under all of this: there is no single “make it faster” lever — you have to match the fix to the actual bottleneck.</p><ul><li><strong>Many independent agent runs</strong> (research N things, or lit+trials per candidate) → asyncio.gather the whole agents. N serial agent runs collapse to roughly one agent&#39;s wall-clock; each agent still reasons internally. This is the real agentic win.</li><li><strong>One agent, several I/O-bound tool calls</strong> → async + ainvoke, batching in one turn. I/O waits overlap on the event loop.</li><li><strong>A CPU-bound step</strong> (busy loop, model.encode) → asyncio.to_thread / a process pool. Async does not parallelize CPU — a busy loop blocks the whole event loop until it&#39;s moved off-loop.</li><li><strong>The LLM won’t reliably parallelize despite the prompt</strong> → force it in code — remove the serial tools, expose only the fan-out tool. Prompt-level “do it in parallel” gets ignored often enough that structure beats prompting.</li><li><strong>Fixed, independent fields, no real decision needed</strong> → a workflow (fan-out graph), not an agent. No decide-turns at all → fastest, at the cost of LLM routing.</li></ul><p>Meanwhile, I was also working on a much bigger project of mine — <a href="https://github.com/dami-gupta-git/indication_scout">IndicationScout</a>, a multi-agent real-world-evidence tool for drug repurposing, live at <a href="https://indicationscout-production.up.railway.app/">indicationscout-production.up.railway.app</a>. I implemented these methods there, and runtime dropped from 260 seconds to 118 seconds. The win there wasn’t speeding up any single agent — it was running about six full sub-agent invocations <em>concurrently</em> via nested gather. The agents themselves were unchanged. The parallelism lived entirely at the level of &quot;how many agents are in flight at once,&quot; not inside any one of them.</p><p>Which leads to the restatement I keep coming back to: collapsing one agent’s tools into a single combiner call kills the agent, because there’s nothing left for it to decide. The real win comes from parallelizing multiple <em>independent units of agent work</em> — if there’s only one unit of work, there’s no fan-out win available without giving up the agent entirely. An agent’s value <em>is</em> its decision-making, and decision-making <em>is</em> the serial round-trips that make it slow. You can’t fully have both at once. Keep the agent where the LLM genuinely needs to choose something; drop to a plain workflow where the steps are fixed.</p><h3>Step 5: Running whole pipelines concurrently</h3><p>A smaller but related discovery: asyncio.gather(supervise(a), supervise(b), ...) runs several independent pipeline runs on a <em>single</em> event loop. While one run awaits an LLM or I/O call, the others make progress, so total wall-clock time approaches the slowest single run, not the sum of all of them. The catch is that this only works inside one asyncio.run(main()) wrapping the gather — two separate asyncio.run() calls do <em>not</em> overlap, because each one spins up and tears down its own event loop, so the second only starts once the first has fully finished. Each run keeps its own independent state (no shared blackboard between them), so they don&#39;t interfere with each other either.</p><h3>Step 6: Making the caching actually work</h3><p>Once I had multiple runs going, cost started to matter, which led me into prompt caching — and this is the part where I initially got the mental model wrong.</p><p>Caching is a <strong>prefix match</strong>. The API hashes the rendered prompt — tools, then system, then messages, in that order — up to each cache_control breakpoint. That means stable content (a frozen system prompt, fixed tool definitions) needs to go first, and volatile content (the topic, a growing message history) after it. Any byte-level change anywhere in that prefix invalidates the cache from that point forward.</p><p>Input tokens actually split into three price tiers: cache_creation (written to cache, 1.25× cost), cache_read (served from cache on reuse, 0.1× cost — this is the actual savings), and plain input (the uncached, new part, full price).</p><p>The first surprise was the <strong>cache floor</strong>: prefixes below the model’s minimum token count silently don’t cache at all — 1,024 tokens for Sonnet 4.6, the model used here (the floor varies by model, not just by tier name, so check the current docs rather than assume). No error, just cache_creation: 0. My first prefix was around 1.3K tokens, so nothing was caching and I had no idea why until I checked the raw numbers. The fix was padding the system prompt past the floor.</p><p>The second surprise was realizing what a breakpoint actually <em>is</em>: a marker on one content block meaning “cache everything from the start of the prompt up to and including this block” — not just that block in isolation. Put the breakpoint on the last system content block and you cache the tools <em>and</em> the system prompt together, since tools render first in the prefix. {&quot;type&quot;: &quot;ephemeral&quot;} is the only breakpoint type (5-minute TTL by default; add &quot;ttl&quot;: &quot;1h&quot; for an hour), and you get a maximum of four breakpoints per request.</p><p>The third thing that tripped me up: a plain Python string has nowhere to attach cache_control, because cache_control is a key on a content-block <em>dict</em>, and a string has no keys at all.</p><pre>system_prompt = &quot;You are a travel research agent...&quot;   # string: no keys<br><br>content = [{                                            # dict: has keys<br>    &quot;type&quot;: &quot;text&quot;,<br>    &quot;text&quot;: &quot;You are a travel research agent...&quot;,<br>    &quot;cache_control&quot;: {&quot;type&quot;: &quot;ephemeral&quot;},            # &lt;- attaches here<br>}]</pre><p>So the fix is upgrading the bare string into a {&quot;type&quot;: &quot;text&quot;, &quot;text&quot;: ...} block, which gives cache_control somewhere to live. None of this happens automatically through LangChain — you have to pass the system prompt as a SystemMessage whose content is that block, and langchain-anthropic forwards it as the breakpoint. The cached prefix then gets re-read on every ReAct turn within one call, and — with concurrent gather — by later runs that start after the first one has already written the cache.</p><p>The way I convinced myself it was actually working was watching cache_read: if it&#39;s always zero on requests with an identical prefix, something is silently invalidating the cache — a timestamp or UUID baked into the prompt, a sub-floor prefix, or tool definitions rendering in a different order between calls. My own trace eventually hit cache_read: 1868 on a later turn, proving the ~2K prefix was being reused at 0.1× cost. (LangChain folds the very first write into plain input, so you can&#39;t see the write directly — but a later cache_read proves it happened, since you can&#39;t read from a cache that was never written.)</p><p>There’s a second layer to this if the message history is growing turn over turn, which is a real defense I saw matter on IndicationScout as context grew: <strong>two breakpoints in one request.</strong> The first sits on the system block (the fixed tools+system prefix); the second sits on the <em>last message</em> each turn (the history tail). Each turn, everything up to message N-1 gets read from cache, only the new message N is billed at full price, and then the extended prefix is written for the next turn. Because the cache is a prefix, each turn’s longer entry shares its start with the previous turn’s entry, so that shared part is the cheap read. The breakpoint has to be applied to a <em>copy</em> of the last message, not the stored one, or stale interior breakpoints drift through history — and the history has to stay byte-identical, since editing anything earlier breaks the read from that point on.</p><p>Having built the manual version, I’ll say plainly: in practice, use the built-in middleware instead of hand-rolling any of this.</p><pre>from langchain_anthropic.middleware import AnthropicPromptCachingMiddleware<br><br>agent = create_agent(<br>    model=llm, tools=[...], system_prompt=SYSTEM_PROMPT,  # plain string<br>    middleware=[AnthropicPromptCachingMiddleware()],<br>)</pre><p>It injects both breakpoints — the system+tools prefix and the per-turn history tail — and manages the transient tail marker itself, with no SystemMessage-block wrapping and no pre_model_hook needed. The one caveat that survives regardless of the middleware: it still can&#39;t beat the model&#39;s token floor. If your prefix is under that floor (1,024 tokens on Sonnet 4.6), the marker gets set but the API still won&#39;t cache, so prompt-padding is still sometimes necessary even with the middleware doing the rest.</p><h3>Step 7: Making the blackboard durable</h3><p>The last piece I wanted to understand was what happens to all this state if the process dies mid-run — which is where LangGraph’s checkpointing comes in.</p><p>builder.compile(checkpointer=...) saves every node&#39;s write, keyed by a thread_id. You read state back via graph.aget_state(config) for the latest snapshot, or aget_state_history(config) for all of them, newest first — each snapshot carries .values, .next, and a resumable checkpoint_id. (Don&#39;t try to read the underlying .sqlite file directly; it&#39;s binary msgpack, not something you can eyeball.)</p><p>Two checkpointer options matter for different situations. MemorySaver lives entirely in RAM — compile it once at import time, and it survives for the life of the process but is gone on restart. A baseline run through my pipeline produced four checkpoints: __start__ → research → write → done. AsyncSqliteSaver is the durable option — it owns an actual DB connection, so it has to live inside async with ...from_conn_string(path) as cp:, and it writes a real binary .sqlite file (plus -wal/-shm companions).</p><p>One thing worth getting right from the start: compile once, not per call. Putting the saver’s async with block inside the function that runs the graph re-opens the database and re-compiles the graph on every single invocation. Instead, open the saver and compile the graph once for the whole application&#39;s lifetime — a lifespan() context manager mapping onto server startup/shutdown (FastAPI, for example) — and reuse that one compiled graph everywhere. MemorySaver sidesteps this problem entirely since it&#39;s just in-RAM state behind a module-level singleton.</p><p>The gotcha that actually caught me: with durable storage and the <em>same</em> thread_id, each run <strong>appends</strong> rather than replaces. History ends up showing every past run&#39;s full __start__ → ... → done family stacked on top of each other — I once found eight checkpoints sitting there, which turned out to be two separate runs with two different final stories, not one run gone wrong. The newest (top) snapshot is always the true current state; you tell separate run families apart by the timestamp prefix baked into each checkpoint_id. For a genuinely clean single run, use a fresh thread_id each time, or delete the .sqlite file between runs.</p><p>That’s the climb so far — four blackboard variants, two ways of passing data between agents, a working mental model of when async actually helps, and prompt caching and checkpointing understood from first principles rather than copy-pasted. None of it required deep LangGraph internals knowledge going in; it required hitting each wall once, slowing down to find out <em>why</em>, and writing down the answer before moving to the next one.</p><p>GitHub Repo: <a href="https://github.com/dami-gupta-git/assistant_agents">https://github.com/dami-gupta-git/assistant_agents</a></p><p><a href="https://www.linkedin.com/in/dami-gupta/">Dami Gupta</a></p><p>dami.gupta@gmail.com</p><p><em>Written with the help of Claude</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7a480fca3eb9" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/learning-langgraph-the-hard-way-a-journey-through-agents-blackboards-and-bottlenecks-7a480fca3eb9">Learning LangGraph : A Journey Through Agents, Blackboards, and Bottlenecks</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lost Between Sessions ? Engineering Persistent Intelligence in Claude Code]]></title>
            <link>https://pub.towardsai.net/the-forgetting-problem-engineering-persistent-intelligence-in-claude-code-bd2e4c59711a?source=rss-4687bd7ed107------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd2e4c59711a</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[memory-management]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[semantics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Dami Gupta]]></dc:creator>
            <pubDate>Sat, 21 Feb 2026 01:35:25 GMT</pubDate>
            <atom:updated>2026-03-09T16:04:20.074Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>The Forgetting Problem : Engineering Persistence in Claude Code</strong></h3><p>Solving Claude Code’s Forgetfulness Problem</p><h4>If you use Claude Code seriously, you eventually get burned by its forgetfulness, the dreaded “amnesia”. What can we do about it?</h4><p>Stateless AI doesn’t need better memory. It needs better constraints.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/411/1*J_DIkNR4_-pGd7naAtmyMg.png" /></figure><p><strong>Beyond Memory Files</strong></p><p>Most advice is operational: create a CLAUDE.md, structure prompts, summarize periodically. That helps. But it doesn’t solve the architectural problem : how do you keep a system coherent across sessions when the model has no persistent state?</p><p><strong>The Illusion of Continuity</strong></p><p>Full disclosure: I love Claude Code.</p><p>It’s well known that Claude Code works in “amnesiac bursts.” Each session starts fresh, rebuilding understanding from whatever artifacts previous sessions left behind. The model doesn’t remember your codebase. It re-comprehends it, every time. And honestly, that’s not a bug. Fresh eyes catch things that accumulated context would miss.</p><p>There are two kinds of forgetting here. Within a session, earlier context drops out as the window fills — the usual problem that summarization and structured prompts try to fix. Across sessions, each one starts from scratch, rebuilding understanding from your notes. That’s not drift so much as twelve engineers reading the same docs and walking away with slightly different takes. And compacting just adds another dimension.</p><p>The real failure comes when these versions quietly diverge — what started as one design pattern becomes conflicting architectures by session twelve, and no single session notices the split.</p><p>The good news: this problem can be mitigated with the right ‘scaffolding’.</p><p><strong>Three Patterns That Help</strong></p><p><strong>1. The Protocol Layer</strong></p><p>Instead of describing constraints in prose, encode them as something that looks more like a spec:</p><pre># .claude/protocols/variant-classification.md<br>## PROTOCOL: Tier Classification Logic<br><br>STATUS: LOCKED<br><br>AUTHORITY: Clinical validation complete 2024–11<br><br>INVARIANTS:<br>- Tier I requires: therapeutic_evidence &gt;= &quot;Level A&quot; AND actionability = true<br>- LLM synthesis NEVER overrides rule-based tier assignment<br>- All edge cases route to human review, not LLM judgment<br><br>VIOLATION_RESPONSE: Halt and surface conflict explicitly</pre><p>The model treats “this is locked” differently than “please try to remember this.” Give it structure and it will respect structure. This isn’t documentation. It’s governance.</p><p><strong>On governance:</strong> only designated owners can flip that LOCKED status. Declare ownership in .claude/OWNERS, require PR review for changes, log everything.</p><p>I learned this the hard way in an app I was developing, <a href="https://github.com/dami-gupta-git/tumor_board_v2">TumorBoard</a> (a hybrid rules-and-LLM engine for variant classification). Early on, tier assignments were strictly deterministic — the LLM could explain them, never determine them. That boundary was deliberate.</p><p>Then, several sessions later, Claude Code suggested consolidating logic into a single synthesis step. Looked reasonable. Passed tests. Didn’t think much of it.</p><p>Weeks later I noticed the same input producing different tier assignments depending on how I phrased the context. The architectural boundary had quietly moved. The original constraint lived only in my head, so the system gradually optimized it away.</p><p>A locked protocol file fixed it. The model stopped proposing that class of change — not because it got smarter, but because the architecture stopped relying on memory.</p><p><strong>2. Decision Archaeology</strong></p><p>Keep a log of important architectural decisions. Not comments in code (those rot). But standalone files the model can consult:</p><pre># .claude/decisions/2024–11-ensemble-rejected.md<br>DECISION: Single-model architecture over ensemble<br>DATE: 2024–11–15<br>OUTCOME: Rejected<br><br>REASONING:<br>- Latency incompatible with sequential model calls<br>- Disagreement resolution introduced more errors than it caught<br>- 89% accuracy achieved with hybrid rules approach<br><br>IMPLICATIONS:<br>- Do not propose ensemble architectures for classification tasks</pre><p>Next time the model’s about to suggest an ensemble, this file gives it the documented memory of why you already tried that. It’s not that the model can’t reason well. It’s that it shouldn’t have to re-derive your entire decision history from scratch.</p><p><strong>3. The Watchdog Pattern</strong></p><p>Here’s a fun one: use Claude Code to audit its own drift. Run a periodic session that’s purely adversarial:</p><pre>Review the current codebase against .claude/architecture-intent.md. <br>Identify divergences between stated intent and implemented reality. <br>Do not fix them. Document them.</pre><p>Identify divergences between stated intent and implemented reality.</p><p>Do not fix them. Document them.</p><p>The model’s fresh perspective (its amnesia, weaponized for good) can spot drift that accumulated gradually across sessions. Each new session is a free code review from someone with no prior assumptions.</p><p><em>Quick aside</em>: Claude Code has built-in memory now — CLAUDE.md, auto memory, the whole thing . It’s great for keeping track of your preferences and habits. But that’s more onboarding than governance. It won’t notice when the model starts drifting off or happily suggests that same ensemble setup you already spent a week shooting down.</p><p><strong>Case Study: OncoMind</strong></p><p>I have been working on a project : <a href="https://huggingface.co/spaces/damigupta/onco_mind">OncoMind</a>, it is a cancer research platform that identifies evidence gaps in variant knowledge. Its architecture shows what “complementary strengths” actually looks like in practice.</p><p>The evidence layer is all rules: aggregation from 8+ databases, match specificity tracking, gap detection with severity scoring. Well-defined transformations, clear correctness criteria. Rules work great here because the logic is testable, auditable, and identical every run.</p><p>The LLM comes in where rules would be painful to write and maintain: synthesizing across messy evidence, generating research hypotheses, writing narratives. Rules handle the “what.” The LLM handles the “so what”.</p><p>The boundary is structural, not just documented. Deterministic modules produce evidence objects; LLM modules consume them. When Claude Code works on either side, responsibilities are obvious.</p><p>The lesson: encode architectural boundaries in code structure, not just docs. Structure survives context rebuilding better than convention. And when you give the model clear boundaries, it operates brilliantly within them.</p><p><strong>Lock it Down: Pick What Fits</strong></p><p>Most teams won’t do all of this. That’s fine. Grab what’s useful.</p><p><strong>Hook protocols into CI.</strong> A shell script that fails the build beats a perfectly documented constraint nobody checks.</p><p><strong>Log which context files loaded.</strong> When something weird ships, ask: did the session even see the relevant protocol? Often it didn’t.</p><p><strong>Link decisions to code.</strong> Reference decision files in PRs that touch their domain. Keeps the same rejected idea from getting re-proposed by sessions that never saw why you rejected it.</p><p><strong>Track structural drift.</strong> Snapshot your codebase structure periodically, compare against a known-good baseline. You’ll see the architecture morphing before you feel it in bugs.</p><p><strong>Flag stale decisions.</strong> If code’s changing but the decision file hasn’t been touched in months, maybe reality moved on.</p><p><strong>Audit random sessions.</strong> Same context, same prompt: do you get similar output? If not, your documented context has blind spots.</p><p><strong>Final Thoughts</strong></p><p>What we’re really doing is serializing cognitive state (reasoning patterns, decision frameworks, institutional memory) into forms a stateless model can reload.</p><p>That’s genuinely hard. Experts don’t think in serializable formats. The oncologist who’s been classifying variants for twenty years has intuitions they can’t fully articulate. Context files are lossy compression of human expertise.</p><p>The hybrid approach helps: don’t try to serialize everything. Use rules for what can be made explicit. Let the LLM do what it’s genuinely great at, which is synthesis, explanation, and handling the messy cases that would take forever to write down as rules.</p><p>The goal isn’t to make Claude Code remember. It’s to make remembering unnecessary. Encode intent so clearly that any session, starting fresh, lands in the same place and builds from there.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd2e4c59711a" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/the-forgetting-problem-engineering-persistent-intelligence-in-claude-code-bd2e4c59711a">Lost Between Sessions ? Engineering Persistent Intelligence in Claude Code</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can your AI Agent Follow Complex Rules ?]]></title>
            <link>https://pub.towardsai.net/your-ai-agent-got-it-right-but-did-it-reason-right-c9a8ad875f8c?source=rss-4687bd7ed107------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9a8ad875f8c</guid>
            <category><![CDATA[bioinformatics]]></category>
            <category><![CDATA[ai-testing]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <dc:creator><![CDATA[Dami Gupta]]></dc:creator>
            <pubDate>Sun, 15 Feb 2026 17:48:01 GMT</pubDate>
            <atom:updated>2026-02-23T03:22:45.500Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Can Your AI Agent Follow Complex Rules?</strong></h3><p><em>How knowledge graphs catch flawed reasoning in biological applications.</em></p><p>Agentic AI has matured rapidly. The opportunity now is going deeper — testing not just whether agents get the right answer, but whether they reasoned well to get there.</p><p>An agent can reach a correct conclusion through incomplete or flawed reasoning. It can skip critical considerations, ignore important trade-offs, and still land on the right output. Current testing frameworks: benchmarks, trajectory matching, LLM-as-judge are evolving rapidly, but none yet address this directly.</p><p>In high-stakes domains, “right answer, incomplete reasoning” is a hidden risk. The agent that succeeded without thorough reasoning will eventually encounter a case where the shortcuts do not work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/246/1*3m8SnnVHUtZ75GBbjwE5uQ.png" /></figure><p>I first hit this problem building a <a href="https://github.com/dami-gupta-git/tumor_board_v2">hybrid rules-and-LLM engine</a> for variant classification. It kept breaking on edge cases — missed combinations, invalid paths. And the LLM was not able to parse the connections. I started thinking about this and wrote <a href="https://medium.com/towards-explainable-ai/can-an-llm-know-that-it-knows-7dc6785d0a19">LLM Reasoning and Recall</a>. And I decided that we needed strict safeguards and validation systems to cover complex systems.</p><h3>The Structure of Real Decisions</h3><p>Here is something easy to miss: most real-world decisions have graph-like logic. Not simple yes/no checks, but conditional paths with sequences, required combinations, and exclusions. Customer service, loan approval, legal research, medical diagnosis, code review — the logic is always more complex than a checklist.</p><p>For example :</p><ul><li>Some conclusions require multiple pieces of evidence together (AND)</li><li>Some conclusions can be reached through alternative paths (OR)</li><li>Some paths are only valid when certain conditions are absent (NOT)</li><li>Some steps must come before others (sequences)</li><li>Sometimes you need to revisit earlier steps (loops)</li></ul><p>Every domain where agents make decisions has this structure.</p><p>The difference is whether the rules are written down.</p><p><em>Note: In practice, typically systems have multiple agents. The reasoning path would span them all. For simplicity, we will focus on validating a single agent’s reasoning here, but the same graph structure applies to the combined path across agents.</em></p><h3>Where Current Approaches Leave a Gap</h3><p>Existing evaluation methods are valuable, and they are improving. But they are optimized for different questions.</p><p><strong>Benchmarks</strong> measure outcomes. Did the agent complete the task? This is necessary but not sufficient. A passing score does not tell you whether the agent followed valid reasoning paths to get there.</p><p><strong>Trajectory evaluation</strong> checks steps. Did the agent take the expected actions? But this does not tell you whether it satisfied required evidence combinations or respected exclusions.</p><p><strong>LLM-as-judge</strong> evaluates coherence. Is the reasoning internally consistent? But without a reference for what valid reasoning looks like in a given domain, a judge cannot identify what is missing or what invalid paths were taken.</p><p>These approaches complement each other. What is missing is a framework that asks: did the agent’s reasoning follow a valid path through the decision logic? And what if there are multiple valid paths, that intersect?</p><h3>The Knowledge Graph Concept</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*3TVTT5RmkRlAnDDCcGmfiA.jpeg" /></figure><p>A knowledge graph for validation encodes what valid reasoning looks like for a given decision type:</p><ul><li><strong>Required information</strong> — What must be considered</li><li><strong>Valid sequences</strong> — What order steps can occur in</li><li><strong>Required combinations (AND)</strong> — What must be true together</li><li><strong>Valid alternatives (OR)</strong> — Different legitimate paths to the same conclusion</li><li><strong>Exclusions (NOT)</strong> — Conditions that invalidate a path</li><li><strong>Loops</strong> — Where iteration and re-evaluation is legitimate</li></ul><p>The graph is built independently of the agent. It represents what a domain expert would expect from valid reasoning. It constitutes the space of all valid paths an agent can take.</p><h3>Why a Graph, Not a Checklist</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/352/1*_BkOZOQD0s1Ivpx-8NHfzw.jpeg" /></figure><p>A checklist says: “Did you consider these things?”</p><p>A knowledge graph says: “Did you reason through these things in valid combinations and sequences?”</p><p>The difference matters. A checklist is flat, items to tick off. A graph encodes structure.</p><p>Consider a simple example: “near a gene” might not be enough to support a conclusion. “In a hotspot region” might not be enough either. But “near a gene AND in a hotspot” together might be sufficient. The checklist approach (did you check gene proximity? did you check hotspot status?) misses that both are required together.</p><p>Or consider exclusions: a reasoning path might be valid only when certain conditions are absent. “This evidence supports the conclusion, but NOT when this other condition is present.”</p><p>Agents face this complexity constantly. The question is whether we are testing for it.</p><h3>A Genomics Use Case</h3><p>The AMP/ASCO/CAP guidelines define how genetic variants should be classified. They are not simple rules — they are complex conditional logic with exactly the structure described above:</p><p><strong>Conjunctions (AND):</strong> Classification requires evidence combinations: functional data AND population frequency AND clinical observations. No single evidence type is sufficient alone.</p><p><strong>Disjunctions (OR):</strong> Tier I (strong clinical significance) requires FDA-approved therapies OR professional guideline consensus. Either path is valid.</p><p><strong>Exclusions (NOT):</strong> Certain evidence only counts in specific contexts. Evidence from solid tumor cell lines might not apply when classifying variants for hematologic malignancies.</p><p><strong>Sequences:</strong> You need to establish what variant you are examining before you can assess its population frequency. You need frequency data before you can weigh it against clinical observations.</p><p><strong>Context dependencies:</strong> The same variant type can be pathogenic in one gene and tolerated in another. The reasoning path depends on context.</p><p>These guidelines are essentially a knowledge graph. They encode which combinations of evidence, in what relationships, support which classifications.</p><h3>Real Agents in This Space</h3><p>This matters because AI agents are being deployed in genomics right now.</p><p>GeneAgent (published in Nature Methods) analyzes gene sets by querying biological databases. Stanford’s Biomni handles variant prioritization and rare disease diagnosis. GPT-4 with retrieval-augmented generation is being used to interpret millions of variant annotations.</p><p>These agents make classification decisions. The question is: are they following valid reasoning paths?</p><p>An agent might classify a variant as pathogenic. But did it:</p><ul><li>Check the required evidence combinations?</li><li>Follow valid sequences?</li><li>Respect the exclusions?</li><li>Take a path that actually exists in the guideline logic?</li></ul><p>Or did it skip steps, ignore required conjunctions, and arrive at the right answer through an invalid path?</p><p>Current evaluation cannot tell you. A knowledge graph can.</p><h3>Example: Valid and Invalid Paths</h3><p><strong>Valid path 1:</strong> Variant identified → Check population frequency (absent from databases) → Check functional evidence (strong impact demonstrated) → Check clinical observations (seen in affected patients, absent in controls) → Classification supported</p><p><strong>Valid path 2:</strong> Variant identified → Check existing expert classifications (ClinVar pathogenic, multiple submitters, review status high) → Verify underlying evidence quality → Classification supported</p><p><strong>Invalid path:</strong> Variant identified → Run computational prediction → Prediction says damaging → Classify as pathogenic</p><p>The third path is invalid because computational predictions alone do not support pathogenic classification under the guidelines. An agent taking this path might get the right answer by luck, but its reasoning does not follow a valid path through the decision logic.</p><p><strong>Conjunction example:</strong> “Near oncogene” alone insufficient. “In hotspot” alone insufficient. Both together support pathogenicity. An agent that checks both but misses they are required together has ticked the checklist but missed the logic.</p><p><strong>Exclusion example:</strong> Functional evidence supports pathogenicity, but NOT when the assay used an unrelated cell type. An agent that cites evidence without checking exclusions has followed an invalid path.</p><h3>Building the Graph</h3><p>Building knowledge graphs is labor intensive.</p><p>You build the tree once for a class of decisions. Then you validate many agent executions against it. It’s a deliberate investment for decision types that matter — where “right for the wrong reasons” is dangerous.</p><p>You do not auto-generate integration tests for critical systems. Engineers who understand the domain write them to reflect correct behavior. Knowledge graphs work the same way: domain experts define what valid reasoning requires. (Of course, there will be some automation, but some of it will have to be curated).</p><p>In regulated domains like clinical genomics, some of the workflow is already done. The AMP/ASCO/CAP guidelines, ClinGen specifications, and ACMG criteria are detailed reasoning frameworks. The task is encoding them in a structure you can validate agents against.</p><p>For domains without formalized guidelines, the work is harder but the logic still exists. Experts know what valid reasoning looks like. The knowledge graph makes it explicit and testable.</p><h3>How Validation Works</h3><ol><li>Encode the domain’s reasoning rules as a knowledge graph</li><li>Run the agent on test cases</li><li>Examine what knowledge the agent used, what sequence it followed, what conclusion it reached</li><li>Check whether the agent’s reasoning corresponds to a valid path in the graph</li><li>Score on path validity, conjunction satisfaction, and exclusion compliance</li><li>Flag cases where conclusions were reached through invalid paths</li></ol><p>The output is not pass/fail. It is a map showing where the agent’s reasoning aligned with valid paths and where it diverged. Think: fully graph-valid, graph-aligned but incomplete, or off-graph.</p><h3>How to Start</h3><p>Begin small and concrete. Choose one tightly scoped decision (for example, BRCA1 missense classification), encode a compact knowledge graph of 6–12 nodes capturing the key evidence types, and curate about a dozen representative test cases. Instrument the agent to emit structured traces (evidence items, retrieval citations, step sequence), run the pilot, and triage divergences through human review. Publish the graph, test cases, and evaluation results to invite community critique and iterative improvement.</p><h3>Open Questions</h3><p><strong>Granularity.</strong> How detailed should the graph be? Too coarse and it misses important distinctions. Too fine and it becomes brittle.</p><p><strong>Novel paths.</strong> What if an agent takes a valid path the graph did not anticipate? Domain knowledge evolves. The framework needs room for expert review.</p><p><strong>Maintenance.</strong> Guidelines change. Graphs need versioning and periodic review.</p><h3>Related Work</h3><p>Similar thinking is emerging. <a href="https://engineering.salesforce.com/agentforces-agent-graph-toward-guided-determinism-with-hybrid-reasoning/">Salesforce’s Agentforce</a> uses a “Graph Runtime” to test whether agents respect business constraints. <a href="https://zitniklab.hms.harvard.edu/projects/KGARevion/">Harvard’s Zitnik Lab developed KGARevion</a>, a knowledge graph-based agent for medical QA that validates reasoning against structured knowledge. Recent work on <a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC12748213/">self-correcting Agentic Graph RAG</a> in clinical hepatology demonstrates validate-execute-evaluate loops for reasoning paths. <a href="https://arxiv.org/html/2410.14211v4">Paths-over-Graph</a> shows how knowledge graph paths can improve LLM reasoning faithfulness by 18.9% through structured path validation.</p><p>These approaches share the insight that structured references help validate agents. But they focus on different questions: Did the agent violate a constraint? Did it contradict known facts?</p><p>The knowledge graph concept addresses something complementary: did the agent follow valid reasoning paths? Did it satisfy required evidence combinations? That is the gap between getting the right answer and actually reasoning correctly.</p><h3>Why This Matters</h3><p>Every domain where agents make decisions has graph-like reasoning logic: sequences, required combinations, exclusions, alternative valid paths.</p><p>Right now, we mostly test outcomes. Did the agent get the right answer? That is necessary but not sufficient.</p><p>Knowledge graphs let us test reasoning. Did the agent follow a valid path? Did it satisfy the required conjunctions? Did it respect the exclusions?</p><p>An agent that gets the right answer through invalid reasoning is a liability. It will fail when the shortcuts stop working.</p><p>Outcomes tell you what happened. Reasoning tells you whether it is repeatable.</p><p><em>Dami is a Software Engineer and MS Data Science candidate at Boston University. She has many years of experience working in the bio-pharmaceutical industry.</em></p><p><em>If you are working on related problems in agentic AI validation, or AI-driven scientific discovery, I would love to connect.</em></p><p><strong>SAMPLE CODE</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/26c013bf7bfed0c7d346ca117d530eff/href">https://medium.com/media/26c013bf7bfed0c7d346ca117d530eff/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9a8ad875f8c" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/your-ai-agent-got-it-right-but-did-it-reason-right-c9a8ad875f8c">Can your AI Agent Follow Complex Rules ?</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reasoning or Recall? Solving AI’s “Silent Knowledge” Problem]]></title>
            <link>https://medium.com/towards-explainable-ai/can-an-llm-know-that-it-knows-7dc6785d0a19?source=rss-4687bd7ed107------2</link>
            <guid isPermaLink="false">https://medium.com/p/7dc6785d0a19</guid>
            <category><![CDATA[drug-discovery]]></category>
            <category><![CDATA[interpretability]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[scientific-discovery]]></category>
            <dc:creator><![CDATA[Dami Gupta]]></dc:creator>
            <pubDate>Wed, 04 Feb 2026 05:40:38 GMT</pubDate>
            <atom:updated>2026-06-02T00:34:51.608Z</atom:updated>
            <content:encoded><![CDATA[<p>Auditing the “Black Box” of Biological LLMs</p><h3>A question about AI-driven scientific discovery</h3><p><em>When an LLM arrives at a correct insight, did it actually reason its way there, or just recall the answer from its training data? This is an open problem for AI-driven scientific discovery, and I examine ways to address it.</em></p><p>Can an LLM look at biological data and generate meaningful scientific insights? I have been playing with some ideas at this intersection: using <a href="https://github.com/dami-gupta-git/esm2_mechanism">protein language model embeddings</a> to analyze genetic variants, <a href="https://github.com/dami-gupta-git/indication_scout">multi-agent architectures for drug repurposing</a>, and synthesizing evidence across genomic databases. (e.g. <a href="https://huggingface.co/spaces/damigupta/onco_mind"><em>Cancer Research Copilot</em></a>). In every case, I hit the same wall.</p><p>Here is an example. CDK4 is a gene that, when overactive, can drive uncontrolled cell growth, one of the hallmarks of cancer. RB1 is the brake that normally keeps it in check. When RB1 is lost, CDK4 runs unchecked — this is a well-documented pattern in aggressive breast cancers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*_tplfwXycm-xvJyAWebE_g@2x.jpeg" /></figure><p>A researcher looking at genomic data from breast cancer tumors notices that CDK4 levels are abnormally high. Beyond just expression levels, she draws on several layers of evidence (pathway biology, mutation patterns, clinical data), all pointing her to suspect RB1 could be a major player. She checks for RB1. It’s gone. The brake is missing. That explains the pattern. This is biological reasoning: you see something unexpected, trace it to its cause, and confirm.</p><p>I wanted to build an LLM that could do the same thing: look at raw data and arrive at insights like this RB1/CDK4 connection. Not because it read a paper that told it the answer, but because it reasoned its way there.</p><p>So you give the LLM the same data. It notices the CDK4 abnormality. It says “let me check RB1,” and finds it’s lost, and announces a connection. It looks like it just did exactly what the researcher did.</p><p>Except the LLM was trained on thousands of papers about RB1 loss in breast cancer. It already knew to look at RB1. It did not discover the connection; it recalled it while performing the outward motions of reasoning. And I had no easy way to tell the difference.</p><p>Current AI capabilities solve for output quality — is the answer correct? But rarely for <strong>epistemic provenance</strong> — where did this knowledge originate? A model can produce a perfectly accurate answer while being unable to identify whether it derived from input data or training weights.</p><p>Another example — I gave an LLM structured annotation data about the ALK F1174L mutation and asked it to write a synthesis. It produced: “ALK F1174L is a known acquired resistance mutation.” The word “acquired” appeared nowhere in my input — the model wove in training-derived knowledge without any signal it had done so. When asked to cite sources, it attributed the claim to the databases I had provided. Incorrectly. I am calling this <strong>silent knowledge contamination</strong>.</p><p>The difficulty of distinguishing reasoning from retrieval is actually what got me excited. I started sketching out a scientific reasoning engine: a system that would treat surprise as the driver of discovery, so that when data contradicts the model’s expectation, it follows the lead instead of explaining it away. At 1 am, this felt like a new research direction.</p><p>The next morning, I found AutoDS.</p><p>AI2 published this in July 2025 — a system for open-ended autonomous scientific discovery that explores hypothesis space automatically. Google’s AI Co-Scientist had launched in February 2025, already generating novel hypotheses and validating them in wet labs. FutureHouse spun out Edison Scientific in November 2025 to commercialize their AI Scientist paradigm. (Note: A few days ago, AI2 released Theorizer.)</p><p>I was one person in a master’s program. They were full research teams with compute budgets I’ll never touch.</p><p>But as I sat with the disappointment, I noticed something. My idea and their systems overlapped, but I was fixating on a specific piece that still feels unresolved. They are tackling how to explore hypothesis space at scale. I was stuck on a different, related question: <strong>when an LLM reasons about scientific data, can it distinguish what it figured out from what it already knew?</strong></p><h3>A Problem Worth Solving Directly</h3><p>This is a real problem for any AI system in scientific discovery. If the model cannot distinguish reasoning from retrieval, then we can’t distinguish genuine discovery from sophisticated pattern matching against memorized literature.</p><p>I think there are three levels to this problem, and they get progressively harder:</p><p><strong>Level 1: Does the LLM know X?</strong> This is straightforward. Just ask it. Benchmark it. We have hundreds of papers on this.</p><p><strong>Level 2: Does the LLM know how it knows X?</strong> Did it reason from the data I provided, or retrieve from training? Recent research suggests models may have some internal signal for which mode they’re in (Hong et al., 2025), but that’s a finding about model internals, not something the model itself can access or report.</p><p><strong>Level 3: Can the LLM constrain itself to only use what it should know?</strong> Can you give it data and have it reason purely from that data, flagging when it’s about to use prior knowledge? This is what you would need for genuine AI-driven scientific discovery, and I haven’t seen it demonstrated rigorously.</p><h3>Why The Obvious Solutions Do not Work</h3><p>My first thought was: just use retrieval-augmented generation (RAG). Feed it only the provided data, let it retrieve and reason from those chunks, and trace claims back to explicit documents. It should block silent contamination from training.</p><p>But parametric knowledge still leaks: it biases query formulation (e.g., “high CDK4” nudges retrieval toward RB1), interpretation of chunks, and structural intuitions that never appear verbatim in context. RAG improves grounding and citations, but the reasoning path remains influenced by memorized biology — provenance is partial, not pure. The model performs grounded reasoning while smuggling in training intuition. For genuine discovery, RAG mitigates but doesn’t solve the separability of recall and reasoning.</p><p>My next thought was: just find a model that doesn’t know biology. Use a code-only model, one trained purely on programming with no biomedical knowledge. Give it biological data and see what it can figure out from scratch.</p><p>The problem is that a model with zero biology won’t even understand what it’s looking at. You need some biological literacy to do biological reasoning. The question isn’t knowledge versus no knowledge; it’s whether the model can use its reasoning capabilities while holding its factual knowledge at arm’s length.</p><p>Then I thought about prompt engineering: just instruct the model to only reason from the data provided. This works on the surface. But knowledge leaks invisibly. The model doesn’t explicitly say “RB1 is a tumor suppressor.” It just happens to suggest checking genes that a knowledgeable human would check, for reasons it frames as logical deduction.</p><p>Chain-of-thought prompting and process supervision get closer. Systems like OpenAI’s o1 force the model to reason step by step, and the reasoning can be genuine. But within any chain, you still can’t tell which steps were driven by the data and which were informed by training knowledge. Web grounding cites external URLs but doesn’t distinguish input-derived claims from training-derived ones. And self-reported citations? The ALK example shows models confabulate sources.</p><p>These capabilities improve transparency — <em>how</em> the model reasons. They don’t address provenance — <em>where</em> its knowledge comes from.</p><p>While we have tools to detect verbatim leakage of training text, we can’t detect the leakage of intuition — where a model uses its prior knowledge to nudge its reasoning toward a known conclusion.</p><h3>Three Ways to Test This (And Why They’re Hard)</h3><p><strong>Machine unlearning: </strong>What if you could make a model forget its biology? Then any biological insight it produces must have come from reasoning, not memory. The problem is that knowledge in LLMs is tangled across billions of parameters. You can’t cleanly remove “RB1 is a tumor suppressor” without damaging related knowledge about cell biology generally.</p><p>But maybe that imperfect forgetting is actually useful. If the model leaks knowledge it was supposed to have forgotten, the leakage itself tells you something about how knowledge and reasoning interact.</p><p><em>Methods: Fine-tuning to forget, gradient ascent, surgical model editing using </em><a href="https://arxiv.org/abs/2202.05262"><em>ROME</em></a><em> and </em><a href="https://arxiv.org/abs/2210.07229"><em>MEMIT</em></a><em>.</em></p><p><strong>Selective distillation : </strong>Instead of removing knowledge from an existing model, build a new smaller model and control what it learns. Teach it only reasoning (logic, pattern recognition, data interpretation) while withholding biomedical knowledge.</p><p>But reasoning and knowledge aren’t cleanly separable. When a teacher model demonstrates how to reason, the reasoning patterns themselves carry domain knowledge implicitly. The distillation ends up as a spectrum:</p><ul><li><strong>Explicit knowledge</strong> (“RB1 is a tumor suppressor”): easy to exclude</li><li><strong>Structural knowledge</strong> (“tumor suppressors regulate cell cycle checkpoints”): harder to exclude, embedded in how concepts relate</li><li><strong>Reasoning patterns</strong> (“when something is too high, look for what normally keeps it in check”): impossible to classify. Is that domain knowledge, or just general logic?</li></ul><p>I have’found an example of anyone building a deliberately ignorant model this way, because nobody wanted one. I do. Not as a product, but as an experiment.</p><p><em>Note: Kim et al. (2025) found that distilling reasoning without new knowledge boosts accuracy but at the expense of capability on the hardest problems — the student learns surface patterns rather than deep reasoning.</em></p><p><strong>Neural ablation : </strong>The neuroscience-inspired approach: identify the neurons responsible for biomedical knowledge and surgically remove them, like a lesion in a living brain. Recent work shows you can selectively prune specific capabilities while leaving others intact (<a href="https://arxiv.org/abs/2403.01267">Pochinkov and Schoots, 2024</a>).</p><p>What’s fascinating: removing just 3 neurons out of 1.1 billion parameters can collapse a 72-billion parameter model’s performance entirely. You might go from “knows biology” straight to “can’t form coherent sentences” with nothing useful in between.</p><p>Each approach has limitations on its own. But combined, they form a toolkit: three different angles on the same question.</p><h3>A Better Experiment: Instrumented Training</h3><p>All three approaches above are post-hoc. They try to modify or study an already-trained model. But there’s a cleaner path.</p><p><strong>What if you trained a small model from scratch with the explicit goal of tracking where knowledge and reasoning emerge?</strong></p><p>Not 70B parameters. More like 100–300M. Small enough to instrument fully, big enough to learn real reasoning. You’d train it on a partitioned corpus:</p><ul><li><strong>Partition A (Pure reasoning)</strong>: logic puzzles, mathematical proofs, scientific method, causal inference, data interpretation</li><li><strong>Partition B (Biomedical knowledge)</strong>: textbooks, papers, pathways, variant databases</li><li><strong>Partition C (Mixed)</strong>: scientific papers where reasoning and biology are entangled</li></ul><p>Save dense checkpoints. Log weight updates. Then:</p><ol><li><strong>Compare A-only versus A+B weights.</strong> Which parameters change when you add biology to reasoning? Those are your knowledge parameters.</li><li><strong>Test at intermediate checkpoints.</strong> At what point during training does the model start “knowing” that RB1 is a tumor suppressor? At what point does it start being able to reason about tumor suppressors? Are those the same point?</li><li><strong>Examine the entangled weights.</strong> What lives in the parameters that updated on both reasoning and biology data? Is that where “biological intuition” gets encoded — the structural knowledge that isn’t quite a fact and isn’t quite logic?</li></ol><p>The infrastructure for this largely exists: <a href="https://www.eleuther.ai/artifacts/pythia">EleutherAI’s Pythia</a> project provides 143+ intermediate checkpoints for interpretability research. Methods for identifying “knowledge neurons” (<a href="https://arxiv.org/abs/2104.08696">Dai et al., 202</a>2) can pinpoint neurons storing specific facts. Anthropic’s circuit tracing methods can map computational pathways.</p><p>What I haven’t seen is the specific combination: training on domain-partitioned data with dense checkpointing to build a complete map of where knowledge versus reasoning lives in the model.</p><p>Instead of reverse-engineering a finished model, you’d be watching the knowledge form in real time.</p><h3>Why This Matters Beyond the Lab</h3><p>The question of whether LLMs can distinguish what they know from what they figured out has immediate practical implications:</p><p><strong>For AI-driven drug discovery:</strong> If we can’t verify that a model’s hypothesis came from reasoning about the data rather than memorizing the literature, we can’t trust its “discoveries.”</p><p><strong>For clinical decision support:</strong> A system that recommends treatment because it reasoned from a patient’s genomic profile is fundamentally different from one that recommends it because similar cases appeared in its training data. The first is personalized medicine. The second is a sophisticated lookup table. When an LLM synthesizes variant evidence, clinicians need to know: is this conclusion derived from the CIViC database entry I provided, or from a 2019 paper in the training set that may have been superseded?</p><p><strong>For scientific integrity:</strong> As AI systems increasingly generate hypotheses and design experiments, the scientific community needs tools to audit whether AI contributions represent genuine reasoning or sophisticated regurgitation.</p><p><strong>For interpretability and alignment:</strong> Understanding how knowledge and reasoning interact inside a model is a prerequisite for building systems we can trust with consequential decisions. The epistemic provenance question is a special case of the broader alignment question: can the model accurately report its own internal states?</p><h3>What I’m Proposing</h3><p>I’m proposing a research direction I’m calling <strong>epistemic provenance for LLMs</strong>: the study of whether a model can identify the source of its own output — training data or in-context reasoning.</p><p>The framework would have three components:</p><ol><li><strong>A small, instrumented model</strong> trained on partitioned data with dense checkpoint logging, creating a ground-truth map of where knowledge and reasoning live in the weights.</li><li><strong>A forensic toolkit</strong> combining unlearning, distillation, and ablation as complementary approaches to detecting when training knowledge contaminates reasoning.</li><li><strong>Domain expert evaluation</strong> to distinguish cases where a model’s reasoning chain includes steps that no one would take from first principles without prior knowledge (like checking RB1 in the example above). Human-in-the-loop evaluation isn’t new, but applying it specifically to the reasoning-vs-retrieval distinction in scientific domains is where existing approaches fall short.</li></ol><p>If this works, even a partial map of where knowledge and reasoning diverge would give AI discovery systems something they currently lack: a way to audit their own outputs. If it turns out that knowledge and reasoning are fundamentally inseparable in the weights, that finding matters too — it would force a reframing of what “AI-driven discovery” can actually claim.</p><p>Either way, this is tooling that scientific AI efforts are likely to need.</p><h3>Where This Leaves Me</h3><p>When I started this project, I wanted to build a system that could make scientific discoveries. What I realized instead is that the more interesting problem is one step upstream: how do you tell whether a model’s insight came from reasoning or recall?</p><p>I think this deserves a direct experimental answer.</p><p>Can an LLM truly reason its way to new insight, or is it only recalling from training? I don’t know yet. But I think it’s the right question to be asking.</p><p>The tool that started all this : <a href="https://huggingface.co/spaces/damigupta/onco_mind">Cancer Research Copilot — HuggingFace</a></p><p>Note: After this angst, I decided to find ways to validate what LLM is doing, and so came — <a href="https://medium.com/@dami.gupta/your-ai-agent-got-it-right-but-did-it-reason-right-c9a8ad875f8c">Your AI Agent Got It Right. But Did It Reason Right?</a></p><h3>Technical Appendix</h3><p>This appendix provides detailed methods and citations for readers interested in the technical foundations of the approaches described above.</p><p><strong>Epistemic Modes in LLMs:</strong> Hong et al. (2025) found linear features in the residual stream that mediate switching between reasoning and memorization modes, suggesting the model has some internal representation of which mode it’s in.</p><p><strong>Machine Unlearning Methods:</strong> Fine-tuning to forget, gradient ascent, and surgical model editing using ROME and MEMIT.</p><p><strong>Distillation Literature:</strong> Kim et al. (2025) found that distilling reasoning patterns without new knowledge boosts accuracy but at the expense of capability on hardest problems.</p><p><strong>Neural Ablation Studies:</strong> Pochinkov and Schoots (2024) demonstrated selective pruning. Research on critical neurons showed disrupting just 3 neurons out of 1.1 billion can collapse a 72-billion parameter model’s performance entirely.</p><p><strong>Instrumented Training Infrastructure:</strong> EleutherAI’s Pythia (143+ checkpoints), knowledge neurons (Dai et al., 2022), Anthropic’s circuit tracing methods.</p><p><em>Dami is a Software Engineer and MS Data Science candidate at Boston University. She has many years of experience working in bio-pharmaceutical industry.</em></p><p><em>This post grew out of a conversation that started with a scientific reasoning engine and ended somewhere much more interesting. The full research direction is in development. If you’re working on related problems in machine unlearning, or AI-driven scientific discovery, I’d love to connect .</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7dc6785d0a19" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towards-explainable-ai/can-an-llm-know-that-it-knows-7dc6785d0a19">Reasoning or Recall? Solving AI’s “Silent Knowledge” Problem</a> was originally published in <a href="https://medium.com/towards-explainable-ai">Towards Explainable AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>