Harnesses, Part 2: Cacheonomics
It was never only about tokens. It’s all about the cache.
Read the article for free here.
When I wrote Part 1, I thought the whole game was that one decision at the first turn, eager or JIT, two camps, pick a side.
I kept building. Kept measuring, logging, time to first byte, time to first turn, cache reads, aiming to drive everything else to zero. And the further down the rabbit hole I went, the more the two camps stopped looking like two. They were the same bet made from opposite ends, driving towards the same destination. They still behave differently, and the rest of this is about the how, different breakpoints, different lifetimes, different shareability ceilings. But underneath both of them was a single rule I could not get away from.
Everything inside a well-written harness is an implementation detail around one law.
The Law of Cache Economics. “Never invalidate the cache.”
Sounds easy. It’s the single most violated rule in every harness I measured, including ones built by labs that wrote caching docs.
The tokens were never the point, the cache was and always will be. And where harnesses actually diverge is mostly in what their platform’s API allows. More on that later.
Note; several harness specifics below are actively moving. Claude Code’s system prompt was reportedly cut ~80% (Boris, YC, July 28 2026), the MCP V2 spec (2026–07–28) changes tool-caching & inheritance, and Gemini CLI has already been retired for Antigravity. Treat harness-specific facts as a snapshot, the law underneath them is what’s stable.
What is Harness Engineering?
OpenAI named the practice harness engineering, and the metaphor tracks so well.
The model is like a horse. It possesses immense power, speed, and potential, but it is directionless and can run wild without external boundaries. Reins to steer it, blinders to keep it focused, a saddle so it can carry weight, that raw power is nearly useless until it is harnessed. The harness is the infrastructure surrounding the model, think native tool definitions, context docs, policies, safety rules (sandboxes), eval beds, feedback loops, the things that guide how the model executes work. The rider is the engineer or agent who sets the destination and the boundaries with an intent, a prompt, rather than doing everything by hand. That is Harness Engineering.
Every decision in this piece is a decision about what the harness does to protect the prefix, and by extension, the cache. Every good harness listed here has made over a hundred small decisions that all lean the same way, keep the prefix stable, keep the volatile stuff late, never rewrite what could have been read.
The Real Cost Is a Reversal
Most arguments about eager versus JIT are denominated in the wrong unit. Eager pays more upfront, JIT spends fewer tokens but risks missing something. That accounting misses the cost that actually hurts, a reversal.
The model produces code, then a later turn surfaces a constraint that invalidates part of what it already wrote. The naive price is the corrective turn. The real price is a blast radius, everything touched between the wrong assumption and the moment it got caught is now suspect and needs re-verifying, whether or not it was actually wrong. Have you ever watched an agent work on partial information, validating a claim that it didn’t trust? That check is usually the largest term, and the easiest to overlook. Turns-to-discovery matters more than the size of the mistake. Caught on turn two, cheap. Caught on turn fifteen, fourteen turns of dependent work have to be unwound.
This reframes the tradeoff. Eager hydration is an insurance premium, paid upfront whether or not a reversal was ever coming. JIT search is a claim, cheap when caught early, brutal when it surfaces late and the blast radius is wide.
To be clear, the cache doesn’t own all of this. Missing information can be devastating for reasons that have nothing to do with caching, a wrong architectural call, a bad security assumption, and it can just as easily be cheap to fix with no eviction at all. What the cache protects is the token cost of unwinding dependent work, real, but not the whole reversal cost. Still, this is where the ledger leans eager. JIT’s drip-fed discovery is what raises turns-to-discovery, so the JIT case rests on keeping reversals rare and caught early, bought with a cleaner, more shareable prefix.
The cost curve. The uncached path grows roughly quadratically in the context you keep re-sending. The cached path stays nearly flat, because the prefix reads at a tenth of the price. OpenAI has blogged about this exactly.
Who Reads This Cache Next?
Eager warms the cache for a single session. JIT keeps the prefix clean enough to warm it for everyone after, from subagent, to team, then the org. Two different answers to the same question, and it comes down entirely to how each one’s built.
A clean prefix is a cacheable prefix.
Anthropic’s cache is strictly prefix-based and byte-exact. It hashes tools, then system, then messages, in that order, up to the block marked with cache_control. A single token changed before that mark misses everything after it, rewritten at full price.
Eager hydration lives closer to that wall. The more workspace state it commits to the front of the context, a live file listing, open editor tabs, a wall-clock stamp, the more of the prefix is perishable. Kept in the tail, that state still caches fine inside a single session, and Cline and Roo Code prove it does.
The ceiling is sharing. A prefix full of session-specific state cannot be shared across sessions, users, or a team, because the parts that differ between them are exactly the parts that got front-loaded. More on this later.
JIT keeps the front of the context simple on purpose. Tools and a stable system prompt sit at the top, the volatile per-turn input sits at the bottom, and the search results that would have been eagerly hydrated arrive as tool calls late in the message array where they belong. The prefix stays identical across requests, sessions, and, crucially, across users.
So where do the breakpoints go?
Anthropic exposes up to four cache breakpoints per request, and the fun part is deciding what each one protects. The standard shape is two: one breakpoint at the end of the stable layer (tool definitions plus the system prompt, the part that does not change for the life of the session), and a second at the tail, anchoring the most recent tool calls as the conversation grows.
Two details bite people here. First, the minimum cacheable prefix differs by model. Anthropic’s baseline was 1,024 tokens, with higher minimums on some models and further changes across newer model families, so you need to handle each model individually. Tiny prompts may not cache at all no matter how you mark them. Second, the automatic lookback only reaches back about ~20 content blocks, so a long agent turn that emits more than that can push an earlier write out of range and quietly start missing. Long turns want an intermediate breakpoint every dozen-odd blocks to stay covered.
This is where JIT earns its extra machinery. Eager writes once, a handful of blocks at the start of the first turn, rarely reaching the twenty-block ceiling because there is rarely enough volume. JIT writes many times: every grep, every file read, every tool result is its own block, appended one after another, and a single investigative turn can burn through a dozen. Stack that across a long session and the tail is dozens of blocks, and the lookback stops reaching the last real breakpoint. Not because JIT is careless, because discovery is drip-fed and eager is not. Eager pays for its stability once, upfront. JIT earns it turn by turn, which is why it needs the intermediate breakpoints, the deferred tool loading, the disciplined append-only ordering. The complexity is not a design flaw, it is the price of retrieving instead of front-loading.
Takeaway: Monitor cache responses across sessions and models, the minimums and lookback behavior are exactly the kind of thing that drifts under you.
That two-breakpoint shape is a floor, not a ceiling. The other two are what get a long session close to a hundred percent: use them as rolling anchors, repositioned as the conversation grows, promoting the last handful of turns into a fresh breakpoint every so often instead of letting them age past the lookback window before they ever get a chance to cache. Two breakpoints protect the beginning and the immediate tail. Four keep re-anchoring the middle so nothing between them falls out of range while the session is still running.
Everything left of the live tail reads from cache at 0.1x. This is the difference between a 96%+ hit rate and a 60% one on the exact same workload. (read anthropics docs on this, pasted at the bottom.)
The single lever behind that gap is where the volatile content sits. Push timestamps, open tabs, and todo lists to the front, ahead of the breakpoints, and the rate falls off a cliff; keep them in the tail, or omit them, and it climbs.
Two Breakpoints, Two Lifetimes: TTL Tiering
Different breakpoints, different lifetimes — tiering handles the gap. Anthropic offers two TTLs, a five-minute default and a one-hour option. Both refresh on every read, so an active session keeps its cache warm indefinitely just by using it.
The stable layer (tools and system prompt) is a natural fit for the one-hour TTL — it is the same bytes for every session and every teammate, so it should survive the quiet stretches between calls. The volatile tail (recent tool calls) fits the five-minute TTL, because it churns fast and there is no point paying to preserve it long.
The one-hour tier only pays for itself when the prefix genuinely lives longer than a five-minute cache would have survived. There is also a strict ordering rule. One-hour entries must appear before five-minute entries in the prefix. That falls out naturally from the layering, long-lived at the top, short-lived at the bottom, exactly the order you would build it in anyway.
A cache miss here and there is fine, it is evidence the system is working, files get edited, a new tool genuinely got pulled in. The failure state is not the occasional miss. Eager hydration hits fine inside a session, ninety percent, same as the best, but only when it is disciplined about what it front-loads, as Cline and Roo Code are. Commit volatile workspace state to the front of the context instead and that same path slides toward the sixty-percent floor. Its structural miss is different and unavoidable, the prefix is personal from the first byte, so it can never be shared across a second session, a teammate, or a fleet.
The Sequential Answer, Cline and Roo Code
I want to be fair to the harnesses that proved eager hydration works, Cline first, Roo Code alongside it. Their base cache hit rate is high, and it is not luck. Roo Code’s default shape illustrates the whole principle without ever arguing for it.
Out of the box it ships five modes. Orchestrator, Architect, Code, Debug, Ask. The Orchestrator’s job is to reason and write clear handoffs to the mode built for the task. That mode runs, returns a summary, and hands control back. Every step is sequential, one agent switching hats, one timeline, nothing running in parallel. Every team building a harness should replicate this: the orchestrator’s job is to reason and write clean handoffs to each subagent below it. Simple.
That is exactly why the cache behavior is so good. Watch the token accounting on a live session and it reads cache, cache, cache. I looked at a stretch of ten to fifteen turns with zero cache writes, which surprised me until I worked it out. On a single repo, handled one mode at a time, whatever a downstream mode needed had almost always already been pulled in by an upstream mode a few turns back. The five-minute TTL refreshes on every read, and sequential handoffs move fast enough that the cache never cools between them.
That is coherence you get for free by refusing to parallelize. No two agents contend for the same files, no model boundary splits the cache, no concurrent write races a read. Sequentiality is a cache-coherence strategy the architecture enforces automatically. That guarantee is exactly what a parallel design gives up.
The Parallel Answer, Subagents and Recursive Admission Control
Spend that coherence carefully, because parallelism is where it stops being free. Sub-agents are where the guarantee has to be rebuilt by hand, and where it collapses quietly if nobody is watching the right layer. A sub-agent typically inherits from its parent, MCP servers being the obvious case. Spawn one and it comes up holding the parent’s entire tool and MCP surface, whether or not its scoped task touches any of it. That inheritance is eager hydration, one level beneath the context window, where the token counters on the dashboard cannot see it. [A lot of change is coming at this level, thanks to the MCP V2 2026–07–28 spec.]
The fix is recursive admission control. In my earlier post, Part 1, I argued for it at the top level, nothing enters the window the task does not need. The extension is that the same gate has to fire at every level of delegation, or the sub-agent becomes the leak.
So how do multiple agents touch a shared repo without fragmenting the cache?
Cline/Roo Code and Claude Code give two different answers, but drive towards the same destination. Cline/Roo Code’s answer, don’t have multiple agents, stay sequential, one warm cache by construction, and the coherence problem never gets the chance to exist.
Get Jordan Carson’s stories in your inbox
Join Medium for free to get updates from this writer.
Claude Code’s answer is to allow multiple agents, and it actually gives you two modes. A standard subagent runs in its own isolated context. It doesn’t inherit the parent’s system prompt or history, it starts fresh, loads the shared project config and tool pool, and builds a clean prefix of its own. It never touches the parent’s cache, so it can’t invalidate it. A fork does the opposite, it inherits the parent’s prefix, model, and full history byte for byte, and reads the cache the parent already paid for instead of assembling its own. And a parent can itself be a subagent, so this nests, depth-first, all the way down. Both paths obey the law. The fork reuses the warm cache; the standard subagent never touches it. Neither one invalidates anything, and that is the point.
Model Binding Is a Cache-Coherence Requirement
Model choice is the other half, and it is not a style preference, it is a cache-coherence requirement. A cached prefix is bound to a specific model, so the moment two sub-agents on the same repo run on different models, they stop sharing cache entirely, each paying its own write premium instead of one fleet paying once. Switching models mid-session does the same damage in miniature. Some harnesses prevent this, other’s don’t.
Scaling the Prefix, from Subagents to the Whole Org
Your prefix should be for your subagents first. Get that right and you can percolate it up to the team, the org, and beyond. This is the team-level cache Part 1 hinted at.
A stable, model-consistent, admission-controlled prefix is one a whole team of agents can read — the payoff eager hydration can’t reach, since its prefix is perishable from the first byte. But be precise about what shared means in practice, in Claude Code, sharing is structural only at the system-prompt and tool-schema layer, and via forks that inherit the parent’s prefix byte-for-byte. Standard subagents start fresh by design and never touch the parent’s cache, so team-level reuse below that top layer is something you architect deliberately, not a default you inherit. The API permits the org-wide warm prefix, the harness only earns it if it’s built to.
Because the system prompt and tool schemas are identical for everyone, that layer is one warm prefix the whole company reads, with the tree branching below it exactly as far as the work is shared. Exercise it constantly and you can hold a warm cache org-wide, indefinitely — as long as everyone is on the same model. Split the fleet across models and you split the cache with it.
Three Labs, Three APIs
Most of what a harness looks like is a response to what its provider allows. Score each harness on cache discipline (1 to 5) and retrieval posture (eager to JIT) and a shape falls out. The disciplined, JIT-leaning corner is crowded, while the eager-yet-disciplined and pure-JIT-yet-cache-blind corners each have a lone occupant, proving the two axes are separable. Aider is disciplined yet eager; Terminus 2 is pure-JIT yet cache-blind.
cache_control, hollow = automatic/implicit.Anthropic — explicit. The breakpoints and tiered TTLs detailed earlier. Four cache_control markers per request, placed by the harness itself. It is the one API that hands the lever directly instead of making a harness earn or forgo the discount — most of why so many harnesses run on Claude in the first place.
OpenAI — automatic. No breakpoints to place, no lifetime to choose. The platform matches an exact prefix of at least 1,024 tokens and discounts the read, full stop. Newer models are starting to add explicit breakpoints, but for most of what runs today the entire discipline sits on the harness’s behavior, the architecture around the API. Codex CLI does that honestly, the agent loop appends new messages rather than editing earlier ones, on purpose, so the old prompt stays an exact prefix of the new one. OpenAI has said plainly that the alternative, letting the transcript grow the way a naive loop would, makes the agent loop quadratic in the JSON sent per turn. Codex earns its hit rate by never giving the platform a reason to miss.
Google — in between. Implicit caching by default, with an explicit CachedContent option worth roughly 90% off on current-generation Gemini models if a harness chooses to manage it. Gemini CLI never did: the system prompt and the full GEMINI.md hierarchy got reassembled and resent every single call, and a tracked issue on two trivial math queries measured roughly 13,000 input tokens burned on nothing.
The Same Shift, From Every Direction
Gemini CLI in that form no longer exists. Google retired it on June 18th, 2026 for most users and replaced it with Antigravity CLI, built explicitly around orchestrated, multi-agent workflows instead of one linear session.
One production write-up from a developer running Antigravity at scale had been embedding timestamps directly in the system prompt, the exact mistake this piece keeps circling, and fixing it, stable content first, volatile content last, took the cache hit rate from 38% to 78%. A harness built explicitly around multi-agent orchestration shipped with the same rookie bug as everything before it, and the same fix produced the same shape of win.
Three labs, three different amounts of control handed to the harness, the same law underneath all three: Anthropic hands you the lever directly; OpenAI makes you earn it through discipline; Google leaves it on the table until a harness decides to pick it up.
It All Traces to Layer One
Every decision traces back to one number. Cold-start prefix size across these harnesses runs ~forty-fold, on my own estimates, not audited benchmarks, from under a thousand tokens to thirty-three thousand, before a single turn of real work happens.
That range is not a scoreboard. A small prefix and a large prefix can both be disciplined; they are just different bets on what layer one should carry. The number that matters is not how small the first turn is. It is whether whatever gets committed there gets read back a hundred times at 90% off, or rewritten from scratch because nobody protected it.
A large prefix read constantly beats a small prefix invalidated constantly, every time. Period.
This is where it is won or lost. Every decision, the clean prefix, the breakpoint placement, the tiered TTLs, the recursive admission control, the single model across the fleet, is really one decision made in one place, layer one, where context gets assembled before anything downstream ever sees it. The cache is not managed by patching misses after they happen. It is managed by assembling the prefix correctly at the root, once, and letting everything downstream obey the law one of two ways: a fork inherits that prefix and rewrites nothing, and a fresh subagent never reads it but builds off the same stable layer everyone shares. Either way the warm prefix survives, and nobody invalidates what they didn’t touch.
Where do we go from here?
Put the labs aside for a second. Gemini CLI died and came back as Antigravity, multi-agent by design. It shipped the same rookie bug, took the same fix, earned the same win. Zoom out and this stops being three products and becomes a shape. The orchestrator dispatching to a fleet is now the default now, and the harness is the control plane for it. Local, sandboxed, or running as an SDK, it is the same thing underneath, an agent spawning subagents. And yes, cost is ultimately denominated in tokens. So the moment teams and companies start running fleets of autonomous agents, the question becomes where to even begin optimizing. Prefix sharing across a fleet of agents churning the same repo is the highest-leverage. One warm prefix read by many, instead of every agent paying its own write premium.
Which raises two questions I keep circling.
The first is where those subagents live. A fleet has to run somewhere, and the environment is not plumbing you set up once and forget. How deep the spawning goes, what each layer inherits, what persists between runs and what gets torn down, all of it either protects the prefix or quietly shreds it. Build it carelessly and every guarantee you earned leaks out one level below the dashboard, where nobody is looking. Be deliberate about depth. Be deliberate about what you let them keep.
The second is memory. A simple prefix solves coherence within a session, and across a fleet inside that session. It does not solve what happens across sessions, when work spans days and the thing that has to stay coherent is not the prefix at all. It is a durable record of what changed and why. That is not a context-loading problem, it is a memory problem.
What I will say is this. Memory is for what must survive, through a repave, persisted to stop incidents and recurring patterns from recurring, and eventually forgotten when something new takes its place. It is not a convenience layer. Reach for memory when your harness can read it back, so nobody re-explains what happened fifteen sessions ago, or reinvents what a teammate already figured out on the same issue.
Everything in the harness ties back to that first turn. Everything after it is an extension of one stance, an implementation detail, keep the prefix stable, keep the volatile stuff late, never rewrite what could have been read.
Never invalidate the cache.
The List
2. Anthropic — How Claude Code uses prompt caching
3. Anthropic — Lessons from building Claude Code: Prompt caching is everything
4. OpenAI — Unrolling the Codex agent loop
5. OpenAI — Prompt caching guide
6. Google — Gemini CLI memory & context management
7. Claude Code — Resume/continue cache invalidation, #43657
8. Claude Code — Session resume invalidates prompt cache, #42338
9. Aider — Unstable repomap breaks caching, #1874
10. Gemini CLI / Antigravity — massive token overhead discussion, #27307
11. OpenCode — Gemini caching missing on OpenRouter, #36069
12. Codex CLI — stable prompt-cache key needed, #21796
13. Franklin — cache_control breakpoint budget bug, #73
14. CLIProxyAPI — round-robin breaks Gemini/Antigravity caching, #307
15. OnlyTerp/prompt-cache-skills — the Cline→Roo→Continue breakpoint bug
16. Aider — Prompt caching docs
17. Cline — Context window explained
18. Hermes Agent — Prompt assembly architecture
19. pi — Prompt caching in agents (earendil.com)
20. Terminus 2 — Harbor framework docs
21. Google Developers Blog — official transition announcement
22. Antigravity Lab — production cache strategy, the 38%→78% report
23. The Register — Bye-bye Gemini CLI coverage
