For years, one way to teach a model the cases of a product was fine-tuning: training it on your own examples. That road is closing. According to its deprecations page, since May 2026 OpenAI no longer lets organizations that have never fine-tuned create new training jobs, and from January 2027 its existing customers won’t be able to either.

If the model is rented, as we argued in the first post, then what has to learn from mistakes is the harness. Here is how we do it: every reply can be reported; a mistake is fixed in the data first, then in the missing fact, and only then in the prompt; and each fix is proven by a family of test cases that run more than once. This is the last post in the series.

Make every mistake reportable on the spot

Back to the note from the second post. The project has two open Novak jobs, the panel installation and a battery quote, and suppose this time Werk gets it wrong: it marks the quote as done. The person replies “it was the installation,” and Werk undoes the change and makes it on the right card. For them, the problem ends there.

For the harness, that’s where the work starts. A mistake that only gets fixed in the chat will happen again with the next client who has two open jobs. That’s why every reply from Werk can be reported: pick a reason and, if you like, add a voice note or a few words explaining it.

It pays to have this from day one, when it still seems early, and to decide who reads the reports. What someone writes in a report is part of their team’s work. At Werk, an agent reads those reports, not a person.

Data first, instructions last

Faced with a model’s mistake, the temptation is to go straight to the prompt and add a line. First, read everything the model received on that turn, in full: not what the code was supposed to send, but the exact request that went out to the provider, with the instructions, the conversation, the cards and the message. Then the questions come in order.

The first is whether the data was right. If the conversation arrived incomplete, the right card wasn’t among the candidates, or two parts of the context contradicted each other, the model had no way to get it right, and the fix belongs in the harness. In the Novak case, the data was right: the model saw both cards and where each one matched.

The second is whether a fact was missing. The model lacked what the person knew by heart: the installation was theirs and they had started it that morning, while the quote belonged to someone else. No instruction can replace a fact the model can’t see. The fix is to show it who holds each card and what state it’s in.

Only after those two questions do you look at the instructions, when the model has correct, complete data and still gets it wrong. Even then, what you write is a principle, never a rule for the case. A line like “if they say Novak, it’s the installation” fixes this note and breaks next week’s, when the quote is the one that’s done.

From one mistake to a family of cases

A fix has to be proven, and one case won’t prove it. The Novak report gives you a class of mistake: two open jobs for the same client, and a message that doesn’t say which.

No instruction can replace a fact the model can’t see.

From that class you write a family of cases: the same situation in other words and other languages, as a voice note, with two different clients who share a name. And the opposite case too, with a single open job, where there’s nothing to be unsure about. If the fix only works for “done with the Novak job,” the mistake is still there.

These cases are best written by someone other than whoever makes the fix. Whoever fixes something tends to test exactly what they fixed; whoever starts from the class of mistake tests what the person needs. That way special cases live in the test battery, not in the prompt: the instructions state principles, and the cases check that those principles hold.

Measure more than once

A model doesn’t always answer the same message the same way. τ-bench (2024), a benchmark of agents that serve customers with tools, measured it: even the best agents of the time solved fewer than half of the tasks. And in its retail setting, the chance of solving the same task correctly on all eight tries fell below 25%.

That’s why every case runs more than once. A fix that passes once and fails on the next run isn’t done.

A case that fails one time in ten is a mistake someone will run into on a roof, sooner or later.

The chance of solving the same task correctly on all eight tries fell below 25%. (τ-bench, 2024)

And when a fix goes in, run the whole battery, not just the new cases. That’s how you know a mistake was fixed without breaking something else.

The harness learns with use

Models will keep getting things wrong now and then, like anyone working in a hurry. What you can build is what happens next: every reported mistake becomes a family of cases, and those cases get checked every time something changes.

Developers and office workers already have a harness built for their work. The person installing solar panels on a roof now has one too, and every mistake someone reports makes it better for everyone.

Originally published at https://blog.getwerk.ai on November 8, 2026.

In the first post of this series, a four-second voice note helped us separate what the model does from what the harness does: “Done with the Novak job.” That was the easy case. A real day sounds more like this: “Done with the Novak job. We’re starting the school job tomorrow at 9, remind me at 8.”

That one voice note holds three requests: close a job, record a new one and set a reminder. Whoever records it on the way down from the roof won’t send them one at a time or say which card they mean, and they shouldn’t have to. This post shows how we built the harness so they don’t, in three decisions: a router that splits the message into requests and hands each to the agent that owns it, code that searches the project while the model chooses the card, and asking only when in doubt, never to confirm.

A router that hands out work and decides nothing

The router is the first thing every message touches, and it should do exactly one thing: split the message into requests and hand each one to the agent that owns that kind of work. In the Novak note, each of the three requests lands with a different agent, and each agent gets only its own.

The router splits a message by the things asked, not by sentences. “We’re starting the school job tomorrow at 9, remind me at 8” is one sentence and two requests: a new job with its start time, and a reminder. Each request travels with the person’s exact words rather than the router’s summary, so the agent reads what was actually said.

Everything else stays out of the router. It records nothing, picks no cards, looks for no keywords and has no rules for any language: it understands a request in whatever language it comes in. If it picked cards too, two pieces would be making the same decision, and sooner or later they would make it differently.

What the router knows about each agent isn’t written in the router either. It comes from what each agent declares about the work it does. Adding an agent, or changing what one does, never means touching the router.

Show the model what the person saw

To make sense of “the Novak job” or “tomorrow at 9,” the model needs what the person had in front of them when they recorded the note: the conversation as it looked on their screen. Every line carries its time, and the current time appears once, at the top. That way “tomorrow” means the same day to the router and to every agent.

Code searches and the model chooses. For “the Novak job,” code looks through everything a card holds, not just its title: the description, the notes, the text in its photos and what was said in its voice notes. It searches by words and by meaning, then hands the model a few candidates, each with the place where it matched: “Novak, in the title” or “Novak, in Tuesday’s voice note.”

With that, the model does what it does best: read the person’s words and decide which of those cards they mean. Showing it the whole project would make it slower, more expensive and easier to confuse. Showing it a single card would mean deciding for it.

There is one search per message, and both the router and the agent see its result. If each ran its own search, they could end up talking about different cards without knowing it.

The router splits a message by the things asked, not by sentences.

The model never handles a card’s internal ID, either. It sees each card by a short number, like #12, and code translates that number back into the real card. A long ID is easy to copy wrong; a short number isn’t.

Ask only when in doubt, never to confirm

The last decision is the one people notice most: when to ask. Werk doesn’t ask for confirmation. It does what it’s asked and says what it did. If something wasn’t right, it can be undone from the same chat, and each card’s history keeps every change.

Asking “Confirm?” before every change looks careful, but with use it stops being careful. Anthropic describes this in its post on how it built auto mode for Claude Code (2026): when people have to approve so many actions, they stop looking at what they approve. In practice, users accept 93% of permission prompts anyway.

For someone working on a roof, every confirmation is also an interruption: phone out again, gloves off again. So Werk asks only when it truly can’t tell. If the project has two open Novak jobs, the panel installation and a battery quote, “done with the Novak job” fits both. If nothing else tells them apart, Werk asks which one instead of guessing.

The two questions look alike, but they aren’t. “Which one?” asks the person for a fact the harness doesn’t have. “Confirm?” asks for permission they already gave when they sent the message.

In practice, users accept 93% of permission prompts anyway. (Anthropic, 2026)

Even so, the harness will sometimes get it wrong. How that mistake gets caught, how it becomes a test and where it gets fixed is the subject of the third post.

That way, people on the move can keep talking the way they do, in a hurry and with three things in one breath, and each one ends up where it belongs.

Prompt caching has been covered a lot. The big providers’ guides explain how it works, and many posts tell how a team’s hit rate jumped after one fix. We want to add a way to watch it happen: one agent’s prompt, drawn the same way at every step, with the measured number in each picture. Sometimes the same idea, shown another way, is what makes it click.

To show it, we built a small recipe agent based on what happened in our own agents: a prompt that looks fine, fixed in four moves, every version sent to a real model API. Same model, same rules, not a word changed: its prompts got 73% cheaper, and the whole test cost less than a cent. At the end we show a test that keeps them from coming undone, and what the same moves did in ours.

How a prompt cache decides what you pay

Every call to a model sends the whole request again: the tools, the instructions and the conversation. A prompt cache lets the provider keep the work it already did on the start of a request, and reuse it when the next request starts exactly the same way. In the guides from OpenAI and Anthropic, reading from the cache costs a tenth of the normal input price or less, and writing to it a quarter more.

The rule behind it is short: a prompt cache reuses everything up to the first thing that changes. That is the whole game: what never changes goes first, and what changes goes last.

Four moves on a recipe agent

Our test agent writes recipes. It has two tools, find_in_pantry and save_recipe, about a thousand words of rules, and a profile for each person. The app has two plans: Pro can save recipes, free cannot. Ana is on the Pro plan and vegetarian; Leo is on the free plan. During one evening, each of them sends three messages.

A prompt that looks fine

This is the prompt many agents start with. The date goes on top, so the agent knows what day it is. Ana’s details come next, so it can personalize. Her plan decides which tools she gets. And like many harnesses, it builds the whole prompt again before every call to the model.

The cache was on, and nothing came back from it. Every message wrote its whole prompt into the cache and none read it back, so the evening cost more than it would have with no cache at all.

Step 1: Move what changes on every message to the end

The date changes every minute, and it sits near the top. Whatever comes after it is new work on every message, the thousand words of rules included. Both guides say the same: what changes on every request goes at the end. So the date moves into the user message.

Each person’s later messages now read almost their whole prompt from the cache, and the evening went from 0% to 65%. Anything that changes on every message goes at the end.

A prompt cache reuses everything up to the first thing that changes.

And yet Leo’s first message still read nothing.

Step 2: Keep personal details out of the shared part

Ana’s details change with the person. With them in the instructions, nobody else can reuse Ana’s prompt. So they move into the user message, and the instructions become the same text for everyone.

The evening went down a little, to 64%, because Ana’s details now travel with each of her messages. And Leo still read nothing. What belongs to one person goes after what everyone shares, but that alone was not enough: something still differed, and it came before the instructions.

Step 3: Give every plan the same tool list

The tools come before the instructions, and Leo’s free plan gave him a shorter list. His request differed from Ana’s at its very first line, so it could not reuse anything she had left in the cache.

The fix is to send every plan the same list. The list only describes the tools; it does not decide what Leo may use. That is decided apart: OpenAI’s guide says to “restrict which tools are callable while keeping the supplied tools list stable”, and the code that runs the tools refuses anything his plan does not allow. Give everyone the same menu, and decide at the table what each one may order.

Leo’s first message now reused what Ana had left in the cache, and the evening reached 80%. The plan still decides what each person can do; it just stopped deciding what the prompt says.

Step 4: Tool calls almost free: append, don’t rebuild

To answer Ana, the agent first checks her pantry: the model asks for find_in_pantry, the agent runs it and calls the model again with the result. If that call only adds the tool call and its result at the end, everything before them comes from the cache. OpenAI's guide puts it plainly: "Append new messages rather than rewriting earlier turns."

The call after the tool read 94% of its prompt from the cache. An agent makes many calls like this one, so they end up almost free. In the first version, the agent rebuilt the whole prompt before every call, with a fresh time on top, and the same call read nothing. A tool call should add to the request, never rebuild it.

What the four moves add up to

Together, the four moves changed what the evening cost. At a price of $1 per million input tokens, its prompts went from $1.76 to $0.48 per thousand calls, and the call after the tool cost $0.25.

None of the four moves changed a word of the rules. We ran the test on gpt-6-luna with reasoning off; for any model, multiply these costs by its price per million input tokens.

Check your own agent

Take two requests your agent sends, from two people a minute apart, and find the first place where they differ. Everything before it can come from the cache; everything after it is paid on every call. If that place is near the top, look at what sits there: a date, a name, a tool list built for one plan.

Then keep it that way with a test. This one never calls a model: it builds two requests and compares how they start.

# two people, two times: the start of their requests must be identical
ana = build_request(ANA, "6:42 PM", "What can I make tonight with chickpeas and spinach?")
leo = build_request(LEO, "6:43 PM", "A quick lunch with eggs for tomorrow?")
assert ana["tools"] == leo["tools"]                 # the same tool list
assert ana["messages"][0] == leo["messages"][0]     # the same instructions
 
# the call after a tool must start with the whole call before it
second = add_tool_result(ana, PANTRY_CALL, PANTRY_RESULT)
assert second["messages"][:2] == ana["messages"]

If someone later puts the date back on top, writes a name into the instructions or trims the tool list for one plan, the first check fails. If someone rebuilds the prompt after a tool call, the second one does.

How we did it in our own agents

In an earlier post on owning the harness we said caching deserved a post of its own. This October we made the same kind of moves on our real agents: the time went to the end of each request, the tool definitions stopped varying by project, role or language, and each tool call adds to the request instead of rebuilding it.

A tool call should add to the request, never rebuild it.

We saw even more improvement than in this small evening, because our agents often make more than one tool call for each message, and those are the calls that read the most from the cache. The more tool calls, the higher the average. Our dashboard went from the 60s to 93%. What we are building is still in development, with few people and long gaps between messages, so many calls find the cache expired, like Ana’s first message of the evening; with continuous use we expect 96 to 98%.

Look at the first lines of your prompt. If they change from one call to the next, move them to the end, and let the cache do the rest.

Originally published at blog.getwerk.ai

Harness² is a framework from Google Cloud AI Research and Northwestern University that improves an AI agent without touching the model. The model stays frozen; only its harness changes: the prompts, tools, skills, and scripts around it. On WorkBuddy-Bench, a benchmark of office, web, and coding tasks, that alone raised GLM-5’s average score by 11.5 points.

I made the same argument in the first post of my series on building a harness: the model is rented, the harness is yours. The next parts cover how a message reaches the right agent and how to test a harness that learns from its mistakes.

Most summaries of Harness² stop at the headline. I read the paper (2026) and its code to see where the gains came from, because that is the part any team can use without running the framework.

How Harness² improves a harness without an answer key

Harness² works one task at a time. An editor model writes a few versions of the harness for the task in front of it, the same frozen model runs the task under each version, and the framework compares how the runs behaved: what each one read, did, and produced.

It then combines the changes that helped. There is no answer key: it judges by behavior.

What worked goes into a library. Each entry holds the adopted harness and a short EXPERIENCE.md file with the situation the lesson applies to, the lesson itself, and the evidence behind it. The next task starts again from the clean base harness, but the editor reads the library before proposing anything.

It does not need a stronger model to do the editing. With the same model as executor and editor, every model they tried improved, and GLM-5 editing its own harness gained 13.7 points on LAB, a benchmark of legal work.

Editing a harness without evidence made it worse

The paper also tried the shortcut: let the editor rewrite the base harness directly, without seeing any runs, any probes or the library. On coding tasks, the score fell from 73.4 to 63.2, and on web tasks from 69.9 to 65.0. Asked to improve its own setup blind, the model made it worse.

That matches what I have seen building agents. When an agent fails, the first instinct is to add a line to its prompt, before looking at what it actually received and what it did. Harness² is built the other way around: every change comes from comparing real runs.

It is careful even with its own winners. The step that writes the lessons is told that being picked is not proof an edit helped: the chosen run may have won for another reason. A lesson that rests only on mixed evidence is narrowed or dropped.

Rewriting the harness without seeing any runs dropped the coding score from 73.4 to 63.2. (Harness², 2026)

The gains were in scripts and skills, not in the prompt

In an appendix (Table 6), the authors take the improved harness and remove one group of components at a time. Removing the prompt-related components costs 1.5 points on average. Removing the skills costs 5.3, and removing the scripts 3.7.

It is one setup, Gemini 3.5 Flash on WorkBuddy-Bench, and the authors note the effects do not simply add up, but the prompt carried the least.

A script, in their terms, is code the agent runs instead of reasoning through a step token by token. A skill is a procedure for one kind of job, loaded only when that job comes up. Both are more specific than a prompt, which the model reads on every call whether it needs it or not.

In my experience, a rule added to a prompt tends to fix the case it came from, and a check in code holds every time. The paper’s own case study shows it.

On a JobBench task, the agent never reopened the spreadsheets it wrote. The improved harness added, among other changes, verify_outputs.py: a script that reopens every file the agent produced and flags, for example, a formula cell a reader would see blank.

Its rule reads: confirming a file exists is not confirming it is correct. The score on that task went from 43.1 to 77.8.

Removing the skills cost 5.3 points. Removing the prompt-related components cost 1.5. (Harness², 2026)

Write the fact, not the instruction to find it

The editor’s instructions are printed in the paper’s appendix, and one line stands out: verified facts about the workspace are “the most valuable thing you can encode”.

If the runs established that the role “HR” in one file maps to “HR Manager” in another, the editor writes that mapping into the harness, not a rule telling the agent to reconcile role names. The rule leaves the hardest step to the agent; the fact removes it.

It also labels those facts (“in this workspace…”) and treats a fact copied from an earlier task as a hypothesis to check again. This is a design choice in their instructions, not a measured result, but it is the one I would copy first: many agent mistakes that look like poor reasoning are a fact the model never saw.

What Harness² leaves open

Lessons are written once. An EXPERIENCE.md entry keeps the confidence it was given when it was written, and nothing changes it when a later task reuses the lesson, and it goes badly. The authors list revising outdated experience as future work.

Its judge is the model itself. The limitations section says plainly that behavioral comparisons are “sensitive to execution variability and shared model biases”, and every probe adds runs and cost.

It also answers the question of learning without retraining differently from Hermes Agent, which I read for the first post. Hermes writes skills after a session and keeps them for the next ones. Harness² starts every task from a clean harness and tests candidates before it adopts anything.

What to take from it without the framework

You can use what Harness² found without running it: look at what the agent received and did before changing anything, move what must happen the same way every time out of the prompt and into code, and give the agent the fact instead of the instruction to find it.

In every comparison in the paper, the model stayed the same. What moved the score was what the harness gave it to work with.

This story is published under the Generative AI publication. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories. Let’s shape the future of AI together!

What Google’s Harness² shows about improving an AI agent was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Developers have Claude Code or Codex. Office workers have Claude Cowork or ChatGPT Work. Each of these is a harness: the scaffolding around an AI model that lets it do real work, with tools, context and rules. The person installing solar panels on a roof has nothing built for their work.

At Werk we built one for teams on the move, and our first decision was to keep it separate from the model: the model understands and proposes, and the harness decides what it reads, what it touches and what gets recorded. Four practices make that possible: talk to models through an interface of your own, have each agent ask for a level of work instead of a model, give every model the same format, and put what stays the same first. This post starts a series on how it is built. Why Werk exists is in Why we built Werk.

What the model does and what the harness does

Take a four-second voice note recorded on a roof: “Done with the Novak job.” Getting that sentence recorded correctly takes two different kinds of work.

The model understands the request: a job is finished, and of the cards in front of it, Novak’s is the one being named. The harness does everything else. It chooses which cards to show the model, checks that this person may change the card, records the change once even if the message is processed twice, and tells whoever needs to know.

A study published on arXiv in 2026 read the source code of eleven coding harnesses, including Claude Code, Codex, Gemini CLI, Hermes and Pi. It defines a harness as what connects a model to tools, context, safety, orchestration and extensions. Put simply: the model understands and proposes. The harness decides what it reads, what it touches and what gets recorded.

The same study found that none of the eleven harnesses studied imports a general-purpose agent framework; all of them run hand-written loops. Anthropic reached the same conclusion in Building effective agents (2024): the most successful implementations it saw did not use complex frameworks, and its advice is to start by calling model APIs directly.

How to keep the harness separate from the model

If the harness makes the decisions, it should not depend on any one model. Models improve every few months, no model is best at everything, and no cloud service is up 100% of the time, as providers report openly on their status pages, such as Anthropic’s and OpenAI’s. These are the four practices that keep the Novak note working whatever model is behind it.

We arrived at them by reading the code of more than fifteen open-source projects (MIT or Apache 2.0), such as Codex, Gemini CLI, OpenHands, Aider and Pi, and the documentation of others. We took ideas, not code.

The first is to talk to models through an interface of your own. No part of the harness calls a provider directly: the provider is configuration. We work with the major providers in the industry, and moving from one to another is a configuration change we measure before we make it.

The second is that each agent asks for a level of work, not a model. The agent that marks Novak’s card as done does not know which model serves it. It declares the level it needs, and our battery of test cases decides, agent by agent, which model fills that slot.

The third is a format every model reads the same way. We follow what the industry recommends, for example the guides from Anthropic and OpenAI: each kind of data inside its own tag, so the model knows where each piece starts and ends. The conversation, the cards and the voice note reach any model in the same shape.

None of the eleven harnesses studied imports a general-purpose agent framework. (Barbaste, Darrigol, Vu and Wiltberger, 2026)

The fourth is to put what stays the same first and what changes last. Each agent’s instructions are identical on every message; the Novak note is not. With the stable part first, the provider reuses what it has already processed, so the answer arrives sooner and costs less: with DeepSeek Harness, one user measured a 98% cache hit rate across 650 requests. Cache optimization deserves a post of its own, and we will write it.

What no SDK can solve for you

Agent SDKs such as the OpenAI Agents SDK and the Claude Agent SDK are a great way to start: they handle the cycle of calling the model, running tools and repeating. We studied them closely, along with LangGraph, the Vercel AI SDK, Pydantic AI and Mastra. But the hardest part of the Novak note is not in that cycle.

It is in what belongs to a team. Novak’s card lives in a record that several people share, each with their own permissions. If Werk asks which of two cards was meant and the answer comes back as “the second one,” that answer has to stay tied to what that person saw on screen. That depends on each product, and the good SDKs say so plainly: Mastra’s documentation recommends binding an approval to the exact arguments the approver saw.

A personal agent is not enough either. Hermes Agent and OpenClaw, two of the open-source projects we read, are built for one person. Hermes says so in its security policy: it is a single-tenant personal agent, and everyone with access is equally trusted. For a personal agent, that is the right call. On a team, each person sees their own work and what the project shares, and the model never decides who sees what.

The model understands and proposes. The harness decides what it reads, what it touches and what gets recorded.

That is why we built our own harness, and why the next part of this series covers its central piece: the router, which receives the Novak note and hands it to the right agent.

The model changes, the harness stays

Because the important decisions live in the harness, the model can be chosen, compared and replaced when a better one comes along. The harness is what is yours: what each agent reads, what the code checks before touching the record, how an error is caught and how it is fixed without breaking something else.

Models will keep getting better. With a well-separated harness, each improvement reaches daily use on its own: a voice note understood better, the right card, the notice to whoever needs to know.

Teams that use Werk do not have to do anything to benefit. The harness does that work.

Originally published at https://blog.getwerk.ai on October 28, 2026.