Your Agent Harness Runs on Knowledge

Prior knowledge has two parts: initial knowledge drives the task, and the acquired knowledge policy decides what happens to what the agent learns along the way

Knowledge-Driven Agent Harness, Part 1 of 6

This story was written with the assistance of an AI writing program. The ideas, the design, and the outline are mine; the AI drafted the text and the figures, and I reviewed every section and directed each revision.

An AI agent is a model plus a harness: the loop, tools, memory, checks, and permissions around the model. Most harness engineering advice focuses on the loop and the tools. Is that enough?

Imagine you are Agent Smith from The Matrix (1999), a program that guards the system. Your task: find Neo, a hacker who exists for you only as a name on the network.

Figure 1. Smith has the tools: he can take over anyone connected to the Matrix. What he doesn’t have is knowledge, so he can only guess. Image created with AI.

Smith doesn’t need more tools. He needs one fact: the hacker “Neo” is Thomas A. Anderson, a programmer at a software company. With that, the search becomes a trip to an office, which is where the Agents catch Neo in the film.

AI agents work the same way. Knowledge is a necessary part of the harness, and prior knowledge, everything you give the agent before it starts, has two parts. Initial knowledge drives the task: the facts, rules, and procedures the agent needs to act. The acquired knowledge policy decides what happens to the knowledge the agent acquires along the way: forget it, keep it for now, or verify it and save it. The longer the task, the more the acquired knowledge policy matters.

Figure 2. Initial knowledge turns a name into a person and a place. The acquired knowledge policy decides what to do with what Smith learns. Image created with AI.

The rest of this article explains what a harness is, where knowledge fits in it, and how initial knowledge and the acquired knowledge policy work.

1. What an Agent Harness Is

The model reasons. The harness is everything around it: the loop that calls the model, the tools, context management, memory, verification, permission control, and knowledge. I’ll use the shorthand Agent = Model + Harness.

Phil Schmid describes the harness as “the infrastructure that wraps around an AI model to manage long-running tasks.” His analogy: the model is the CPU, the context window is the RAM, and the harness is the operating system.

Two engineering posts show what a harness looks like in practice, and a large part of both is about knowledge:

• OpenAI’s harness engineering post (Ryan Lopopolo, February 2026) made repository knowledge the system of record. After “one big AGENTS.md” failed, the lesson was simple: “give Codex a map, not a 1,000-page instruction manual.”
• Anthropic’s post on harnesses for long-running agents (November 2025) starts from the fact that “each new session begins with no memory of what came before.” Its fix is a progress file, a feature list, and git history that each new session reads first.

Both come from coding agents. This series asks the general question: which kinds of knowledge does a harness need, where does each one go, how is it governed, and how does it grow?

Figure 3. Knowledge is one of the harness’s parts, next to the loop, tools, context, memory, verification, and permissions. Image created with AI.

2. Four Words That Get Mixed Up

These four words are often used as if they meant the same thing. In this series they don’t:

• Knowledge is verified information that is reused across tasks. It needs governance: someone owns it, and it can be wrong. For Smith: “Neo is Thomas A. Anderson.”
• Memory is what the agent writes down across sessions. It’s a candidate source of knowledge, not knowledge yet. Smith’s notes from earlier chases.
• Context is what’s in the model’s window right now. What Smith sees at this moment.
• State is where the current task stands. The harness manages it, and it sits outside knowledge governance. “Neo found. Next: interrogation.”

Figure 4. Only knowledge is verified and reused across tasks. Memory becomes knowledge only after it’s verified. Image created with AI.

3. More Knowledge Isn’t Better

If knowledge helps, why not load all of it? Because the context window isn’t free. Anthropic’s context engineering guide describes “context rot”: as the number of tokens in the context grows, the model’s ability to recall information from it goes down. The guide calls context “a finite resource with diminishing marginal returns,” and recommends “the smallest possible set of high-signal tokens.”

Think of Smith again. Millions of people are connected to the Matrix. If he watched all of them at once, the one signal that matters would be buried. Pasting the whole wiki into the prompt is the agent version of that.

This is why the easier-looking fixes don’t solve the problem:

• A bigger model still has to guess what it was never told, such as your approval rules.
• More tools give the agent more ways to act, not more reasons to pick the right action.
• Everything in the prompt buries the useful part.
• Retrieval alone helps the agent find facts, but it doesn’t decide what to do with the knowledge the agent acquires.

What works is the right knowledge at the right time: a small map that is always loaded (OpenAI’s “map, not a manual”), and details loaded when a step needs them. I wrote about the second half in an earlier article on progressive disclosure.

Figure 5. Loading everything buries the signal. A small map plus the details this step needs lets it stand out. Image created with AI.

4. Knowledge Is One Part of the Harness

Knowledge doesn’t replace the other parts of the harness, and they can’t replace it. Each part needs knowledge to do its own job well:

• Model. The model reasons, but it can’t know what nobody told it. Smith can reason about where a hacker might hide, but no amount of reasoning tells him that “Neo” is Thomas A. Anderson.
• Loop. The loop runs until the task is done, and knowledge defines “done.” For Smith, done means Neo in custody, not Neo spotted.
• Tools. Tools act and return facts. Knowledge tells the agent which tool to use and where to point it. Smith can take over anyone connected to the Matrix; knowledge narrows that down to the people around Neo.
• Context management. It decides what goes into the model’s window right now, so knowledge comes in small pieces, when needed. Smith opens Neo’s file, not the records of the whole city.
• Memory. Memory keeps what the agent writes down, and the acquired knowledge policy decides what’s worth keeping. Most of Smith’s notes on this chase can go once it ends.
• Verification. The verification part checks the result of each agent action. It can check the result against verified knowledge, or check acquired knowledge against what actually happened. When acquired knowledge passes the check, its status changes to ai-verified. Cypher, a rebel who betrays his crew, tips off the Agents. Once the ambush proves the tip right, it’s ai-verified.
• Permission control. It runs rules as deterministic, programmatic checks before a tool runs, so a rule doesn’t depend on the model remembering it. “No production database changes without approval” is checked before the tool runs, every time.

Figure 6. Knowledge doesn’t replace the other parts of the harness. It feeds each one. Image created with AI.

Get Ning Xia’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This series zooms in on knowledge, and marks where it meets each of these parts.

5. The Two Parts of Prior Knowledge

Prior knowledge is everything people give the agent before it starts. It has two parts.

Initial knowledge drives the task. Facts, rules, procedures, and heuristics: what the agent needs in order to act. For Smith: “Neo is Thomas A. Anderson, a programmer at a software company. Go to his office.”

The acquired knowledge policy handles what the agent learns. As the agent works, it keeps acquiring new knowledge: facts, observations, tips, findings. Not all of it is true, and not all of it is worth keeping. The acquired knowledge policy, written in advance, decides what happens to each piece:

• Forget it: noise. A coworker walking past.
• Keep it for this task: useful now, not later. Which floor Neo is on, until the chase ends.
• Verify it, then save it: if it passes the check, it’s saved as ai-verified, and the next task can use it. Cypher’s tip.
• Block it: anything that tries to change the agent’s rules, such as text inside a tool result that says “ignore your instructions.”
• Report it: a gap, or a conflict with prior knowledge, that the policy can’t settle. Neo does what the briefing says no one inside the Matrix can do.

Figure 7. Initial knowledge drives the task. The acquired knowledge policy decides what happens to every piece of acquired knowledge. Image created with AI.

Because the policy is written in advance, the agent doesn’t need to ask a person about every new finding. People only hear about what the policy says to report.

Some knowledge can’t wait to be learned on the job, though. “No production database changes without approval” has to be there before the first action.

6. The Longer the Task, the More the Acquired Knowledge Policy Matters

The longer the task, the more the acquired knowledge policy matters.

This is the most important idea in this article.

A short task, like a single chase, acquires little new knowledge. Everything fits in the context, and the policy has almost nothing to do.

A long task is different. Think of a hunt that runs for days: sightings, tips, dead ends, and false alarms keep arriving. Without the policy, two things go wrong:

• Noise fills the context. Every new item stays in the window, and the useful ones get buried. That’s context rot again.
• Findings are lost when the session ends. As Anthropic’s long-running agent post puts it, “each new session begins with no memory of what came before.” Anything the agent learned but didn’t save is gone.

With the policy, each new item is handled as it arrives: forgotten, kept for the task, verified and saved, blocked, or reported. The context stays small, and the saved knowledge grows, so each new session starts with more knowledge than the last one.

Figure 8. Short tasks barely need the acquired knowledge policy. Long tasks struggle without it. Image created with AI.

That’s also why the policy has to be written in advance. On a long task, nobody has time to rule on each new item as it arrives.

7. The Knowledge Loop

Put the two parts together, and knowledge moves through the harness in a loop. It moves at three moments:

1. At start, people seed prior knowledge: initial knowledge and the acquired knowledge policy. The harness puts only the small part that every step needs, the map and the rules, into the model’s context.
2. During work, the agent looks up details when a step needs them. Smith doesn’t read every file at once; he opens Neo’s file when he needs it.
3. After each step, the agent acquires new knowledge, and the acquired knowledge policy decides what to do with each piece. What passes verification is saved as ai-verified, so the next task starts with more knowledge than this one did.

Only gaps and conflicts go back to people, who update the prior knowledge.

Figure 9. People seed knowledge and handle the exceptions. The agent verifies and grows knowledge within their rules. Image created with AI.

8. What’s Next

Smith had the tools from the start. What turned a name on the network into an arrest at an office was knowledge. Your agent works the same way.

The next article goes one level down: the kinds of knowledge an agent needs, and where each one goes in the harness. Spoiler: give each kind of knowledge its place in the harness before the task starts, so the model’s best guess is the one you expect.

Knowledge-Driven Agent Harness

1. Your Agent Harness Runs on Knowledge (this article)
2. Where Knowledge Goes in Your Agent Harness (next)
3. Governing Agent Knowledge
4. Storing and Retrieving Knowledge
5. Evaluating Knowledge
6. Case Study: End to End

Related articles

• Eliminating Agent Hallucinations via Progressive Disclosure and Google ADK
• Why Your AI Gives Wrong Answers — And the Design Pattern That Fixes It
• Build It: How I Implemented “Claude Agent Skills” with Google ADK

References

• Ryan Lopopolo, “Harness engineering: leveraging Codex in an agent-first world,” OpenAI, February 11, 2026. https://openai.com/index/harness-engineering/
• Anthropic, “Effective harnesses for long-running agents,” November 26, 2025. https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
• Anthropic, “Effective context engineering for AI agents,” September 29, 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
• Phil Schmid, “The importance of Agent Harness in 2026,” January 5, 2026. https://www.philschmid.de/agent-harness-2026