<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Luis Valencia on Medium]]></title>
        <description><![CDATA[Stories by Luis Valencia on Medium]]></description>
        <link>https://medium.com/@luisevalencia?source=rss-c3bed98a7757------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*mhpAsJO6CWIxOLIsTNRh5Q.jpeg</url>
            <title>Stories by Luis Valencia on Medium</title>
            <link>https://medium.com/@luisevalencia?source=rss-c3bed98a7757------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 23:06:35 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@luisevalencia/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Sandboxing Agent-Generated Code]]></title>
            <link>https://python.plainenglish.io/sandboxing-agent-generated-code-3f427f977eaf?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/3f427f977eaf</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[agent-harness]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Mon, 26 Oct 2026 12:26:37 GMT</pubDate>
            <atom:updated>2026-10-29T14:09:49.027Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Article 8 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>. Previous: </em><a href="https://medium.com/@luisevalencia/durable-runs-replay-and-auditability-28f35870bc72"><em>Durable Runs, Replay, and Auditability</em></a><em>. Articles </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>1</em></a><em>, </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>2</em></a><em>, </em><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87"><em>3</em></a><em>, </em><a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5"><em>4</em></a><em>, </em><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1"><em>5</em></a><em>, and </em><a href="https://medium.com/@luisevalencia/context-is-not-memory-80b40eb7e047"><em>6</em></a><em> are also published.</em></p><h3>The problem</h3><p>Your agent writes Python and you want it to run that Python. Maybe it is a calculation, a data transform, or a shell one-liner to inspect a file.</p><p>The obvious implementation is subprocess.run([&quot;python&quot;, &quot;-c&quot;, code]). It works in the demo. It is also a remote code execution endpoint whose author is a language model reading untrusted text.</p><p>The model does not need to be malicious. A web page it summarized can contain an instruction. A document a user uploaded can contain one. The model repeats it faithfully, and now your backend process, with its database credentials and provider keys in its environment, executes it.</p><p>So teams add a filter. Block rm -rf. Block sudo. Block curl | sh. It feels like security.</p><p><strong>Thesis:</strong> Generated code must execute behind a boundary the code cannot argue with. A pattern filter is a guard rail. A container, a network namespace and a syscall filter are walls.</p><h3>Mental model</h3><p>Think of three rings, and ask of each one: <em>what happens if the code inside is hostile?</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*HIYLc01pvYEsMyntnBLJiQ.png" /></figure><ul><li><strong>Ring 1, policy and approval</strong> decide <em>whether</em> code runs at all. That is Article 4. It is a decision, not a containment.</li><li><strong>Ring 2, the runner container</strong> limits what the process can reach: no network, read-only filesystem, no capabilities, bounded memory and PIDs.</li><li><strong>Ring 3, the child’s syscall filter</strong> limits what the process can <em>do</em> even inside that container.</li></ul><p>Rings 2 and 3 exist because ring 1 is a judgment call that can be wrong. Everything below assumes the code is already running and may be hostile.</p><h3>What the pattern filter is, and is not</h3><p>Cogentrex has one. It is in <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/tools/terminal.py#L13">terminal.py</a>:</p><pre># Defense in depth only. Docker is the security boundary.<br>BLOCKED_PATTERNS: list[re.Pattern[str]] = [<br>    re.compile(r&quot;\brm\s+(-\w*r\w*f|-\w*f\w*r)\b&quot;),<br>    re.compile(r&quot;\bsudo\b&quot;),<br>    ...<br>]</pre><p>Read the comment. The author is telling the next reader not to trust this.</p><p>Why? Because regexes on shell text lose to any encoding: r&#39;&#39;m -rf, a variable, base64 piped to sh, a Python one-liner that calls shutil.rmtree. The filter catches the lazy mistakes and gives a readable error. It does not stop an adversary.</p><p>The failure mode here is a <em>social</em> one. Someone sees a blocklist in the code review and concludes execution is “guarded”. Keep the list if you like the early error message. Never count it.</p><h3>Boundary 1: execution never happens in the backend process</h3><p>Cogentrex does not run code in the FastAPI process, and it does not shell out to docker run from there either. An earlier version used a Docker executor. The live path now sends one JSON frame over a private Unix socket to a separate container.</p><p>The client is <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/tools/sandbox_client.py#L27">SandboxRunnerClient</a>. Its docstring states the contract: <em>&quot;Execute only the versioned runner contract; never falls back to host execution.&quot;</em></p><p>Three things make that real:</p><p><strong>It validates its own limits.</strong> Timeout must be a finite number between 0.1 and 120 seconds. Output and input must be integers between 1 KiB and 1 MiB. A boolean or NaN is rejected, because True is an int in Python and NaN compares false against every bound.</p><p><strong>It validates the response as strictly as the request.</strong> The response must have exactly nine expected keys, the right types, the same request_id it sent, and isolation == &quot;runner-container&quot;. Anything else raises. The backend does not trust the runner to be well-formed, because a compromised runner is the scenario you are designing for.</p><p><strong>It fails closed.</strong> If the socket is missing, the connection drops, or the answer times out, the client raises RuntimeError(&quot;Sandbox runner is unavailable&quot;). There is no except: run it locally branch. Absence of the sandbox means absence of execution.</p><p>The tool wrapper agrees. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/tools/terminal.py#L40">terminal_tool</a> raises if no executor was injected. The test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_sandbox_runner.py#L302">test_unix_client_contract_has_no_host_fallback</a> exists to keep it that way.</p><h3>Boundary 2: the runner container</h3><p>The runner is a separate Compose service. The relevant block of <a href="https://github.com/levalencia/cogentrex/blob/dev/docker-compose.local.yml#L98">docker-compose.local.yml</a>:</p><pre>sandbox-runner:<br>  user: &quot;10001:10001&quot;<br>  network_mode: none<br>  read_only: true<br>  cap_drop: [ALL]<br>  security_opt:<br>    - no-new-privileges:true<br>    - seccomp=./sandbox_runner/seccomp-bootstrap.json<br>  pids_limit: 64<br>  mem_limit: 128m<br>  cpus: 0.50<br>  tmpfs:<br>    - /tmp:rw,noexec,nosuid,nodev,size=16m,...</pre><p>Read it as a list of removals:</p><ul><li><strong>No network.</strong> network_mode: none means no interface except loopback. Exfiltration and calling your internal services both die here.</li><li><strong>No writable root.</strong> The filesystem is read-only. Scratch space is a 16 MB tmpfs with noexec, so a dropped binary cannot run from it.</li><li><strong>No capabilities, no privilege gain.</strong> All Linux capabilities dropped, and no-new-privileges stops setuid tricks.</li><li><strong>Non-root user.</strong> UID 10001.</li><li><strong>Bounded resources.</strong> 64 processes, 128 MB, half a CPU. A fork bomb hits pids_limit instead of your host.</li></ul><p>What is <em>absent</em> matters as much: no Docker socket, no project directory mount, no published port, no provider credentials. The only shared thing is a volume holding the Unix socket. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_sandbox_runner.py#L417">test_compose_runner_boundary</a> asserts these flags and that neither the runner nor the backend mounts docker.sock. It reads the Compose YAML, so a config regression fails CI.</p><p>Why does mounting docker.sock matter so much? Because access to it is root on the host. Many &quot;sandboxes&quot; that run code through the Docker API from the app container hand the code a way out. Keeping the socket away from both sides removes the class.</p><h3>Boundary 3: the child gets a stricter filter than the container</h3><p>Here is a subtle problem. The runner process and the code it executes run as the <strong>same user</strong> in the same container. Container flags protect the host from the container. They do not protect the <em>runner</em> from the code it launched.</p><p>If the child could open a socket, it could connect to the runner’s control socket. If it could unlink or rename the socket file, it could break the channel. If it could ptrace its parent, it could read the runner&#39;s memory.</p><p>So the runner installs a second seccomp filter in the child just before it starts, via preexec_fn. The list in <a href="https://github.com/levalencia/cogentrex/blob/dev/sandbox_runner/server.py#L41">_SECCOMP_DENIED</a> groups into four jobs:</p><ul><li><strong>Network:</strong> socket, connect, bind, listen, accept</li><li><strong>Inspecting or controlling other processes:</strong> ptrace, process_vm_readv, process_vm_writev, kill, tgkill, pidfd_send_signal</li><li><strong>Escaping the process group:</strong> setsid, setpgid</li><li><strong>Mutating the control path:</strong> unlink, rename, chmod and their *at variants</li></ul><p>The filter default is <em>allow</em>, with those calls denied. That is a <strong>denylist</strong>, and it is worth being clear about what that means. It is tuned to the specific risks of this runner. It is not a general minimal profile.</p><p>The outer container profile is different. <a href="https://github.com/levalencia/cogentrex/blob/dev/sandbox_runner/SECCOMP-PROFILE.md">SECCOMP-PROFILE.md</a> records that the outer profile is Moby&#39;s default, vendored with a recorded upstream hash, which keeps a default-deny stance. It must allow the seccomp syscall itself, so the child can load its stricter filter. The document is blunt: never replace it with seccomp=unconfined.</p><p>The test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_sandbox_runner.py#L252">test_runner_child_seccomp_blocks_socket_and_control_file_mutation</a> runs real Python in the child. It tries to open a socket, unlink a protected file, write into a read-only control directory, and chmod it. Each one must raise OSError, and the protected file must still exist afterwards.</p><p><strong>Important limit:</strong> that test is marked skipif(sys.platform != &quot;linux&quot;). Seccomp is a Linux kernel feature. When I ran the sandbox tests on my Mac, <strong>15 passed and 1 skipped</strong>, and the skipped one was exactly this. The Linux evidence comes from a separate run recorded in the <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/IMPLEMENTATION-EVIDENCE.md">implementation evidence</a>, not from my laptop. A green macOS run does not prove the syscall filter.</p><h3>Boundary 4: bounded in time, size and survivors</h3><p>A sandbox that cannot be stopped is a denial of service. The runner’s <a href="https://github.com/levalencia/cogentrex/blob/dev/sandbox_runner/server.py#L114">execute</a> handles four ways a run ends:</p><ol><li><strong>Command allow-list.</strong> Only two commands exist: python -I - and sh -s. A kind of /bin/bash is rejected with a generic invalid_request. The rejected content is not echoed back, so a secret in the payload does not leak through an error message.</li><li><strong>Code goes by stdin.</strong> There is no argv built from model text, which removes shell-injection into the launcher.</li><li><strong>ye</strong></li><li><strong>Output cap.</strong> Reading stops at the configured limit and sets truncated. The test prints 5,000 characters against a 1,024-byte limit.</li></ol><p>Then the part people forget: <strong>killing the process is not killing its children.</strong></p><p>The child starts with start_new_session=True, so it leads its own process group. On timeout, cap, cancellation, <em>and normal exit</em>, the runner calls os.killpg(pid, SIGKILL). The comment says why: the leader may already have exited while background descendants still own the group.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_sandbox_runner.py#L148">test_successful_command_kills_detached_descendants</a> runs sleep 30 &amp;, lets the shell exit successfully, and then checks the background process is gone. Compose also sets init: true so orphans get reaped.</p><p>Finally, only <strong>one execution runs at a time</strong>. A second request gets a closed runner_busy error instead of queueing. That is a deliberate trade-off: no concurrency in exchange for a simple state machine.</p><h3>The preflight that proves the wall exists</h3><p>A health check that returns ok only proves the runner started. It does not prove the filter loaded.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/tools/sandbox_client.py#L46">SandboxRunnerClient.preflight</a> does two things. It asks for health. Then it <strong>executes a probe</strong>: Python that tries socket.socket() and prints seccomp-ok only if it gets PermissionError. Any other result, including a clean exit that opened a socket, raises &quot;execution boundary failed&quot;. Backend startup fails closed when execution is enabled.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_sandbox_runner.py#L341">test_preflight_rejects_health_only_runner_without_seccomp</a> fakes a runner that is healthy but lets a socket open, and asserts startup is refused. This is the pattern I would carry to any system: <strong>test the property, not the process.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*LNbjvQNnO9iRvjP6eUU-4w.png" /></figure><h3>What frameworks provide</h3><p>Different tools sit at different rungs, and it is worth naming them.</p><ul><li><strong>Process-level helpers</strong> (subprocess, a restricted exec, a &quot;safe eval&quot;) share your process or your kernel privileges. Python-level restriction is famously escapable.</li><li><strong>Container sandboxes</strong>, like this one, give kernel-enforced isolation but share the host kernel. A kernel exploit is still a path out.</li><li><strong>MicroVMs or gVisor-style runtimes</strong> put a separate kernel or a syscall-interception layer between the code and the host. That is the stronger answer for hostile multi-tenant code, with more cost and operational weight.</li><li><strong>Managed code interpreters</strong> from agent platforms give you someone else’s sandbox. That is often the right call, as long as you check what network, file and lifetime guarantees they actually document.</li></ul><p>Cogentrex sits at the container rung. I picked it because it is reproducible on a laptop and in Compose, and because the boundary is readable in a page of YAML. If I were running other people’s code for strangers, I would move up a rung.</p><h3>Failure modes and limitations</h3><p>The canonical documentation states the status boundary plainly: this is local container isolation, <strong>not VM-grade hostile multi-tenant certification</strong>, not live-provider evidence, and not a production SLO.</p><ul><li><strong>Shared kernel.</strong> A container escape through a kernel bug defeats rings 2 and 3.</li><li><strong>Same-UID child.</strong> The child filter is a denylist. A syscall nobody listed is allowed.</li><li><strong>Seccomp is Linux-only.</strong> On macOS the test is skipped. Apple Silicon needs the runner kept native so nested seccomp is not attempted under QEMU. The evidence file documents that split-platform setup.</li><li><strong>One execution at a time.</strong> Throughput is intentionally capped.</li><li><strong>Resource limits are per runner,</strong> not per user or per run.</li><li><strong>No network at all</strong> also means the agent cannot install packages or call APIs from inside code. That is a feature, and a limitation for some tasks.</li><li><strong>Pattern filter.</strong> Cosmetic, as discussed.</li></ul><h3>Reader exercise</h3><ol><li>Open <a href="https://github.com/levalencia/cogentrex/blob/dev/sandbox_runner/server.py#L41">_SECCOMP_DENIED</a>. Name one syscall that is <em>not</em> on the list and that a hostile script could still use. Is it a real risk for this runner? What would you add?</li><li>Remove no-new-privileges from a scratch copy of the Compose file. Which test fails, and does the failure name the right flag?</li><li>Write a probe for a second property you want proven at startup, such as “cannot write to the root filesystem”. Where would it run, and what output proves it?</li><li>An agent needs to pip install a package. Design the smallest change that allows it without opening general network access. What new risk does it add?</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Concept:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/execution-isolation-sandbox.md">Execution isolation and the sandbox runner</a></li><li><strong>Module:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/modules/05-policy-and-approvals/README.md">Policy and approvals</a></li><li><strong>Seccomp record:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/sandbox_runner/SECCOMP-PROFILE.md">SECCOMP-PROFILE.md</a></li></ul><h3>Series navigation</h3><ul><li><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratch</a> — published</li><li><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a> — published</li><li><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87">3. Tool Contracts: The Boundary Between Intent and Execution</a> — published</li><li>4<a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5">. Why Tool Access Is Not Authorization</a> — published</li><li><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1">5. State Machines, Stop Reasons, and Bounded Execution</a> — published</li><li><a href="https://medium.com/@luisevalencia/context-is-not-memory-80b40eb7e047">6. Context Is Not Memory</a> — published</li><li><a href="https://medium.com/@luisevalencia/durable-runs-replay-and-auditability-28f35870bc72">7. Durable Runs, Replay, and Auditability</a> — published</li><li><strong>8. Sandboxing Agent-Generated Code</strong> — <strong>this article</strong></li></ul><p><em>Full series plan: 30 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan</em></a><em>.</em></p><h3>What comes next</h3><p>That closes Season 1: a harness with a typed loop, tool contracts, authorization, bounded execution, separated context and memory, a durable ledger, and a contained execution path.</p><p>Season 2 asks a harder question about everything you just read: <strong>which parts did a framework already give you, and which did it hide?</strong> Article 9 puts the custom ReAct loop next to LangGraph and compares them fairly, including where the framework wins.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3f427f977eaf" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/sandboxing-agent-generated-code-3f427f977eaf">Sandboxing Agent-Generated Code</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Durable Runs, Replay, and Auditability]]></title>
            <link>https://python.plainenglish.io/durable-runs-replay-and-auditability-28f35870bc72?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/28f35870bc72</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agent-harness]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Tue, 13 Oct 2026 06:56:33 GMT</pubDate>
            <atom:updated>2026-10-13T06:56:33.830Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Article 7 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>. Previous: </em><a href="https://medium.com/@luisevalencia/context-is-not-memory-80b40eb7e047"><em>Context Is Not Memory</em></a><em>. Articles </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>1</em></a><em>, </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>2</em></a><em>, </em><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87"><em>3</em></a><em>, 4, and </em><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1"><em>5</em></a><em> are also published.</em></p><h3>The problem</h3><p>A customer says: “Your agent deleted the wrong file on Tuesday.” You open the logs. You find a stream of text lines, some from the model provider, some from your framework, some from your own print statements. Nothing says which tool call came first, whether a human approved it, or whether the run actually finished.</p><p>Most teams respond with “we have logs.” Logs are not a run record. A log line tells you something happened. It does not tell you the sequence was complete, the run reached a terminal state, or that nobody edited the evidence afterwards.</p><p><strong>Thesis:</strong> A run is not durable because logs exist. Terminal state, event ordering, and export integrity each need an explicit contract.</p><h3>Mental model</h3><p>Think of an aircraft flight recorder, not a CCTV camera.</p><p>A flight recorder does not film everything. It records a fixed set of control signals, in order, in a box that can only be written while the flight is active and is sealed after landing. Investigators do not re-fly the plane. They read the box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2JooZV0qsVOfldbtX7sMqA.png" /></figure><p>Four contracts hold this together:</p><ol><li><strong>What is recorded</strong> , an allow-list, not “everything”.</li><li><strong>In what order</strong>, a database-allocated sequence, not a timestamp.</li><li><strong>When it stops</strong>, a terminal state that rejects further writes.</li><li><strong>Whether it can be trusted later</strong> , checksums verified on every read.</li></ol><p>The rest of this article walks through each one.</p><h3>Contract 1: record an allow-list, never the raw payload</h3><p>The first instinct is to store every prompt, every tool argument, every result. That makes the ledger a second copy of your most sensitive data.</p><p>Cogentrex goes the other way. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_ledger.py#L224">safe_event_payload</a> projects each event down to fields declared in <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_ledger.py#L31">_SAFE_FIELDS</a>:</p><pre>allowed = _SAFE_FIELDS.get(kind)<br>if allowed is None:<br>    raise ValueError(&quot;unsupported runtime event kind&quot;)<br>projected = {key: payload[key] for key in allowed if key in payload}<br>if &quot;error&quot; in projected:<br>    projected[&quot;error&quot;] = bool(projected[&quot;error&quot;])</pre><p>Two details matter.</p><p>First, an unknown event kind is <strong>rejected</strong>, not stored. A new event type has to be added to the allow-list on purpose.</p><p>Second, an error is persisted as True or False, never as its message. The code comment explains why: exception messages can embed prompts, provider responses, credentials, or tool arguments.</p><p>What does a tool call look like in the ledger? Not the arguments. A hash:</p><pre>AgentEventKind.TOOL_CALL_COMPLETED.value: frozenset(<br>    {&quot;id&quot;, &quot;name&quot;, &quot;arguments_hash&quot;, &quot;output_hash&quot;, &quot;output_size&quot;, &quot;status&quot;}<br>),</pre><p>You can prove <em>which</em> call ran and that its output had a given fingerprint, without keeping the content.</p><p>The test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_run_ledger.py#L77">test_redaction_and_allowlist_leave_no_raw_sensitive_payload</a> feeds an email, a password, and a raw result into a tool_call_completed event and asserts none of them appear in the stored row.</p><p><strong>Trade-off:</strong> you lose the ability to replay <em>content</em>. You keep the ability to answer “what happened, in what order, under which policy decision.” That is the right trade for audit. It is the wrong one if you wanted a debugger that re-runs the model.</p><h3>Contract 2: order comes from the database, not the clock</h3><p>Timestamps lie. Two events in the same millisecond have no order. Two workers have skewed clocks.</p><p>Cogentrex gives every run a counter, RunRow.next_sequence, and increments it atomically inside the same transaction that inserts the event. The relevant lines in <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_ledger.py#L401">RunRepository.append</a>:</p><pre>allocated = await session.execute(<br>    update(RunRow)<br>    .where(RunRow.run_id == run_id, RunRow.user_id == user_id,<br>           RunRow.status == &quot;running&quot;, RunRow.completed_at.is_(None))<br>    .values(next_sequence=RunRow.next_sequence + 1)<br>    .returning(RunRow.next_sequence)<br>)</pre><p>The UPDATE ... RETURNING is the allocator. The WHERE clause is the guard: the run must belong to this owner and still be running. If zero rows match, nothing is appended.</p><p>The test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_run_ledger.py#L40">test_concurrent_append_is_unique_contiguous_and_restart_safe</a> fires 30 appends with asyncio.gather, builds a fresh repository to simulate a restart, and asserts the sequences are exactly 1..30.</p><p><strong>Limit, stated plainly:</strong> the sequence orders events <em>within one run</em>. It is not a global clock across runs, workers, or regions. Do not use it to claim “run A happened before run B.”</p><h3>Contract 3: a terminal state that refuses writes</h3><p>A durable run needs an end. Without one, “completed” is a guess.</p><p>When the runtime emits run_stopped, the same transaction that inserts the event also flips the run row to completed, failed, or cancelled, stamps completed_at, and records token totals and latency. The status derivation is three lines:</p><pre>status=(&quot;cancelled&quot; if reason == &quot;cancelled&quot;<br>        else &quot;failed&quot; if error else &quot;completed&quot;)</pre><p>After that, the WHERE guard from Contract 2 no longer matches. Any further append raises ValueError(&quot;run is not running&quot;).</p><p>Why does this matter for audit? Because a record that can still grow after completion is not evidence. It is a draft.</p><p>The test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_run_ledger.py#L126">test_append_after_terminal_status_is_rejected_without_allocating_sequence</a> goes further than &quot;it raises.&quot; It checks next_sequence did not move and no extra event row exists. A rejected write must leave no trace.</p><p>There is also <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_run_ledger.py#L159">test_concurrent_append_racing_finalize_is_linearizable</a>: an append racing the terminal write either lands before the stop or is rejected. It cannot land after.</p><p>This connects to Article 5. The stop reason you saw there is the value that ends up in stop_reason here. The state machine decides; the ledger remembers.</p><h3>Contract 4: exports you can verify later</h3><p>A ledger inside your database is useful to you. An auditor, a customer, or a support engineer needs a file.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_exports.py#L126">RunExportService.create_export</a> builds an immutable bundle with seven sections: manifest, run, events, context, citations, evaluations, omissions.</p><p>Look at that last one. The export <strong>declares what it left out</strong>:</p><pre>omissions = [&quot;raw_prompts&quot;, &quot;raw_tool_arguments&quot;, &quot;raw_tool_results&quot;,<br>             &quot;memory_plaintext&quot;, &quot;secrets&quot;, &quot;chain_of_thought&quot;,<br>             &quot;artifacts&quot;, &quot;approval_credentials&quot;]</pre><p>An honest audit artifact lists its own blind spots. If events were truncated at the 10,000-event cap, an events_after_sequence_N entry is appended.</p><p>Each section gets a SHA-256 over canonical JSON (sorted keys, fixed separators). The manifest holds those checksums plus a checksum of itself. Then, on <strong>every download</strong>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_exports.py#L391">_verify_bundle</a> re-checks:</p><ul><li>the bundle has exactly the expected sections and manifest keys,</li><li>the manifest binds to this export id, run id, and creation time,</li><li>the manifest checksum matches, using hmac.compare_digest,</li><li>the disclosure scanner finds nothing to redact on a re-scan,</li><li>every section checksum matches.</li></ul><p>Any failure raises ExportIntegrityError, which the route turns into HTTP 409.</p><p>The security tests make this concrete. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_run_exports.py#L162">test_disclosure_rescan_rejects_tampered_export</a> edits the stored bundle directly in the database, injecting api_key=supersecretvalue into the answer summary, then asserts the download returns 409. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_run_exports.py#L223">test_manifest_tamper_is_rejected</a> does the same for manifest edits, parametrized by tamper kind.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1CQcVcslDFFL3gvsfBlRnA.png" /></figure><h3>Replay is not re-execution</h3><p>The word “replay” overpromises. Here is what Cogentrex means by it.</p><p><strong>Replay</strong> reads stored safe events back in order. It does not call the model again.</p><p><strong>Fork</strong> is the interesting one. <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_ledger.py#L592">RunRepository.fork</a> takes a run and a source_sequence, validates that the event exists, snapshots the conversation messages up to that event&#39;s timestamp (redacted, capped at 10,000 characters each), and creates a checkpoint plus a fork draft. The next run in that conversation consumes the draft and records parent_run_id and fork_source_sequence.</p><p>The checkpoint id is deterministic: a uuid5 over user, run, and sequence. Forking the same point twice reuses the checkpoint but yields distinct drafts. The integration tests in <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/integration/test_run_fork_compare.py">test_run_fork_compare.py</a> cover owner scoping, bad sequences, the event-time cutoff, repeated and concurrent forks, and a draft being consumed exactly once even when two child runs race for it.</p><p>And the boundary, hard-coded in the checkpoint:</p><pre>&quot;workspace_restoration&quot;: &quot;none&quot;,</pre><p>A fork restores conversation state. It does <strong>not</strong> restore files, sandbox state, or external side effects. If your agent wrote to a database in step 4, forking at step 3 does not undo it.</p><h3>Evidence</h3><p>I ran the three relevant suites against the dev branch:</p><pre>tests/unit/test_run_ledger.py              10 passed<br>tests/security/test_run_exports.py          6 passed<br>tests/integration/test_run_fork_compare.py  5 passed<br>21 passed</pre><p>These are deterministic tests on SQLite fixtures. They prove the contracts hold in that setup.</p><p>I am not claiming production durability evidence from them. For PostgreSQL behavior, see the migration test <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/integration/test_run_export_migration.py">test_run_export_migration.py</a> and the canonical <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/IMPLEMENTATION-EVIDENCE.md">implementation evidence</a>.</p><h3>What frameworks provide</h3><p>Frameworks give you something different, and it is worth being fair about it.</p><p>Graph frameworks such as LangGraph offer <strong>checkpointing</strong>: persist graph state so a run can pause and resume, or time-travel to an earlier state and re-execute. That is a <em>recovery</em> feature. It answers “continue this run.”</p><p>Cogentrex’s ledger answers a different question: “prove what this run did.” It stores hashes and metadata, not resumable state, and it deliberately cannot re-execute.</p><p>These are complementary. If you need resumable long-running workflows, you want checkpointing. If you need an audit trail you can hand to a third party, you need something with the four contracts above. Check whether your framework’s checkpoint store gives you an allow-list, a terminal state that refuses writes, and verifiable exports. Often it does not, because that was never its job.</p><h3>Failure modes and limitations</h3><ul><li><strong>Not tamper-proof.</strong> Checksums detect modification <em>of an export or stored bundle</em>. Someone with database write access can still alter the ledger events and recompute. There is no external anchor, signature chain, or WORM storage. The concept documentation says the same: the ledger is “neither chain-of-thought, a full transcript, tamper-proof storage, nor semantic truth.”</li><li><strong>Hashes are not content.</strong> You can verify a tool output matches a hash if you have the content. You cannot recover it.</li><li><strong>Per-run ordering only.</strong> No cross-run causality.</li><li><strong>Event cap.</strong> Exports stop at 10,000 events and declare the truncation.</li><li><strong>No workspace restore on fork.</strong> Side effects stay.</li><li><strong>Evaluation summaries.</strong> The export pulls evaluation rows for the owner and filters by run id in Python; that is fine at this scale and worth revisiting if evaluation volume grows.</li></ul><h3>Reader exercise</h3><ol><li>Open <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/run_ledger.py#L31">_SAFE_FIELDS</a>. Pick one event kind and decide: what field would you <em>want</em> to add for debugging? What would it leak?</li><li>Write a test that appends to a run after run_stopped and asserts both the exception and an unchanged next_sequence. Compare with the existing one.</li><li>Tamper with an export events entry in a test database. Which check in _verify_bundle fires first? Why does the order of checks matter?</li><li>Design: what would you add to make the ledger tamper-evident against a database administrator? (Hint: think hash chaining and external anchoring, then think about what that costs.)</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Concept:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/run-ledger.md">Run Ledger</a></li><li><strong>Concept:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/replay-fork-compare.md">Replay, fork, compare</a></li><li><strong>Concept:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/durable-approvals.md">Durable approvals</a></li><li><strong>Walkthrough:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/code-walkthroughs/run-ledger.md">Run ledger code walkthrough</a></li></ul><h3>Series navigation</h3><ul><li><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratc</a>h</li><li><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a></li><li><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87">3. Tool Contracts: The Boundary Between Intent and Execution</a></li><li><a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5">4. Why Tool Access Is Not Authorization</a></li><li><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1">5. State Machines, Stop Reasons, and Bounded Execution</a></li><li><a href="https://medium.com/@luisevalencia/context-is-not-memory-80b40eb7e047">6. Context Is Not Memory</a></li><li><strong>7. Durable Runs, Replay, and Auditability</strong> — <strong>this article</strong></li></ul><p>Sandboxing Agent-Generated Code</p><p><em>Full series plan: 30 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan</em></a><em>.</em></p><h3>What comes next</h3><p>The ledger tells you what the agent did. It does not stop the agent from doing something dangerous in the first place. The next article covers <strong>sandboxing agent-generated code</strong>: what “network none” and a seccomp profile actually enforce, and what they do not.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28f35870bc72" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/durable-runs-replay-and-auditability-28f35870bc72">Durable Runs, Replay, and Auditability</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Is Not Memory]]></title>
            <link>https://medium.com/@luisevalencia/context-is-not-memory-80b40eb7e047?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/80b40eb7e047</guid>
            <category><![CDATA[agent-harness]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Fri, 09 Oct 2026 13:38:10 GMT</pubDate>
            <atom:updated>2026-10-09T13:38:10.581Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Article 6 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>. Previous: </em><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1"><em>State Machines, Stop Reasons, and Bounded Execution</em></a><em>. Articles </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>1</em></a><em>, </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>2</em></a><em>, </em><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87"><em>3</em></a><em>, </em><a href="https://medium.com/@luisevalencia"><em>4</em></a><em> and </em><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1"><em>5</em></a><em> are also published.</em></p><h3>The problem</h3><p>Your agent has a 200K context window. Sounds enormous. Then a real conversation happens:</p><ul><li>The system prompt uses 4K tokens</li><li>The user sends a 2K document</li><li>The agent calls six tools, each returning 3K of output</li><li>Five iterations of back-and-forth adds 10K more</li><li>After twenty minutes, you’re at 34K and accelerating</li></ul><p>At some point, the window fills up. What happens next depends entirely on your harness. Most agents do one of three things: crash with a context length error, silently drop older messages, or rush to finish before running out of space , what researchers call “context anxiety.”</p><p>All three are symptoms of the same confusion: <strong>treating context and memory as the same thing.</strong></p><p><strong>Thesis:</strong> Context, history, retrieval, and durable encrypted memory solve different problems and should not be collapsed into one abstraction.</p><h3>Mental model</h3><p>Think of four separate physical spaces:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a58VAlowj2F4fu-N9LPwiA.png" /></figure><p>Four separate systems. Four separate codebases. Four separate test suites. Collapsing them into “memory” is like calling your desk, your filing cabinet, your bookshelf, and your journal all “storage.”</p><h3>System 1: The context window</h3><p>The context window is everything the model sees on this call, system prompt, message history, tool results, retrieved documents. It has a hard limit (200K tokens for most providers) and it fills up.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/context_filters.py#L19-L43">ContextBudget</a> tracks how the window is allocated:</p><pre>@dataclass(frozen=True, slots=True)<br>class ContextBudget:<br>    total: int = 200_000<br>    system_prompt: int = 8_000<br>    history: int = 120_000<br>    tool_results: int = 40_000<br>    retrieval: int = 24_000<br>    reserve: int = 8_000</pre><p>Each block has a ceiling. If tool results consume 45K tokens, the budget rejects them before they reach the model, not after. The engine checks this on every iteration:</p><pre># engine.py:330-340<br>if estimate_request_input_tokens(history, tool_definitions) &gt; context_allowance:<br>    return await self._stop(StopReason.CONTEXT_BUDGET_EXHAUSTED, ...)</pre><p>The model never sees a request that would overflow its window.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/context_filters.py#L47-L85">ContextUsage</a> tracks how much of each block is consumed and reports utilization percentages. When history hits 90% of its allocation, the compaction pipeline kicks in.</p><p>Message filters shape what enters the window:</p><pre># Filter out tool messages for summary-only contexts<br>role_filter(&quot;tool&quot;)<br># Truncate any message over 2000 chars<br>content_length_filter(max_length=2000)<br># Strip internal metadata before sending to model<br>metadata_filter(&quot;_recovery_pointer&quot;, &quot;_compaction_layer&quot;)</pre><p>These are composable functions , MessageFilter = Callable[[list[dict]], list[dict]]. Chain them to build the exact context shape you need.</p><h3>System 2: Multi-layer compaction</h3><p>When the context window fills up, compaction reduces it in three layers , each progressively more aggressive:</p><h3>Layer 1: Lossless prune — zero information loss</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py#L75-L116">lossless_prune</a> removes redundancy without losing any information:</p><ul><li><strong>Duplicate tool results</strong> , same tool, same output → replaced with a pointer: [Duplicate tool result — see earlier search call]</li><li><strong>Empty assistant messages</strong>, the model sometimes emits blank responses between tool calls</li><li><strong>Consecutive system messages</strong> , merged into one</li></ul><p>Every removed item gets a _recovery_pointer , a content hash that allows reconstruction if needed.</p><h3>Layer 2: Structured distill — truncate, don’t summarize</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py#L122-L155">structured_distill</a> truncates verbose outputs while preserving the beginning and end:</p><pre># A 10K tool result becomes:<br>&quot;[First 800 chars]... [TRUNCATED — 8400 chars removed] ...[Last 200 chars]&quot;</pre><p>No LLM involved. No interpretation. Just mechanical truncation with recovery metadata.</p><h3>Layer 3: Lossy summarize — LLM condenses old messages</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py#L160-L209">lossy_summarize</a> is the last resort. It takes older messages (everything except the most recent N), sends them to the LLM for summarization, and replaces them with a summary block:</p><pre>{<br>    &quot;role&quot;: &quot;system&quot;,<br>    &quot;content&quot;: &quot;[CONTEXT COMPACTION — 15 messages summarized]\n\n{summary}\n\n&quot;<br>               &quot;[End of summary — 10 recent messages follow]&quot;,<br>    &quot;_compaction_layer&quot;: &quot;lossy_summarize&quot;,<br>    &quot;_recovery_pointers&quot;: [&quot;hash1&quot;, &quot;hash2&quot;, ...]<br>}</pre><p>If the LLM summarization fails, an <strong>extractive fallback</strong> kicks in , the last 5 messages are concatenated by their first 100 characters. Degraded, but functional.</p><h3>Circuit breaker</h3><p>The full pipeline <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py#L215">multi_layer_compact</a> orchestrates all three layers and includes a circuit breaker:</p><pre>if cycles &gt;= config.max_cycles:<br>    break  # Circuit breaker — stop compacting</pre><p>Without this, a pathological conversation could compact → still too large → compact again → still too large → infinite loop. The circuit breaker stops after a configurable number of cycles.</p><h3>System 3: Session tree — branching without losing work</h3><p>A session is not a flat list of messages. It is a <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/session_tree.py#L57">tree</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t46TpHNfHyuCzpDMeG_bKQ.png" /></figure><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/session_tree.py#L57">SessionTree</a> supports: - <strong>branch()</strong> — fork from any node, preserving the original path - <strong>switch_to()</strong> — move to a different branch without losing the current one - <strong>resume</strong> — pick up exactly where you left off</p><p>This is how Claude Code handles /resume and session forking. The context window loads the active branch. Other branches are preserved but not in the window.</p><h3>System 4: Encrypted per-conversation memory</h3><p>Memory survives between sessions. Context does not.</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/memory/encrypted_memory.py#L29">EncryptedMemoryStore</a> provides durable, encrypted storage with <strong>per-conversation key derivation</strong>:</p><pre>class EncryptedMemoryStore:<br>    def _derive_key(self, conversation_id: str) -&gt; Fernet:<br>        # HKDF derives a unique key per conversation<br>        # from a master key + conversation ID</pre><p>Each conversation gets its own encryption key derived from the master key + conversation ID. Compromising one conversation’s key does not expose others. This is the same pattern used by end-to-end encrypted messaging apps.</p><p>Memory loads into the context window at session start — but it is not the context window. It is one of many sources that populate the window.</p><h3>Why the separation matters</h3><ul><li>one model call — per window overflow</li><li>token count — reduction target</li><li>none (input) — gradual (3 layers)</li><li>consumed by — layer 3 only</li><li>no — metadata only</li></ul><p>When you collapse these into “memory,” you get agents that:</p><ul><li>Lose context because the window overflowed (no compaction)</li><li>Can’t try alternative approaches (no branching)</li><li>Start from zero every session (no durable memory)</li><li>Expose conversation data across sessions (no per-conversation encryption)</li></ul><h3>Evidence and tests</h3><p><strong>Compaction tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py">test_multi_layer_compact.py</a>):</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L41">test_removes_empty_assistant_messages</a> — proves lossless prune removes empty messages</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L52">test_deduplicates_identical_tool_results</a> — proves duplicate tool outputs are replaced with pointers</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L159">test_full_pipeline_reduces_tokens</a> — proves the full 3-layer pipeline reduces token count</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L173">test_circuit_breaker_trips</a> — proves compaction stops after max cycles</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L200">test_stats_contain_recovery_pointers</a> — proves recovery metadata is preserved</li></ul><p><strong>Context filter tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_context_filters.py">test_context_filters.py</a>):</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_context_filters.py#L34">test_over_allocated_budget</a> — proves budget validation catches over-allocation</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_context_filters.py#L59">est_exceeds</a> — proves usage tracking detects violations</li></ul><p><strong>Session tree tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_session_tree.py">test_session_tree.py</a>): - Branching, switching, message isolation, and node counting</p><pre>cd backend &amp;&amp; uv run pytest tests/unit/test_multi_layer_compact.py tests/unit/test_context_filters.py tests/unit/test_session_tree.py -v</pre><h3>What frameworks provide</h3><p><strong>LangChain:</strong> Has ConversationBufferWindowMemory, ConversationSummaryMemory, and similar. These collapse context management and memory into one class. Practical for simple cases, but you can&#39;t separate compaction strategy from memory persistence.</p><p><strong>LangGraph:</strong> Checkpointing gives you session persistence, and you can implement compaction as a graph node. The separation is possible but not built-in — you architect it yourself.</p><p><strong>Claude Code:</strong> Has the best implementation of this separation. Five-layer compaction pipeline, session resume/fork, auto-memory from corrections, and 200-line/25KB caps. Cogentrex’s design was directly informed by studying Claude Code’s approach.</p><p><strong>Microsoft Agent Framework (MAF):</strong> Foundry Memory gives you managed session state with persistence. The Agent Harness handles context management. The separation exists but across two different services rather than in-process.</p><h3>When I would use a framework</h3><p>When conversations are short (under 20 messages) and sessions don’t need to persist — a framework’s built-in memory is sufficient. Don’t build a compaction pipeline for a chatbot that resets every time.</p><p>When you need:</p><ul><li>Conversations that last hundreds of messages without degradation</li><li>The ability to branch and explore alternatives</li><li>Memory that survives across sessions and deploys</li><li>Per-conversation encryption for compliance</li><li>Compaction that degrades gracefully (lossless → distill → summarize → circuit breaker)</li></ul><p>…you need the four-system separation. Context, compaction, sessions, and memory are not the same thing.</p><h3>Limitations</h3><ul><li>Lossy summarization uses the same LLM provider as the agent. If the provider is down, summarization falls back to extractive (degraded quality).</li><li>Token estimation uses tiktoken, not exact provider counts. Budget thresholds may trigger slightly early or late.</li><li>The session tree is in-memory. Persistence requires serialization that is not yet implemented as a durable store.</li><li>Encrypted memory uses Fernet (AES-128-CBC). For enterprises requiring AES-256 or HSM-backed keys, the encryption layer needs replacement.</li><li>Recovery pointers allow reconstruction in theory, but no automated recovery tool exists yet — you must manually look up hashes.</li></ul><h3>Reader exercise</h3><ol><li>Clone the repository: git clone <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></li><li>Run cd backend &amp;&amp; uv sync --extra dev &amp;&amp; uv run pytest tests/unit/test_multi_layer_compact.py -v</li><li>Read <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L173">test_circuit_breaker_trips</a> — it creates a conversation that&#39;s always too large and proves compaction stops after max_cycles instead of looping forever</li><li>Read <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_multi_layer_compact.py#L52">test_deduplicates_identical_tool_results</a> — it calls the same tool twice with the same output and proves the duplicate is replaced with a pointer</li><li>Open <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py#L215">multi_layer_compact.py:215</a> — trace how the three layers compose: lossless first, then distill, then lossy only if still over threshold</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Understand:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/context-windows.md">Context windows concept</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/context-compaction.md">Context compaction</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/session-branching.md">Session branching</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/encrypted-memory.md">Encrypted memory</a></li><li><strong>Follow:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/modules/06-context-and-memory/README.md">Module 06 — Context and memory</a></li><li><strong>Walk through code:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/code-walkthroughs/context-compaction.md">Context compaction walkthrough</a></li><li><strong>See:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/visual-learning/modules/context-compaction/">Context compaction Visual Learning pack</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/visual-learning/modules/session-branching/">Session branching Visual Learning pack</a></li><li><strong>Inspect:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/multi_layer_compact.py">multi_layer_compact.py</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/context_filters.py">context_filters.py</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/services/session_tree.py">session_tree.py</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/memory/encrypted_memory.py">encrypted_memory.py</a></li><li><strong>Verify:</strong> cd backend &amp;&amp; uv run pytest tests/unit/test_multi_layer_compact.py tests/unit/test_context_filters.py tests/unit/test_session_tree.py -v</li><li><strong>Continue:</strong> Article 7 — Durable Runs, Replay, and Auditability</li></ul><h3>Series navigation</h3><ul><li><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratch</a></li><li><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a></li><li><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87">3. Tool Contracts: The Boundary Between Intent and Execution</a></li><li><a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5">4. Why Tool Access Is Not Authorization</a></li><li><a href="https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1">5. State Machines, Stop Reasons, and Bounded Execution</a></li><li><strong>6. Context Is Not Memory</strong>, <strong>this article</strong></li></ul><p><em>Full series plan: 30 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan</em></a><em>.</em></p><h3>What comes next</h3><p>The next article examines what happens AFTER a run completes: <strong>durable runs, replay, and auditability.</strong> An append-only run ledger that records every event, tool call, and decision — so you can replay any run and prove exactly what happened.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80b40eb7e047" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[State Machines, Stop Reasons, and Bounded Execution]]></title>
            <link>https://medium.com/@luisevalencia/state-machines-stop-reasons-and-bounded-execution-e9847be6a6f1?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/e9847be6a6f1</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[agent-harness]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Tue, 06 Oct 2026 11:23:36 GMT</pubDate>
            <atom:updated>2026-10-06T11:23:36.585Z</atom:updated>
            <content:encoded><![CDATA[<h3>State Machines, Stop Reasons, and Bounded Execution</h3><p><em>This is Article 5 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>.</em></p><p><em>Previous: </em><a href="https://medium.com/@luisevalencia"><em>Why Tool Access Is Not Authorization</em></a><em>.</em></p><p><em>Articles </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>1</em></a><em>, </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>2</em></a><em>, </em><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87"><em>3</em></a><em> and </em><a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5"><em>4</em></a><em> are also published.</em></p><h3>The problem</h3><p>Your agent finished a run. Someone asks: “Why did it stop?”</p><p>If the answer is “it just… stopped”, you have a problem. You cannot debug what you cannot name. You cannot alert on “something went wrong.” You cannot build dashboards on “it ended somehow.”</p><p>Most agent loops have two states: running and done. Maybe three if you count errors. That is not enough for production. A run that completed successfully, a run that hit its token limit, a run that was denied by policy, and a run where the human never responded to an approval request , these are four fundamentally different outcomes that require different responses from your system.</p><p><strong>Thesis:</strong> Explicit, named stop reasons make agent behavior operable, testable, and explainable. Every run must end with a typed reason, never a silent exit.</p><h3>Mental model</h3><p>Think of the agent loop as a traffic intersection with twenty possible exits. Every car that enters MUST leave through a named exit. No car disappears. No car sits at the intersection forever. And every exit has a camera recording which car left through which gate and when.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HDeFmIR-WeRVrlybbd464g.png" /></figure><p>Twenty exits. Zero silent failures. Every run produces a typed result that tells you exactly what happened.</p><h3>The three primitives</h3><p>A bounded agent loop needs three typed primitives working together:</p><h3>1. StopReason , why it ended</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L70-L90">StopReason</a> is a StrEnum with twenty values:</p><pre>class StopReason(StrEnum):<br>    COMPLETED = &quot;completed&quot;<br>    CANCELLED = &quot;cancelled&quot;<br>    ITERATION_BUDGET_EXHAUSTED = &quot;iteration_budget_exhausted&quot;<br>    TOOL_BUDGET_EXHAUSTED = &quot;tool_budget_exhausted&quot;<br>    TOKEN_BUDGET_EXHAUSTED = &quot;token_budget_exhausted&quot;<br>    CONTEXT_BUDGET_EXHAUSTED = &quot;context_budget_exhausted&quot;<br>    TIME_BUDGET_EXHAUSTED = &quot;time_budget_exhausted&quot;<br>    POLICY_DENIED = &quot;policy_denied&quot;<br>    APPROVAL_TIMEOUT = &quot;approval_timeout&quot;<br>    APPROVAL_UNAVAILABLE = &quot;approval_unavailable&quot;<br>    PROVIDER_CAPABILITY_UNSUPPORTED = &quot;provider_capability_unsupported&quot;<br>    STRUCTURED_OUTPUT_INVALID = &quot;structured_output_invalid&quot;<br>    PROVIDER_LENGTH_LIMIT = &quot;provider_length_limit&quot;<br>    PROVIDER_REFUSAL = &quot;provider_refusal&quot;<br>    PROVIDER_CONTENT_FILTER = &quot;provider_content_filter&quot;<br>    MONETARY_BUDGET_EXHAUSTED = &quot;monetary_budget_exhausted&quot;<br>    MODEL_CHARGE_DUPLICATE = &quot;model_charge_duplicate&quot;<br>    MODEL_CHARGE_INDETERMINATE = &quot;model_charge_indeterminate&quot;<br>    PROVIDER_ERROR = &quot;provider_error&quot;<br>    ERROR = &quot;error&quot;</pre><p>Why twenty? Because each triggers a different response:</p><ul><li>COMPLETED → success, record result</li><li>ITERATION_BUDGET_EXHAUSTED → force final synthesis, then stop</li><li>MONETARY_BUDGET_EXHAUSTED → alert billing, block future runs</li><li>POLICY_DENIED → log security event, notify admin</li><li>APPROVAL_TIMEOUT → retry with escalation, or abandon</li><li>PROVIDER_REFUSAL → content filter triggered, redact and retry</li></ul><p>A generic “error” tells you nothing. A named stop reason tells you exactly what happened and what to do next.</p><h3>2. RuntimeBudget , the immutable fence</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L93-L104">RuntimeBudget</a> is a frozen dataclass , set once, enforced on every iteration, impossible to modify mid-run:</p><pre>@dataclass(frozen=True, slots=True)<br>class RuntimeBudget:<br>    max_iterations: int = 15<br>    max_tool_calls: int = 20<br>    max_tokens: int = 64_000<br>    max_seconds: float = 300.0<br>    max_tool_result_chars: int = 12_000<br>    final_synthesis_tokens: int = 2_048<br>    max_structured_retries: int = 1<br>    max_context_tokens: int = 200_000<br>    context_output_reserve_tokens: int = 4_096</pre><p>Nine parameters. Each one is a hard boundary that the model cannot negotiate with. frozen=True means Python raises FrozenInstanceError if anything tries to modify the budget after construction.</p><h3>3. AgentResult , the typed receipt</h3><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L120-L127">AgentResult</a> is what every run returns , no exceptions:</p><pre>class AgentResult:<br>    content: str                          # the answer (may be empty)<br>    stop_reason: StopReason               # WHY it stopped<br>    iterations: int                       # how many loops<br>    tool_calls: tuple[dict[str, Any], ...]  # what it did<br>    usage: TokenUsage                     # what it cost<br>    error: str | None = None              # sanitized error if any<br>    structured_output: object | None = None  # typed output if contracted</pre><p>The caller never gets a bare string. They get a structured receipt with the reason, the cost, the iteration count, and every tool call made. This is what makes the loop operable , upstream systems can branch on result.stop_reason without parsing error messages.</p><h3>The event timeline</h3><p>Stop reasons tell you WHY. Events tell you WHEN and IN WHAT ORDER. The engine emits <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/events.py#L13-L39">25 typed events</a> at every lifecycle point:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/874/1*ZyeO-sgrKL4q8nUIv0Q23g.png" /></figure><p>Every event carries: the event kind, the iteration number, a data payload, and token usage. They stream via SSE to the frontend in real-time and are persisted in the run ledger for replay.</p><h3>The deadline enforcer</h3><p>Time budgets need special treatment. You cannot rely on checking time.time() at the top of each iteration, a single model call or tool execution could take 60 seconds, blowing past your budget.</p><p>Cogentrex uses a <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/deadline.py#L15">DeadlineExceededError</a> mechanism with asyncio.wait_for:</p><pre># engine.py:250-285 (simplified)<br>async def run(self, messages, ...) -&gt; AgentResult:<br>    started_at = self._clock()<br>    deadline = started_at + self._budget.max_seconds<br>    try:<br>        return await await_before_deadline(<br>            self._run(messages, ...),<br>            deadline=deadline,<br>        )<br>    except asyncio.CancelledError:<br>        return await self._stop(StopReason.CANCELLED, ...)<br>    except DeadlineExceededError:<br>        return await self._deadline_result(state)</pre><p>The await_before_deadline wrapper enforces the wall-clock deadline at the asyncio level , not just at iteration boundaries. If a provider call hangs for 5 minutes, the deadline fires and the run stops with TIME_BUDGET_EXHAUSTED.</p><p>The _deadline_result handler is careful: it captures whatever partial state exists (content, iterations, tool calls) and returns a valid AgentResult. A timed-out run is NOT an unhandled exception , it is a typed, inspectable result.</p><h3>The _stop method , one exit point</h3><p>Every stop reason flows through a single <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L1921">_stop</a> method:</p><pre>async def _stop(self, reason, content, iterations, calls, usage, error=None, ...):<br>    # 1. Enforce compliance on output<br>    if self._compliance is not None:<br>        content = self._compliance.enforce_output(content)<br>    # 2. Emit terminal event<br>    await self._emit(AgentEventKind.RUN_STOPPED, iterations, {<br>        &quot;stop_reason&quot;: reason.value,<br>        &quot;error&quot;: error,<br>    })<br>    # 3. Record result in ledger<br>    if record_result and self._recorder:<br>        await self._recorder(content)<br>    # 4. Return typed result<br>    return AgentResult(content, reason, iterations, tuple(calls), usage, error)</pre><p>One method. Four responsibilities. Every run passes through it, success, failure, timeout, cancellation. This guarantees that compliance is always enforced, the terminal event is always emitted, and the result is always recorded.</p><h3>Evidence and tests</h3><ul><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_v2.py#L145">test_explicit_budget_stop_reasons</a> — parametrized test proving iteration, tool, and token budgets each trigger the correct StopReason</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_v2.py#L154">test_tool_budget_exhaustion_balances_pending_tool_use_before_final_synthesis</a> — proves the engine closes pending tool calls before forcing synthesis</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_v2.py#L221">test_timeout_stop_reason</a> — proves wall-clock deadline enforcement returns TIME<em>BUDGET</em>EXHAUSTED</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_v2.py#L76">test_cancellation_persists_terminal_event_without_recording_empty_answer</a> — proves cancelled runs still emit the terminal event</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_v2.py#L191">test_final_synthesis_failure_is_not_silently_reported_as_success</a> — proves a failed synthesis is never reported as COMPLETED</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_observability.py#L110">test_exact_event_to_metric_span_and_correlation_mapping</a> — proves every event maps to a traceable span</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_runtime_observability.py#L224">test_cancelled_runtime_is_terminal_in_durable_ledger</a> — proves cancelled runs are persisted, not lost</li></ul><pre>cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py tests/unit/test_runtime_observability.py tests/unit/test_runtime_budget_regressions.py -v</pre><h3>What frameworks provide</h3><p><strong>LangGraph:</strong> Runs can be interrupted and resumed via checkpoints, which is powerful. But the stop reason is typically “the graph reached an end node” or “an error occurred” , not twenty named outcomes. You build your own stop taxonomy on top.</p><p><strong>CrewAI:</strong> Tasks complete or fail. There’s no concept of “iteration budget exhausted” vs “monetary budget exhausted” , both are failures from the framework’s perspective.</p><p><strong>AutoGen:</strong> Conversations end when a termination message is detected. The termination condition is customizable, but the result is a message, not a typed enum. Distinguishing termination reasons requires parsing conversation content.</p><p><strong>Microsoft Agent Framework (MAF):</strong> The Agent Harness has structured outcomes and budget enforcement built in. This is the closest to Cogentrex’s approach , MAF was designed with the same insight that every exit needs a name.</p><h3>When I would use a framework</h3><p>When your system only needs to know “success or failure” , a framework’s built-in termination is sufficient. Many applications only care about the final answer.</p><p>When your system needs to:</p><ul><li>Route alerts based on WHY a run stopped</li><li>Build dashboards showing budget exhaustion trends</li><li>Auto-scale based on timeout frequency</li><li>Trigger different retry strategies for different failures</li><li>Prove to compliance that a denied run was actually denied</li></ul><p>…you need typed stop reasons. Either build them or extend your framework’s output with a typed envelope.</p><h3>Limitations</h3><ul><li>Twenty stop reasons is a fixed enum. Adding a new reason requires a code change and migration of any downstream consumers.</li><li>The _stop method runs compliance enforcement synchronously. A slow compliance check delays the terminal event.</li><li>Token counting is estimated (tiktoken), not exact provider counts. Budget enforcement may trigger slightly early or late.</li><li>The deadline enforcer depends on asyncio.wait_for. CPU-bound tool handlers that don&#39;t yield to the event loop can temporarily exceed the deadline.</li><li>Event persistence is append-only with a budget-based pruning strategy. Very long sessions with many events consume growing storage.</li></ul><h3>Reader exercise</h3><ol><li>Clone the repository: git clone <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></li><li>Run cd backend &amp;&amp; uv sync --extra dev &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py::test_explicit_budget_stop_reasons -v</li><li>Read the test. It uses @pytest.mark.parametrize to prove that three different budgets (iteration, tool, token) each produce the correct StopReason</li><li>Now run test_timeout_stop_reason — it advances the clock past max_seconds and asserts TIME_BUDGET_EXHAUSTED</li><li>Open <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L1921">engine.py:1921</a> — the _stop method. Trace how compliance, events, recording, and result construction all flow through this single exit point</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Understand:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/state-machines.md">State machines and stop reasons</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/typed-runtime.md">Typed runtime</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/cost-usage-budgets.md">Cost and usage budgets</a></li><li><strong>Follow:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/modules/02-typed-runtime/README.md">Module 02 — Typed runtime</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/modules/03-react-loop/README.md">Module 03 — ReAct loop</a></li><li><strong>Walk through code:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/code-walkthroughs/runtime.md">Runtime code walkthrough</a></li><li><strong>See:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/visual-learning/packs/system-overview.json">System overview Visual Learning pack</a></li><li><strong>Inspect:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L70-L90">engine.py — StopReason</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/engine.py#L1921">engine.py — _stop</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/runtime/events.py#L13-L39">events.py — AgentEventKind</a></li><li><strong>Verify:</strong> cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py tests/unit/test_runtime_observability.py -v</li><li><strong>Continue:</strong> Article 6 — Context Is Not Memory</li></ul><h3>Series navigation</h3><p><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratch</a></p><p><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a></p><p><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87">3. Tool Contracts: The Boundary Between Intent and Execution</a></p><p><a href="https://medium.com/python-in-plain-english/why-tool-access-is-not-authorization-6de47238e7b5">4. Why Tool Access Is Not Authorization</a></p><p><strong>5. State Machines, Stop Reasons, and Bounded Execution</strong></p><p>6. Context Is Not Memory (next)</p><p><em>Full series plan: 30 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan</em></a><em>.</em></p><h3>What comes next</h3><p>The next article tackles a confusion that costs teams weeks of debugging: <strong>context is not memory.</strong> The context window is what the model sees RIGHT NOW. Memory is what persists between sessions. Treating them as the same thing leads to agents that forget everything, repeat themselves, or crash when the window fills up.</p><p>We will trace how multi-layer compaction, session branching, and encrypted per-conversation memory create three separate systems from what most people call “context.”</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e9847be6a6f1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Tool Access Is Not Authorization]]></title>
            <link>https://python.plainenglish.io/why-tool-access-is-not-authorization-6de47238e7b5?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/6de47238e7b5</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[agent-harness]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Mon, 05 Oct 2026 09:24:44 GMT</pubDate>
            <atom:updated>2026-10-05T09:29:54.269Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Article 4 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>.</em></p><p><em>Previous: </em><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87"><em>Tool Contracts: The Boundary Between Intent and Execution</em></a><em>.</em></p><p><em>Articles </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>1</em></a><em> and </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>2</em></a><em> are also published.</em></p><h3>The problem</h3><p>Your agent has ten tools registered. Does that mean it can use all ten freely?</p><p>Most agent frameworks say yes. If a tool is in the tool list, the model can call it. The tool definition IS the authorization. This works fine for read-only search tools. It falls apart the moment you add write_file, send_email, or execute_sql.</p><p>The confusion is subtle: <strong>visibility is not permission.</strong> Showing the model a tool’s schema means the model knows the tool exists and can propose calls to it. It does not mean those calls should execute without checks.</p><p>In a real system, three different users calling the same tool with the same arguments might get three different outcomes: allowed, denied, or paused for human approval , depending on who’s asking, what risk rules apply, and whether the operation is reversible.</p><p><strong>Thesis:</strong> Making a tool visible to a model does not authorize its effects. Policy, approval, and execution must remain separate decisions governed by separate code.</p><h3>Mental model</h3><p>Think of it as three separate questions asked by three different people:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eYN0M7JG3oCbLj5WGG9tZA.png" /></figure><p>The registry answers: <strong>can the model see this tool?</strong><br>The policy answers: <strong>may this call proceed?</strong><br>The approval answers: <strong>does a human agree?</strong></p><p>Three questions. Three separate components. Three separate test suites.</p><h3>Why frameworks collapse these into one</h3><p>Most frameworks have a single concept: the tool list. If it’s in the list, it’s callable. This is a design choice that trades governance for simplicity.</p><p>The problem surfaces when you need:</p><ul><li>The model to see delete_record in its tool list (so it can plan) but NOT execute it without approval</li><li>read_file to auto-execute but write_file to require human confirmation</li><li>The same tool to be allowed for one project but denied for another</li><li>All tools to be visible but none to execute in “plan mode” , the model shows its plan, a human reviews, then execution proceeds</li></ul><p>None of these are possible when visibility equals authorization. You need the separation.</p><h3>The three layers in Cogentrex</h3><h3>Layer 1: Registry , what exists</h3><p>The <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/tools/registry.py#L273">SecureToolRegistry</a> holds every registered tool with its schema, risk classes, and capability ID. The model sees the tool definitions , names, descriptions, input schemas. This is visibility.</p><p>But registration includes metadata the model never sees:</p><pre># What the model sees:<br>ToolDefinition(name=&quot;write_file&quot;, description=&quot;Write content to a file&quot;,<br>               input_schema={&quot;properties&quot;: {&quot;path&quot;: ..., &quot;content&quot;: ...}})# What the harness also knows (invisible to model):<br>risk_classes=frozenset({RiskClass.WRITE, RiskClass.EXTERNAL_SIDE_EFFECT})<br>resource_resolver=workspace_path_resolver(root)<br>effectful=True<br>idempotency_key_parameter=&quot;effect_key&quot;</pre><p>The model proposes write_file(path=&quot;config.yaml&quot;, content=&quot;...&quot;). It has no idea that this tool is tagged WRITE + EXTERNAL<em>SIDE</em>EFFECT, that the path will be resolved against a workspace root, or that the call will go through the effect ledger for exactly-once semantics.</p><h3>Layer 2: Policy , deterministic rules</h3><p>The <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/policy.py#L265-L268">PolicyEngine</a> is a Protocol with one method:</p><pre>class PolicyEngine(Protocol):<br>    def evaluate(self, request: PolicyRequest) -&gt; PolicyDecision: ...</pre><p>The request carries the canonical tool name, resolved resource patterns, and risk classes. The decision is always one of ALLOW, ASK, or DENY , with the matched rule ID and reason:</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/policy.py#L250-L262">PolicyDecision</a>:</p><pre>@dataclass(frozen=True, slots=True)<br>class PolicyDecision:<br>    action: PolicyAction       # allow | ask | deny<br>    risk_classes: frozenset[RiskClass]<br>    matched_rule_id: str | None<br>    reason: str<br>    specificity: tuple[int, int, int] = (0, 0, 0)</pre><p>Rules match by tool name (exact or wildcard), resource patterns (path, hostname), and risk classes. When multiple rules match, the most specific wins. Equal specificity with conflicting actions? DENY wins , fail closed.</p><p>The policy engine is deliberately <strong>stateless and I/O-free</strong>. It takes a request, applies rules, returns a decision. No database calls, no LLM calls, no network. This makes it deterministic and testable , the same request always gets the same answer.</p><h3>Layer 3: Approval, human in the loop</h3><p>When policy returns ASK, the call enters the <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/live_approvals.py#L30">ApprovalBroker</a>. The runtime blocks. A human receives the request via SSE and decides.</p><p>The approval is <strong>cryptographically bound</strong> to the exact call:</p><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/approvals.py#L34-L56">AuthorizationRequest</a>:</p><pre>@dataclass(frozen=True, slots=True)<br>class AuthorizationRequest:<br>    tool_call_id: str          # native provider ID<br>    tool_name: str             # canonical name<br>    arguments_hash: str        # SHA-256 of canonical arguments<br>    risk_classes: frozenset[RiskClass]<br>    matched_rule_id: str | None</pre><p><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/approvals.py#L60-L84">AuthorizationOutcome</a>:</p><pre>@dataclass(frozen=True, slots=True)<br>class AuthorizationOutcome:<br>    approved: bool<br>    tool_call_id: str<br>    tool_name: str<br>    arguments_hash: str<br>    reason_code: str<br><br>    def binds(self, request: AuthorizationRequest) -&gt; bool:<br>        return (self.tool_call_id == request.tool_call_id<br>                and self.tool_name == request.tool_name<br>                and self.arguments_hash == request.arguments_hash)</pre><p>The binds() method is the integrity check. If the arguments changed between when the human saw them and when the outcome is consumed, the binding fails. This prevents a class of attack where the model proposes a safe-looking call, gets approval, then substitutes dangerous arguments.</p><h3>Graduated permission modes</h3><p>Article 3 covered binary allow/deny. But real governance needs graduation. Cogentrex implements <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/graduated_permissions.py#L22-L44">seven permission modes</a>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yZzOVoiYJ000nFPQJ-QRWg.png" /></figure><p>The same tool call gets different treatment depending on the active mode:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IwAOacFPvTm0ouApLWMr2A.png" /></figure><p>Notice the last row: .env is <strong>always blocked</strong> regardless of mode. Bypass-immune paths and commands cannot be overridden. This is the &quot;even ALLOW_ALL has limits&quot; principle.</p><p>The mode auto-downgrades when too many denials occur. Three denials in five minutes? AUTO<em>APPROVE</em>SAFE → ASK_UNKNOWN. The system gets more restrictive when it detects repeated risky behavior , without human intervention.</p><h3>Evidence and tests</h3><p>The separation is proven by two test suites:</p><p><strong>Policy engine tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_policy_engine.py">test_policy_engine.py</a>):</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_policy_engine.py#L38">test_public_enum_values_are_stable</a> — proves ALLOW, ASK, DENY are stable strings</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_policy_engine.py#L64">test_decision_is_immutable</a> — proves decisions cannot be mutated after creation</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_policy_engine.py#L84">test_resource_canonicalization</a> — proves paths and hostnames are normalized before matching</li></ul><p><strong>Approval tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_durable_live_approvals.py">test_durable_live_approvals.py</a>):</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_durable_live_approvals.py#L48">test_cross_process_prepare_decide_and_poll</a> — proves the full prepare→decide→poll lifecycle</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_durable_live_approvals.py#L143">test_exact_run_binding_and_concurrent_decision_has_one_winner</a> — proves two concurrent approvals don&#39;t double-execute</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_durable_live_approvals.py#L169">test_database_schema_has_no_raw_arguments</a> — proves raw arguments are never persisted (only hashes)</li><li><a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_durable_live_approvals.py#L120">test_expiry_is_persisted_and_returns_sanitized_denial</a> — proves expired approvals are denied, not silently ignored</li></ul><p><strong>Graduated permissions tests</strong> (<a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/unit/test_graduated_permissions.py">test_graduated_permissions.py</a>):</p><ul><li>26 tests proving all seven modes, bypass-immune paths/commands, denial tracking, and mode transformation</li></ul><pre>cd backend &amp;&amp; uv run pytest tests/security/test_policy_engine.py tests/unit/test_durable_live_approvals.py tests/unit/test_graduated_permissions.py -v</pre><h3>What frameworks provide</h3><p><strong>LangChain:</strong> Tools are functions. If registered, they’re callable. No built-in policy layer. You can add guardrails via LangSmith or external libraries, but it’s not part of the tool contract.</p><p><strong>LangGraph:</strong> Human-in-the-loop is a graph node, closer to the approval concept. But there’s no deterministic policy evaluation between “tool exists” and “human sees it.” The graph decides the flow, not a rule engine.</p><p><strong>CrewAI:</strong> Agent roles imply tool restrictions (“the researcher can search but not write”), but this is enforced through prompt instructions, not deterministic code. The model can ignore prompt-level restrictions.</p><p><strong>Microsoft Foundry Agent Service:</strong> Has built-in safety filters and managed tool governance. The rules are configured in the portal, which means they’re deterministic , but you can’t unit-test them locally or version-control them in your repo.</p><h3>When I would use a framework</h3><p>When tool governance means “don’t call tools that aren’t in the list” , a framework is sufficient. Read-only search, calculator, date/time tools don’t need per-call policy.</p><p>When tool governance means “the same tool has different authorization depending on the caller, the arguments, the risk level, and whether a human has approved this specific call” , you need the three-layer separation. Build it, or find a framework that provides it.</p><h3>Limitations</h3><ul><li>Policy rules are code-defined, not database-driven. Adding a new rule requires a code change and deployment.</li><li>The approval broker is in-process. Cross-service approval (e.g., a separate approval microservice) is not implemented.</li><li>Graduated permission modes are evaluator-scoped, not per-user. All users share the same mode in a single evaluator instance.</li><li>binds() uses argument hash comparison, not semantic comparison. Two calls with identical effect but different argument ordering produce different hashes.</li><li>No audit dashboard for reviewing past approvals. The data is in the database but there’s no UI to browse it.</li></ul><h3>Reader exercise</h3><ol><li>Clone the repository: git clone <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></li><li>Run cd backend &amp;&amp; uv sync --extra dev &amp;&amp; uv run pytest tests/security/test_policy_engine.py -v</li><li>Read <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/tests/security/test_policy_engine.py#L64">test_decision_is_immutable</a>, it creates a PolicyDecision and tries to mutate it. The frozen dataclass prevents mutation.</li><li>Run uv run pytest tests/unit/test_graduated_permissions.py::TestProtectedPaths -v — proves .env is blocked even in ALLOW_ALL mode</li><li>Open <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/policy.py#L265">policy.py:265</a> — the PolicyEngine Protocol. Notice it has exactly one method: evaluate. That&#39;s the entire contract. No I/O. No state. Pure function.</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Understand:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/policy-engine.md">Policy engine concept</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/durable-approvals.md">Durable approvals</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/graduated-security-model.md">Graduated security model</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/concepts/idempotency.md">Idempotency</a></li><li><strong>Follow:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/modules/05-policy-and-approvals/README.md">Module 05 — Policy and approvals</a></li><li><strong>Walk through code:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/course/code-walkthroughs/policy-and-approval.md">Policy and approval walkthrough</a></li><li><strong>See:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/docs/visual-learning/modules/graduated-security/">Graduated security Visual Learning pack</a></li><li><strong>Inspect:</strong> <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/policy.py#L265-L268">policy.py — PolicyEngine</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/approvals.py#L34-L56">approvals.py — AuthorizationRequest</a>, <a href="https://github.com/levalencia/cogentrex/blob/dev/backend/app/security/graduated_permissions.py#L22-L44">graduated_permissions.py — PermissionMode</a></li><li><strong>Verify:</strong> cd backend &amp;&amp; uv run pytest tests/security/test_policy_engine.py tests/unit/test_durable_live_approvals.py tests/unit/test_graduated_permissions.py -v</li><li><strong>Continue:</strong> Article 5 — State Machines, Stop Reasons, and Bounded Execution</li></ul><h3>Series navigation</h3><p><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratch</a></p><p><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a></p><p>3. Tool Contracts: The Boundary Between Intent and Execution</p><p><strong>4. Why Tool Access Is Not Authorization =&gt; this article</strong></p><p>5. State Machines, Stop Reasons, and Bounded Execution</p><p>6. Context Is Not Memory</p><p><em>Full series plan: 32 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan</em></a><em>.</em></p><h3>What comes next</h3><p>The next article zooms into <strong>state machines and stop reasons,</strong> the twenty named exit paths that make every run inspectable. We will trace how the typed StopReason enum, the RuntimeBudget frozen dataclass, and the event system work together to guarantee that no run ends silently.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6de47238e7b5" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/why-tool-access-is-not-authorization-6de47238e7b5">Why Tool Access Is Not Authorization</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tool Contracts: The Boundary Between Intent and Execution]]></title>
            <link>https://python.plainenglish.io/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/1b0ca30ded87</guid>
            <category><![CDATA[agent-harness]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[ai-agents-in-action]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Tue, 29 Sep 2026 17:48:18 GMT</pubDate>
            <atom:updated>2026-09-29T17:48:18.720Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is Article 3 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>.</em></p><p><em>Previous: </em><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981"><em>Building a Typed ReAct Loop Without a Framework</em></a><em>.</em></p><p><em>First: </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>I Built an AI Agent Harness from Scratch</em></a><em>.</em></p><h3>The problem</h3><p>The model says: <em>“Call write_file with path=/etc/passwd and content=pwned.”</em></p><p>What happens next depends entirely on your harness. In a naive agent loop, the tool executes. In a governed harness, that call never reaches the filesystem, it fails at the contract boundary.</p><p>A tool call is where probabilistic intent crosses into deterministic software. The model <em>proposes</em> an action. The harness must <em>validate</em> it before any side effect touches the world. This is not a security feature you bolt on later. It is the fundamental architecture of safe tool execution.</p><p><strong>Thesis:</strong> A tool call crosses from probabilistic intent into deterministic software and must be treated as a contract boundary, validated, gated, bounded, and recorded before execution.</p><h3>Mental model</h3><p>Think of a tool call as a package crossing a national border. Before it enters the country, it passes through four checkpoints:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nRE9EY79MkcmGkJkXY19rg.png" /></figure><p>Four gates. Every gate can stop the call. Every decision is logged. The model never bypasses the pipeline, it only sees the result after all gates pass.</p><h3>The four gates</h3><h3>Gate 1: Schema validation</h3><p>Every tool declares its input contract as a JSON Schema subset:</p><p><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L206-L262">ToolDefinition</a>:</p><pre>@dataclass(frozen=True, slots=True)<br>class ToolDefinition:<br>    name: str<br>    handler: Callable[..., Any]<br>    input_schema: Mapping[str, Any] | None = None<br>    risk_classes: frozenset[RiskClass] = frozenset()<br>    resource_resolver: ResourceResolver | None = None<br>    effectful: bool = False<br>    idempotency_key_parameter: str | None = None<br>    capability_id: str | None = None</pre><p>The schema is validated at <strong>registration time</strong> (not at call time). _validate_input_schema enforces a strict subset , only JSON types the harness can enforce. No $ref, no allOf, no unbounded recursion.</p><p>At call time, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L332-L369">_validate_arguments</a> checks every argument against the declared schema:</p><pre>@staticmethod<br>def _validate_arguments(tool, arguments):<br>    # Required fields must be present<br>    for field in required_fields:<br>        if field not in arguments:<br>            raise ValueError(f&quot;Missing required parameter: {field}&quot;)<br>    # Unknown fields fail closed<br>    if not additional_properties_allowed:<br>        if any(field not in allowed_fields for field in arguments):<br>            raise ValueError(&quot;Unexpected parameter(s) supplied&quot;)<br>    # Type checking per declared property<br>    for field, declaration in properties.items():<br>        if not _value_matches_type(value, declared_type):<br>            raise ValueError(f&quot;Invalid parameter: {field}&quot;)<br>    # Return a deep copy — never share references with the model<br>    return copy.deepcopy(dict(arguments))</pre><p>Two critical design decisions here. First: unknown fields <strong>fail closed</strong>. The model cannot sneak extra parameters past the schema. Second: the validated arguments are <strong>deep-copied</strong> before execution , preventing TOCTOU attacks where the model mutates arguments between validation and execution.</p><h3>Gate 2: Policy evaluation</h3><p>After schema validation, the call enters the <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/security/policy.py">policy engine</a>. This is pure deterministic logic , no LLM involved:</p><pre>class PolicyAction(StrEnum):<br>    ALLOW = &quot;allow&quot;   # proceed without human<br>    ASK = &quot;ask&quot;       # pause for human approval<br>    DENY = &quot;deny&quot;     # reject immediately</pre><p>The policy evaluates the tool name, argument hash, and resolved resource patterns against a rule set. The result is always one of three actions. Never “maybe.” Never “it depends on the context.” Allow, ask, or deny.</p><p>The engine uses <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/security/policy.py#L76">canonical tool names</a> and <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/security/policy.py#L52">canonical argument hashes</a> , the exact same string representation is used for policy evaluation, approval binding, and audit recording. No ambiguity.</p><h3>Gate 3: Human approval</h3><p>When policy returns ASK, the runtime <strong>blocks</strong> until a human decides. This is not a notification , it is a synchronous gate:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mgTUKS-eoB8WD8-8sDw0Ag.png" /></figure><p>The approval is <strong>cryptographically bound</strong> to the exact tool call , tool name + call ID + SHA-256 arguments hash. If the arguments change between policy evaluation and approval, the binding fails. This prevents a subtle attack: the model proposes a safe-looking call, gets approval, then substitutes dangerous arguments.</p><h3>Gate 4: Bounded execution</h3><p>After all gates pass, the tool executes with:</p><ul><li><strong>Timeout</strong> , asyncio.wait_for with per-tool configurable limits</li><li><strong>Permission check</strong> , handler-level permission validation</li><li><strong>Audit logging</strong> , every execution recorded with correlation ID</li><li><strong>Deep-copied arguments</strong> — the handler receives a copy, not a reference</li></ul><p><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L371-L410">_execute_registered</a> — the actual execution boundary:</p><pre>async def _execute_registered(self, tool_name, tool, handler_parameters, ...):<br>    # Permission check<br>    for permission in tool.required_permissions:<br>        if not await self._permissions.check(...):<br>            raise PermissionError(f&quot;Permission denied for {permission}&quot;)<br>    # Bounded execution<br>    result = await asyncio.wait_for(<br>        tool.handler(**handler_parameters),<br>        timeout=tool.timeout<br>    )</pre><h3>Risk classification</h3><p>Every tool declares its risk classes at registration:</p><pre>class RiskClass(StrEnum):<br>    READ = &quot;read&quot;<br>    WRITE = &quot;write&quot;<br>    EXECUTE = &quot;execute&quot;<br>    NETWORK = &quot;network&quot;<br>    SECRET = &quot;secret&quot;<br>    EXTERNAL_SIDE_EFFECT = &quot;external_side_effect&quot;</pre><p>Risk classes feed into:</p><ul><li><strong>Policy rules</strong> , different rules for read vs write vs execute</li><li><strong>Capability visibility</strong> , tools can be hidden based on active capabilities</li><li><strong>Effect tracking</strong> , effectful tools go through the durable effect ledger</li><li><strong>Concurrency</strong> , read-only tools can run in parallel; write tools serialize</li></ul><p>A read_file tool has risk_classes={READ}. A shell_exec has risk_classes={EXECUTE, WRITE, EXTERNAL_SIDE_EFFECT}. The harness treats them very differently , same pipeline, different policy.</p><h3>Evidence and tests</h3><p>The tool contract pipeline has extensive test coverage in <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py">test_runtime_policy.py</a> 20 test cases proving every gate works:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L73">test_allow_orders_policy_event_before_execution</a> proves policy evaluation runs BEFORE tool execution</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L98">test_policy_execution_uses_detached_nested_argument_snapshot</a> proves arguments are deep-copied to prevent TOCTOU</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L131">test_provider_calls_are_snapshotted_before_model_events_can_mutate_them</a> proves the model cannot mutate calls after they&#39;re captured</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L392">test_ask_rejects_snapshot_mutated_during_authorization</a> proves approval binding rejects mutated arguments</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L503">test_policy_deny_is_terminal_and_never_executes</a> proves denied tools never execute</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L560">test_metadata_error_fails_closed_without_leaking_exception</a> proves internal errors don&#39;t leak to the model</li></ul><pre>cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_policy.py -v</pre><p>Total project: 1,205 tests passing, 69% coverage.</p><h3>What frameworks provide</h3><p>LangChain tools are Python functions decorated with @tool. The framework handles serialization and basic argument passing, but policy evaluation and approval are not built in , you add them yourself or use a separate guardrails library.</p><p>LangGraph adds the concept of “human-in-the-loop” nodes in the graph, which is closer to the approval gate. But the tool definition itself has no schema enforcement or risk classification.</p><p>CrewAI tools declare a description and args schema, but execution governance (policy, approval, timeouts, audit) is not part of the tool contract.</p><p>Microsoft Foundry Agent Service provides managed tool execution with built-in safety filters, but the policy logic is opaque , you configure rules in the portal, not in deterministic code you can test.</p><h3>When I would use a framework</h3><p>When the tool set is small (5–10 tools), the risk is low (read-only data queries), and you don’t need per-tool policy rules or human approval gates, a framework’s built-in tool handling is sufficient. Don’t over-engineer read-only search tools.</p><p>When you need:</p><ul><li>Different policy rules per tool based on risk classification</li><li>Human approval for sensitive operations</li><li>Cryptographic binding between approval and execution</li><li>Audit trails that prove which arguments were validated and by whom</li><li>TOCTOU protection for tool arguments</li></ul><p>…you need a contract boundary. Build it or find a framework that implements it.</p><h3>Limitations</h3><ul><li>Schema validation supports a JSON Schema subset, not the full spec. No $ref, allOf, anyOf, oneOf, or patternProperties.</li><li>Policy rules are statically configured. Dynamic rules (e.g., “allow this tool only between 9am-5pm”) require code changes.</li><li>Human approval is synchronous — the runtime blocks until the human responds or the timeout fires. There is no “come back later” workflow.</li><li>Risk classification is per-tool-registration, not per-call. The same tool always has the same risk classes regardless of arguments.</li><li>Error messages are deliberately vague (“Invalid parameter: field”) to avoid leaking schema details to the model. This makes debugging harder.</li></ul><h3>Reader exercise</h3><ol><li>Clone the repository: git clone <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></li><li>Run cd backend &amp;&amp; uv sync --extra dev &amp;&amp; uv run pytest tests/unit/test_runtime_policy.py::test_policy_deny_is_terminal_and_never_executes -v</li><li>Read the test. It creates a tool call, configures a DENY policy, runs the engine, and asserts the tool handler was <strong>never called</strong></li><li>Now run test_ask_rejects_snapshot_mutated_during_authorization — it proves that mutating arguments between policy check and approval causes the binding to fail</li><li>Open <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L332">registry.py:332</a> — the _validate_arguments method. Trace how copy.deepcopy creates the execution copy</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Understand:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/tool-contracts.md">Tool contracts concept</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/json-schema.md">JSON Schema subset</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/policy-engine.md">Policy engine concept</a></li><li><strong>Follow:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/modules/04-tools-and-schemas/README.md">Module 04 — Tools and schemas</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/modules/05-policy-and-approvals/README.md">Module 05 — Policy and approvals</a></li><li><strong>Walk through code:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/code-walkthroughs/tool-registry.md">Tool registry walkthrough</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/code-walkthroughs/policy-and-approval.md">Policy and approval walkthrough</a></li><li><strong>See:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/visual-learning/packs/system-overview.json">System overview Visual Learning pack</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/visual-learning/modules/hooks-and-extensions/">Hooks and extensions pack</a></li><li><strong>Inspect:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L206-L262">registry.py — ToolDefinition</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/tools/registry.py#L332-L369">registry.py — _validate_arguments</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/security/policy.py#L26-L30">policy.py — PolicyAction</a></li><li><strong>Verify:</strong> cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_policy.py tests/security/test_policy_engine.py -v</li><li><strong>Continue:</strong> Article 4 Why Tool Access Is Not Authorization</li></ul><h3>Series navigation</h3><p><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">1. I Built an AI Agent Harness from Scratch</a></p><p><a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">2. Building a Typed ReAct Loop Without a Framework</a></p><p><strong>3. Tool Contracts: The Boundary Between Intent and Execution (this article)</strong></p><p>4. Why Tool Access Is Not Authorization</p><p>5. State Machines, Stop Reasons, and Bounded Execution</p><p>6. Context Is Not Memory</p><p><em>Full series plan: 32 articles across 5 seasons. See the </em><a href="https://github.com/levalencia/cogentrex/blob/dev/docs/articles/SERIES-PLAN.md"><em>series plan.</em></a></p><h3>What comes next</h3><p>The next article examines a subtlety most agent builders miss: <strong>tool access is not the same as authorization.</strong> Having a tool registered doesn’t mean the model can use it freely. Policy, approval, and risk classification create layers of governance that sit between “the tool exists” and “the tool may execute.”</p><p>We will trace how the same tool call gets three different policy outcomes depending on who’s asking, what arguments are passed, and which risk rules apply.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1b0ca30ded87" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87">Tool Contracts: The Boundary Between Intent and Execution</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Typed ReAct Loop Without a Framework]]></title>
            <link>https://python.plainenglish.io/building-a-typed-react-loop-without-a-framework-0c7b88c1c981?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/0c7b88c1c981</guid>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Tue, 29 Sep 2026 07:43:12 GMT</pubDate>
            <atom:updated>2026-09-29T17:50:09.940Z</atom:updated>
            <content:encoded><![CDATA[<h3>Building a Typed ReAct Loop Without a Framework</h3><p><em>This is Article 2 in the </em><a href="https://github.com/levalencia/cogentrex"><em>Cogentrex Agent Harness series</em></a><em>. Article 1: </em><a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em>I Built an AI Agent Harness from Scratch</em></a>,<a href="https://medium.com/python-in-plain-english/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79"><em> Here’s What Frameworks Abstract Away</em></a><em>.</em></p><h3>The problem</h3><p>Most agent tutorials show something like this:</p><pre>while True:<br>    response = llm.chat(messages)<br>    if response.has_tool_calls:<br>        for call in response.tool_calls:<br>            result = execute(call)<br>            messages.append(result)<br>    else:<br>        return response.content</pre><p>It works for demos. It breaks in production. There is no iteration limit, no token budget, no timeout, no way to know <em>why</em> the loop ended. The model decides everything, when to call tools, when to stop, how many times to loop. You have zero control over the boundaries.</p><p>Frameworks solve this by adding their own abstractions: LangGraph gives you a state graph, CrewAI gives you agent roles, AutoGen gives you conversation patterns. But each framework also takes control of the loop itself. When you need to customize how the model’s tool calls are authorized, how budgets are enforced, or why the loop stopped, you’re fighting the framework’s abstractions.</p><p><strong>Thesis:</strong> A reliable agent loop is a typed state machine with explicit budgets and stop reasons, not an unbounded while loop with hope as an exit strategy.</p><h3>Mental model</h3><p>Think of the loop as a bounded conveyor belt in a factory:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/902/1*3sTk4FEtfe7fXAwzAisseA.png" /></figure><p>Every exit produces a named StopReason, never a silent break:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YHMtLnFHXrljkENtPZ4-hg.png" /></figure><p>The model does the thinking. The harness controls the boundaries. The model never decides when to stop, the harness does.</p><h3>The minimal typed loop</h3><p>Before looking at Cogentrex, here is what a typed ReAct loop needs at minimum:</p><pre>@dataclass(frozen=True)<br>class RuntimeBudget:<br>    max_iterations: int = 15<br>    max_tool_calls: int = 20<br>    max_tokens: int = 64_000<br>    max_seconds: float = 300.0<br><br>class StopReason(StrEnum):<br>    COMPLETED = &quot;completed&quot;<br>    ITERATION_BUDGET_EXHAUSTED = &quot;iteration_budget_exhausted&quot;<br>    TOOL_BUDGET_EXHAUSTED = &quot;tool_budget_exhausted&quot;<br>    TIME_BUDGET_EXHAUSTED = &quot;time_budget_exhausted&quot;<br>    # ... every exit path has a name<br><br>@dataclass(frozen=True)<br>class AgentResult:<br>    content: str<br>    stop_reason: StopReason<br>    iterations: int<br>    tool_calls: list[dict]<br>    usage: TokenUsage</pre><p>Three things make this different from a naive loop:</p><ol><li><strong>Typed budget</strong>, a frozen dataclass, not magic numbers scattered in code.</li><li><strong>Named stop reasons</strong>, every exit is an enum value, never a silent break</li><li><strong>Typed result</strong>, callers know <em>what</em> happened and <em>why</em> it stopped.</li></ol><h3>From demo to harness</h3><p>The naive loop asks: “did the model say something?” The harness loop asks six questions on every iteration:</p><ol><li><strong>Budget check,</strong> have I exceeded iterations, tokens, time, or cost?</li><li><strong>Capability check</strong>, does this provider support what I need?</li><li><strong>Context check, </strong>will this request fit in the context window?</li><li><strong>Policy check</strong>, is this tool call allowed by deterministic rules?</li><li><strong>Approval check, </strong>does this tool call need human authorization?</li><li><strong>Result check, </strong>does the output conform to a structured contract?</li></ol><p>If any check fails, the loop stops with the corresponding StopReason. The model never sees the budget logic, it just receives messages and produces responses.</p><h3>Cogentrex implementation</h3><p>The full engine is in <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py">backend/app/runtime/engine.py</a>. Here are the key decisions. Here are the key decisions.</p><h3>Budget as a frozen dataclass</h3><p><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L93-L104">RuntimeBudget</a>:</p><pre>@dataclass(frozen=True, slots=True)<br>class RuntimeBudget:<br>    max_iterations: int = 15<br>    max_tool_calls: int = 20<br>    max_tokens: int = 64_000<br>    max_seconds: float = 300.0<br>    max_tool_result_chars: int = 12_000<br>    final_synthesis_tokens: int = 2_048<br>    max_structured_retries: int = 1<br>    max_context_tokens: int = 200_000<br>    context_output_reserve_tokens: int = 4_096</pre><p>frozen=True means you cannot accidentally modify the budget mid-run. slots=True makes it memory-efficient. The budget is set once at construction and enforced on every iteration.</p><h3>Twenty stop reasons</h3><p><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L70-L90">StopReason</a>twenty named exit paths:</p><pre>class StopReason(StrEnum):<br>    COMPLETED = &quot;completed&quot;<br>    CANCELLED = &quot;cancelled&quot;<br>    ITERATION_BUDGET_EXHAUSTED = &quot;iteration_budget_exhausted&quot;<br>    TOOL_BUDGET_EXHAUSTED = &quot;tool_budget_exhausted&quot;<br>    TOKEN_BUDGET_EXHAUSTED = &quot;token_budget_exhausted&quot;<br>    CONTEXT_BUDGET_EXHAUSTED = &quot;context_budget_exhausted&quot;<br>    TIME_BUDGET_EXHAUSTED = &quot;time_budget_exhausted&quot;<br>    POLICY_DENIED = &quot;policy_denied&quot;<br>    APPROVAL_TIMEOUT = &quot;approval_timeout&quot;<br>    APPROVAL_UNAVAILABLE = &quot;approval_unavailable&quot;<br>    PROVIDER_CAPABILITY_UNSUPPORTED = &quot;provider_capability_unsupported&quot;<br>    STRUCTURED_OUTPUT_INVALID = &quot;structured_output_invalid&quot;<br>    PROVIDER_LENGTH_LIMIT = &quot;provider_length_limit&quot;<br>    PROVIDER_REFUSAL = &quot;provider_refusal&quot;<br>    PROVIDER_CONTENT_FILTER = &quot;provider_content_filter&quot;<br>    MONETARY_BUDGET_EXHAUSTED = &quot;monetary_budget_exhausted&quot;<br>    MODEL_CHARGE_DUPLICATE = &quot;model_charge_duplicate&quot;<br>    MODEL_CHARGE_INDETERMINATE = &quot;model_charge_indeterminate&quot;<br>    PROVIDER_ERROR = &quot;provider_error&quot;<br>    ERROR = &quot;error&quot;</pre><p>Every exit path is a named enum value. When a run finishes, the caller knows exactly why, not “it stopped” but “time budget exhausted at iteration 12 after 298 seconds.”</p><h3>The core loop (simplified)</h3><p>The core loop (<a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L287-L340">_run</a>, simplified):</p><pre>async def _run(self, messages, ...) -&gt; AgentResult:<br>    history = list(messages)<br>    iterations = 0<br>    while iterations &lt; self._budget.max_iterations:<br>        if self._expired(started_at):<br>            return await self._stop(StopReason.TIME_BUDGET_EXHAUSTED, ...)<br>        if estimate_tokens(history) &gt; context_allowance:<br>            return await self._stop(StopReason.CONTEXT_BUDGET_EXHAUSTED, ...)<br>        iterations += 1<br>        response = await self._model.chat(history, tools=tool_definitions)<br>        if response.tool_calls:<br>            for call in response.tool_calls:<br>                # Policy check → Approval check → Execute → Record<br>                result = await self._execute_tool(call, ...)<br>                history.append(tool_result_message)<br>        else:<br>            # Final answer - apply reflection, compliance, structured output<br>            return await self._stop(StopReason.COMPLETED, response.content, ...)<br>    return await self._stop(StopReason.ITERATION_BUDGET_EXHAUSTED, ...)</pre><p>The real engine is ~1,900 lines because it handles every edge case: provider capability mismatches, structured output retries, monetary budget tracking, cancellation cleanup, deadline enforcement with asyncio.wait_for, and 30+ runtime events at every lifecycle point. But the skeleton is this: <strong>check budget → call model → process tool calls or final answer → loop or stop.</strong></p><h3>Native tool calls, not text parsing</h3><p><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/models.py#L42-L53">ToolCall</a>the runtime never parses tool calls from text:</p><pre>@dataclass(frozen=True, slots=True)<br>class ToolCall:<br>    &quot;&quot;&quot;A provider-decoded tool call. Runtime code never parses calls from text.&quot;&quot;&quot;<br>    id: str<br>    name: str<br>    arguments: Mapping[str, Any] = field(default_factory=dict)<br><br>    def __post_init__(self) -&gt; None:<br>        if not self.id or not self.name:<br>            raise ValueError(&quot;tool call id and name must be non-empty&quot;)<br>        object.__setattr__(self, &quot;arguments&quot;, MappingProxyType(dict(self.arguments)))</pre><p>The runtime never does json.loads(response.content) to find tool calls. It uses the provider&#39;s native tool-calling API. The MappingProxyType makes arguments read-only, preventing accidental mutation between policy check and execution (a TOCTOU defense).</p><h3>Evidence and tests</h3><p>The runtime engine has extensive test coverage. Key test contracts:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_v2.py#L145">test_runtime_v2.py::test_explicit_budget_stop_reasons</a> proves iteration, tool, and token budgets trigger the correct StopReason</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_v2.py#L154">test_runtime_v2.py::test_tool_budget_exhaustion_balances_pending_tool_use</a> proves the engine closes pending tool calls before final synthesis</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_v2.py#L221">test_runtime_v2.py::test_timeout_stop_reason</a> proves wall-clock deadline enforcement</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_budget_regressions.py#L68">test_runtime_budget_regressions.py::test_token_budget_exhaustion_forces_tool_free_synthesis</a> proves token budget triggers synthesis without tools</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_runtime_policy.py#L73">test_runtime_policy.py::test_allow_orders_policy_event_before_execution</a> proves policy evaluation happens before tool execution</li><li><a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/tests/unit/test_budgeted_provider.py">test_budgeted_provider.py</a> proves monetary budget enforcement</li></ul><pre># Run the focused tests<br>cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py tests/unit/test_runtime_budget_regressions.py tests/unit/test_runtime_policy.py tests/unit/test_budgeted_provider.py -v</pre><p>Total project: 1,205 tests passing, 69% coverage.</p><h3>What frameworks provide</h3><p>LangGraph adds a state graph where nodes are functions and edges are conditions. This is powerful for complex workflows, fan-out, fan-in, human-in-the-loop at graph nodes, persistent checkpoints. If your task naturally decomposes into a graph of specialized steps, LangGraph earns its complexity.</p><p>CrewAI adds roles and task delegation, a project manager, researcher, writer pattern. If your task needs multiple specialized agents with different system prompts, CrewAI reduces boilerplate.</p><p>AutoGen adds conversation patterns between agents. If your task is best modeled as a multi-party discussion, AutoGen’s abstractions help.</p><h3>When I would use a framework</h3><p>I would reach for LangGraph when the task needs explicit branching, parallelism, or shared state across steps , and I need persistent checkpoints between those steps. The graph abstraction earns its complexity when you have five or more specialized nodes with conditional routing between them.</p><p>I would keep the custom loop when:</p><ul><li>I need to control exactly how tool calls are authorized (policy + approval)</li><li>I need typed stop reasons that my upstream systems can act on</li><li>I need to enforce monetary budgets per-run</li><li>I need the loop to emit structured events at every lifecycle point</li><li>I want to understand every line of the execution path</li></ul><p>The honest answer: most agent tasks don’t need a graph. They need a bounded loop with good error handling. Start with the loop. Add the graph when you outgrow it.</p><h3>Limitations</h3><ul><li>The engine is ~1,900 lines of Python. This is not simple code. Understanding every edge case takes time.</li><li>Token counting uses tiktoken estimates, not exact provider counts. The budget may trigger slightly early or late.</li><li>The loop is single-agent. Multi-agent coordination uses separate delegation patterns (Coordinator, Fork, Swarm) with their own loops.</li><li>Structured output retries add latency. A failing schema contract can consume the entire structured retry budget before returning.</li><li>The engine does not hot-reload tools mid-run. Tool definitions are fixed at run() entry.</li></ul><h3>Reader exercise</h3><ol><li>Clone the repository: git clone <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></li><li>Run cd backend &amp;&amp; uv sync --extra dev &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py::test_explicit_budget_stop_reasons -v</li><li>Read the test. It creates a RuntimeBudget with max_iterations=3, runs the engine, and asserts result.stop_reason == StopReason.ITERATION_BUDGET_EXHAUSTED</li><li>Change max_iterations to 1 and rerun. The test should still pass — the budget is the boundary, not the model</li><li>Now look at the real engine: open <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L312">engine.py:312</a> the while iterations &lt; self._budget.max_iterations line. That is the entire loop boundary</li></ol><h3>Go deeper in Cogentrex</h3><ul><li><strong>Understand:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/react.md">ReAct concept</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/typed-runtime.md">Typed runtime</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/concepts/state-machines.md">State machines and stop reasons</a></li><li><strong>Follow:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/modules/02-typed-runtime/README.md">Module 02 — Typed runtime</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/modules/03-react-loop/README.md">Module 03 — ReAct loop</a></li><li><strong>Walk through code:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/docs/course/code-walkthroughs/runtime.md">Runtime code walkthrough</a></li><li><strong>Inspect:</strong> <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L250-L285">engine.py — AgentRuntime.run</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/engine.py#L93-L104">engine.py — RuntimeBudget</a>, <a href="https://github.com/levalencia/cogentrex/blob/0998e37/backend/app/runtime/models.py#L42-L53">models.py — ToolCall</a></li><li><strong>Verify:</strong> cd backend &amp;&amp; uv run pytest tests/unit/test_runtime_v2.py tests/unit/test_runtime_budget_regressions.py tests/unit/test_runtime_policy.py -v</li><li><strong>Continue:</strong> Article 3 Tool Contracts: The Boundary Between Intent and Execution</li></ul><h3>What comes next</h3><p>The next article examines the boundary between what a model <em>wants</em> to do and what it’s <em>allowed</em> to do:</p><p><a href="https://medium.com/@luisevalencia/tool-contracts-the-boundary-between-intent-and-execution-1b0ca30ded87?postPublishedType=initial"><strong>Tool Contracts: The Boundary Between Intent and Execution</strong></a></p><p>We will trace how a tool call gets its schema validated, its arguments checked against policy, and its execution sandboxed, before any side effect touches the world.</p><p>Cogentrex is open source: <a href="https://github.com/levalencia/cogentrex">https://github.com/levalencia/cogentrex</a></p><p><strong>Build agents you can explain.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0c7b88c1c981" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/building-a-typed-react-loop-without-a-framework-0c7b88c1c981">Building a Typed ReAct Loop Without a Framework</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Built an AI Agent Harness from Scratch, Here’s What Frameworks Abstract Away]]></title>
            <link>https://python.plainenglish.io/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a1174a98f79</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[harness-engineering]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Mon, 21 Sep 2026 10:52:04 GMT</pubDate>
            <atom:updated>2026-09-29T11:11:56.997Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>A practical architecture for model loops, tools, state, budgets, policy, evidence, and observability, before reaching for LangGraph, AutoGen, or CrewAI.</em></blockquote><p>I did not build an AI agent harness from scratch because frameworks are bad.</p><p>I built one because I wanted to understand exactly where the model stops and the software begins.</p><p>That boundary matters. A language model can propose a tool call, but it should not decide whether the call is authorized. It can generate an answer, but it should not certify that its own claims are grounded. It can request another iteration, but it should not own an unlimited budget.</p><p>Those are harness responsibilities.</p><p>Frameworks such as LangChain, LangGraph, AutoGen, and CrewAI can accelerate development. They provide useful abstractions, integrations, orchestration patterns, and ecosystem support. But an abstraction is easiest to evaluate when you understand the machinery beneath it.</p><p>This article explains that machinery through <a href="https://github.com/levalencia/cogentrex"><strong>Cogentrex</strong></a>, an open-source agent harness I built in Python. The goal is not to convince you to avoid frameworks. The goal is to make the build-versus-adopt decision legible.</p><blockquote><strong><em>Series thesis:</em></strong><em> Build agents you can explain. Use frameworks when their abstractions earn their complexity.</em></blockquote><h3>Mental model: a boundary is a controlled handoff</h3><p>In this article, a boundary is not necessarily a physical wall. It is a place where responsibility, data, or authority passes from one component to another, and where explicit rules decide what may cross.</p><p>Think of an airport checkpoint. The line on the floor is not the control. The control comes from the required documents, the decision process, the authority of the person checking them, and the recorded outcome. Software boundaries work the same way.</p><p>A boundary begins as an architectural decision. It becomes concrete through interfaces, schemas, validators, policy checks, process isolation, network controls, and tests. A Python file can implement part of a boundary, but no single file is automatically the whole boundary. If nothing enforces the separation, the boundary still exists only in a diagram or in our heads.</p><p>This article uses two related but distinct ideas:</p><ul><li>A <strong>contract boundary</strong> is a controlled handoff between responsibilities, for example, when model output becomes a proposed tool call that deterministic software must validate.</li><li>An <strong>operational limit</strong> is a numerical cap , for example, the maximum number of iterations, tool calls, tokens, seconds, or dollars available to a run.</li></ul><p>That distinction matters because “the prompt asks the model to behave” is not the same as “the software enforces what may happen.”</p><h3>Try Cogentrex locally</h3><p>The public development environment is available at <a href="https://dev.cogentrex.com/">dev.cogentrex.com</a>. It demonstrates the deployed product and Azure operating path; it is not presented as a production SLA, anonymous sharing service, or substitute for the reproducible local setup below. Everything you see in the local repo after deploying its available through <a href="https://dev.cogentrex.com/">dev.cogentrex.com</a> so the easy path its just click the link above, register, use the app, and click on learn to see all the training content.</p><p>You can follow this article without running the project, but <a href="https://github.com/levalencia/cogentrex">Cogentrex</a> is designed to be inspected. The complete and maintained setup instructions live in the repository <a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/README.md#quick-start">Quick start</a>.</p><p>Prerequisites: Docker Desktop, Python 3.11 or newer, uv, and Node.js 22 or newer.</p><pre>git clone https://github.com/levalencia/cogentrex.git<br>cd cogentrex</pre><p>Choose one learning mode <strong>before the first stack start</strong>.</p><h3>Base mode</h3><p>The base clone includes the application, course Markdown, Roadmap, Stories, Architecture, Evidence, and searchable Glossary.</p><pre>COGENTREX_LOCAL_PORT=80 ./scripts/local-stack.sh start<br>./scripts/local-stack.sh status<br>./scripts/local-stack.sh url</pre><h3>Rich learning mode</h3><p>Videos, audio, transcripts, presentation decks, high-fidelity diagrams, infographics, mind maps, flashcards, quizzes, and study guides are distributed as a separate checksummed GitHub Release asset rather than committed binaries. Install that package before starting the stack:</p><pre>make media-install<br>COGENTREX_LOCAL_PORT=80 ./scripts/local-stack.sh start<br>./scripts/local-stack.sh status<br>./scripts/local-stack.sh url</pre><p>Open the printed loopback URL, create a local account, and visit /learn. The default start uses deterministic mock mode, so the interface can be explored without provider credentials and clearly reports that no live inference occurred.</p><p>The media installer downloads the release declared in <a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/visual-learning/release-manifest.json">docs/visual-learning/release-manifest.json</a>, checks its byte size and SHA-256 digest, and installs it as a sibling cogentrex-learning-media directory. See the <a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/visual-learning/README.md#base-vs-rich-behavior">Visual Learning setup</a> for online and offline installation details.</p><h3>A model is not an agent</h3><p>A model accepts input and produces output.</p><p>An agent can pursue an objective over multiple steps, select capabilities, observe results, update its working state, and eventually stop.</p><p>A harness is the deterministic software around that probabilistic model. It decides what the model is allowed to see and do, how long it may continue, what evidence is recorded, and which outcomes count as valid.</p><p>A useful first approximation is:</p><pre>agent = model + loop + tools + state + policy + budgets + evidence</pre><p>The model supplies judgment. The harness supplies control.</p><p>That distinction is easy to miss in a demo. A minimal tool-calling example can look agentic with fewer than fifty lines of code:</p><pre>while True:<br>    response = await model.complete(messages, tools)<br>    if not response.tool_calls:<br>        return response.text<br><br>    for call in response.tool_calls:<br>        result = await execute(call)<br>        messages.append(result)</pre><p>This is a useful learning loop, but it leaves unanswered questions:</p><ul><li>What stops repeated or cyclic tool calls?</li><li>What happens when the context window is nearly full?</li><li>Can a model invoke a destructive tool without approval?</li><li>Is an unknown argument ignored or rejected?</li><li>What happens after a timeout while an external effect is in flight?</li><li>Can a retry duplicate a payment, message, or deployment?</li><li>Which events are safe to persist?</li><li>How do we distinguish provider failure from budget exhaustion?</li><li>Can we explain why the run ended?</li></ul><p>The difference between a tool-calling demo and an agent harness is the quality of those answers.</p><h3>One request, two possible paths</h3><p>The easiest way to understand the harness is to follow one request. After the model responds, the flow splits: it either has a final answer, or it wants to use a tool.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gfJfTLzejJt0UkJD9Lho1g.png" /></figure><p>Read it as two paths that share the first four steps.</p><h3>Shared steps</h3><ol><li><strong>User sends an objective.</strong> This is the new task or message.</li><li><strong>Runtime checks budgets.</strong> Before spending more time, tokens, or tool calls, deterministic code checks whether the run may continue.</li><li><strong>Runtime asks the model.</strong> The model receives bounded context and the schemas of tools it is allowed to see.</li><li><strong>Runtime interprets the model output.</strong> The output is either a final answer or a request to use a tool.</li></ol><h3>Path A: the model returns a final answer</h3><p>5A. The runtime validates the answer and records the terminal evidence.</p><p>6A. The caller receives a typed result containing the answer, usage, iterations, and explicit stop reason.</p><h3>Path B: the model requests a tool</h3><p>5B. The registry validates the exact tool name and arguments. Unknown tools or unexpected arguments fail before execution.</p><p>6B. Policy decides ALLOW, ASK, or DENY. If the decision is ASK, the exact call must receive approval before it becomes authorized.</p><p>7B. Only an authorized call reaches the tool executor. A denied or rejected call follows path 7C and stops or returns governed feedback instead.</p><p>8B. The tool result is recorded as an observation and added to the next bounded iteration. The flow returns to step 2; it does not restart with unlimited authority.</p><p>py.</p><p>Every solid arrow crosses a contract boundary. The rest of this article examines those boundaries one by one.</p><h3>Boundary 1: provider-neutral interfaces</h3><p><a href="https://github.com/levalencia/cogentrex">Cogentrex</a> begins with Python Protocol interfaces instead of depending on one agent framework or one model provider.</p><p>The core interfaces include an LLM client, memory store, tool executor, audit log, permission checker, and guardrail engine.</p><pre>@runtime_checkable<br>class ToolExecutor(Protocol):<br>    async def execute(self, tool_name: str, parameters: dict) -&gt; dict: ...<br>    def list_tools(self) -&gt; list[dict]: ...</pre><p>The important idea is not the amount of code. It is the direction of dependency: the runtime depends on contracts, while provider and infrastructure adapters implement those contracts.</p><p>That gives me three useful properties:</p><ul><li>I can test the runtime with deterministic collaborators.</li><li>I can replace a model provider without rewriting policy or tools.</li><li>I can identify provider-specific behavior instead of hiding it behind a universal claim.</li></ul><p>See the current protocol definitions:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/agents/protocols.py#L1-L92">backend/app/agents/protocols.py, lines 1–92</a></li></ul><p>Provider neutrality does not mean every provider behaves identically. It means differences become explicit adapter and capability contracts rather than accidental assumptions.</p><h3>Boundary 2: a bounded runtime loop</h3><p>The runtime owns iteration, context, token, tool, time, structured-output, and monetary boundaries.</p><p><a href="https://github.com/levalencia/cogentrex">Cogentrex</a> represents terminal outcomes with explicit stop reasons:</p><pre>class StopReason(StrEnum):<br>    COMPLETED = &quot;completed&quot;<br>    ITERATION_BUDGET_EXHAUSTED = &quot;iteration_budget_exhausted&quot;<br>    TOOL_BUDGET_EXHAUSTED = &quot;tool_budget_exhausted&quot;<br>    CONTEXT_BUDGET_EXHAUSTED = &quot;context_budget_exhausted&quot;<br>    TIME_BUDGET_EXHAUSTED = &quot;time_budget_exhausted&quot;<br>    POLICY_DENIED = &quot;policy_denied&quot;<br>    PROVIDER_ERROR = &quot;provider_error&quot;</pre><p>The actual enum contains more cases, including approval failures, provider refusals, unsupported capabilities, invalid structured output, and indeterminate model charges.</p><p>See:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/runtime/engine.py#L70-L127">StopReason` and `RuntimeBudget</a></li></ul><p>The run entry point creates a deadline before entering the inner loop. The loop checks capabilities and budgets before calling the provider:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/runtime/engine.py#L250-L285">AgentRuntime.run</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/runtime/engine.py#L287-L394">The bounded iteration loop</a></li></ul><p>Why does this matter?</p><p>Because “the agent failed” is not an actionable terminal state. A system operator needs to know whether the run:</p><ul><li>completed normally;</li><li>exhausted a budget;</li><li>was denied by policy;</li><li>timed out waiting for approval;</li><li>encountered unsupported provider capabilities;</li><li>received a provider refusal;</li><li>reached an indeterminate billing state.</li></ul><p>Explicit stop reasons turn an opaque loop into an operable state machine.</p><h3>Boundary 3: tool intent is not tool execution</h3><p>A model does not call a Python function directly. It proposes a structured intent.</p><p>The harness must validate that intent before an effect occurs.</p><p><a href="https://github.com/levalencia/cogentrex">Cogentrex</a> registers tools with metadata such as:</p><ul><li>canonical name;</li><li>input schema;</li><li>required permissions;</li><li>timeout;</li><li>risk classes;</li><li>approval requirement;</li><li>whether the tool is effectful;</li><li>idempotency-key behavior.</li></ul><p>See:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/tools/registry.py#L273-L329">SecureToolRegistry.register</a></li></ul><p>Unknown arguments fail closed unless a schema deliberately opts into additional properties. Validation occurs before permission or execution hooks:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/tools/registry.py#L331-L369">Argument validation</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/tools/registry.py#L477-L505">The final execution boundary</a></li></ul><p>This is one of the most important lessons I learned while building the harness:</p><blockquote><em>Native tool calling solves the serialization problem. It does not solve authorization, idempotency, containment, or auditability.</em></blockquote><p>Those remain application responsibilities.</p><h3>Boundary 4: policy is not prompting</h3><p>A system prompt can say, “Do not perform destructive actions without permission.”</p><p>That is useful guidance, but it is not enforcement.</p><p>A deterministic policy engine should decide whether a proposed action is allowed, denied, or requires approval. <a href="https://github.com/levalencia/cogentrex">Cogentrex</a> represents those outcomes explicitly:</p><pre>class PolicyAction(StrEnum):<br>    ALLOW = &quot;allow&quot;<br>    ASK = &quot;ask&quot;<br>    DENY = &quot;deny&quot;</pre><p>Risk classes distinguish reads, writes, execution, network access, external side effects, and destructive actions.</p><p>See:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/security/policy.py#L26-L48">PolicyAction` and `RiskClass</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/security/policy.py#L297-L360">RulePolicyEngine</a></li></ul><p>The principle is simple:</p><pre>model proposes<br>policy decides<br>executor acts<br>ledger records</pre><p>Combining those responsibilities into one model response makes the system easier to demo and harder to trust.</p><h3>Boundary 5: events are part of the architecture</h3><p>When an agent runs for several iterations, the final answer is not enough to understand the system.</p><p><a href="https://github.com/levalencia/cogentrex">Cogentrex</a> emits typed events for:</p><ul><li>run and iteration start;</li><li>model responses;</li><li>budget blocks;</li><li>capability rejection;</li><li>tool requests and completion;</li><li>policy and approval decisions;</li><li>evidence retrieval and claim verification;</li><li>delegation and reflection;</li><li>terminal stop.</li></ul><p>See:</p><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/backend/app/runtime/events.py#L13-L63">AgentEventKind` and `EventSink</a></li></ul><p>These events can feed logs, traces, metrics, UI progress, or durable run records. But observability has its own boundary: arguments, prompts, retrieved content, and model output can contain sensitive data.</p><p>Telemetry should describe execution without automatically becoming a copy of every private payload.</p><h3>Boundary 6: evidence is not self-certification</h3><p>A model should not be the sole judge of whether its own answer is correct.</p><p><a href="https://github.com/levalencia/cogentrex">Cogentrex</a> separates runtime execution from evaluation and deterministic checks. Different workflows can preserve source IDs, hashes, citations, numerical constraints, or structured-output schemas.</p><p>This does not make the system omniscient. A lexical verifier is not semantic truth. A passing test is not a live deployment. A mock provider is not evidence of provider behavior.</p><p>The harness becomes more reliable when it names those limits instead of hiding them.</p><p>Later articles in this series will cover paired evaluations, grounding, structured outputs, observability, and the negative experiments that did not earn promotion.</p><h3>What frameworks provide</h3><p>Building these boundaries yourself is educational, but it is not free.</p><p>Frameworks can provide substantial value.</p><h3>LangChain</h3><p>LangChain offers model and tool integrations, retrieval components, middleware, and a broad ecosystem. It is useful when integration breadth and development speed matter more than owning every runtime contract.</p><h3>LangGraph</h3><p>LangGraph makes graph-shaped, stateful workflows explicit. It is attractive when the application naturally needs branching, resumability, human-in-the-loop transitions, and graph-level orchestration.</p><h3>AutoGen</h3><p>AutoGen focuses on conversational and multi-agent collaboration patterns. It can accelerate experiments where specialized agents exchange messages or coordinate tasks.</p><h3>CrewAI</h3><p>CrewAI offers role-, task-, and crew-oriented abstractions. It can be productive when that mental model matches the domain and the team wants a higher-level orchestration API.</p><p>The right question is not “Which framework is best?”</p><p>It is:</p><blockquote><em>Which contracts do I want to own, and which contracts am I comfortable delegating to a framework?</em></blockquote><p>A framework earns its place when it removes more accidental complexity than control it obscures.</p><h3>Why I chose to build the core first</h3><p>I wanted to be able to answer questions such as:</p><ul><li>Where is the maximum number of iterations enforced?</li><li>What exact data is bound to an approval decision?</li><li>Can a provider mutate a tool call after policy evaluation?</li><li>How are duplicate effects prevented?</li><li>Which terminal state is persisted after cancellation?</li><li>Can unsupported provider capabilities fail before a billable call?</li><li>Which payloads are safe to export to telemetry?</li></ul><p>If I cannot locate those answers in code and tests, I do not yet understand the runtime well enough to operate it.</p><p>Building <a href="https://github.com/levalencia/cogentrex">Cogentrex</a> from first principles forced those questions into the architecture.</p><p>It also exposed costs. The custom runtime is large. It requires its own compatibility tests, policy model, event schema, persistence contracts, observability, and operational discipline. A framework could remove some of that maintenance burden.</p><p>That is why the conclusion is not “always build.”</p><p>The conclusion is:</p><blockquote><em>Build enough to understand the contracts. Then adopt abstractions deliberately.</em></blockquote><h3>What Cogentrex proves , and what it does not</h3><p>At the frozen source commit used for this article, <a href="https://github.com/levalencia/cogentrex">Cogentrex</a> contains:</p><ul><li>provider-neutral interfaces;</li><li>a bounded typed runtime;</li><li>native tool schemas;</li><li>policy and approval boundaries;</li><li>durable run and effect records;</li><li>encrypted memory;</li><li>MCP integration;</li><li>evaluation infrastructure;</li><li>OpenTelemetry instrumentation;</li><li>a separately contained code sandbox.</li></ul><p>The source and automated tests prove those components exist and exercise their contracts.</p><p>Current limitations matter just as much:</p><ul><li>Local Compose is the reproducible baseline, and an Azure VM + hardened Compose development environment is live at dev.cogentrex.com; neither is a production/SLA claim.</li><li>Retrieval uses SQL-JSON cosine, not pgvector.</li><li>Azure Container Apps and Foundry Hosted Agent deployment are future, evidence-gated tracks in this series.</li><li>Framework comparisons require hands-on equivalent implementations; they should not be reduced to feature tables.</li><li>Passing deterministic tests does not prove every live provider path.</li></ul><p>Those limitations are not footnotes. They define the next experiments.</p><h3>A practical decision framework</h3><p>Before choosing custom code or a framework, ask:</p><h3>Choose a framework when</h3><ul><li>time-to-market dominates;</li><li>its state model matches your workflow;</li><li>the team understands its operational behavior;</li><li>integrations remove real work;</li><li>framework-specific persistence and tracing are acceptable;</li><li>migration cost is understood.</li></ul><h3>Own more of the harness when</h3><ul><li>tool effects require strict policy and approval;</li><li>budgets and stop reasons must be deterministic;</li><li>provider portability is a requirement;</li><li>audit and compliance boundaries are product features;</li><li>framework state is too opaque for your incident model;</li><li>evaluation requires control over the full execution contract.</li></ul><p>Many production systems will be hybrid: custom domain and governance contracts around selected framework or managed-platform components.</p><h3>Reader exercise</h3><p>Take the smallest tool-calling agent you have built and answer these questions:</p><ol><li>What are all of its terminal states?</li><li>Where is the iteration limit enforced?</li><li>Can unknown tool arguments reach a handler?</li><li>Which actions require deterministic approval?</li><li>What happens if execution times out after an external effect begins?</li><li>Can a retry duplicate the effect?</li><li>Which event explains why the run stopped?</li><li>Which claim in the final answer can be independently verified?</li></ol><p>If any answer is “the prompt tells the model not to do that,” you have found a harness boundary that still lives in probabilistic behavior.</p><h3>Go deeper in Cogentrex</h3><p>This article is the entry point. Use the following paths when you want a slower explanation, a visual walkthrough, or implementation evidence.</p><h3>Understand the concepts</h3><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/concepts/agent-anatomy.md">Agent anatomy</a> — model, runtime, tools, policy, memory, evidence and terminal state.</li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/concepts/typed-runtime.md">Typed runtime</a> — immutable messages, tool calls and results.</li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/concepts/react.md">ReAct</a> — reasoning/action/observation iteration without exposing hidden chain-of-thought.</li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/concepts/tool-contracts.md">Tool contracts</a> — where model intent crosses into deterministic execution.</li></ul><h3>Follow the ordered course in Github</h3><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/modules/00-agent-anatomy/README.md">Module 00 — Agent anatomy</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/modules/01-python-architecture/README.md">Module 01 — Python architecture</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/modules/02-typed-runtime/README.md">Module 02 — Typed runtime</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/modules/03-react-loop/README.md">Module 03 — ReAct loop</a></li></ul><h3>See, watch and listen</h3><p>Run <a href="https://github.com/levalencia/cogentrex">Cogentrex</a> locally and open /learn:</p><ul><li><strong>Roadmap</strong> sequences the modules.</li><li><strong>Stories</strong> shows the request lifecycle as labeled transitions.</li><li><strong>Architecture</strong> explains stable system layers and trust boundaries.</li><li><strong>Present</strong> contains decks, diagrams, infographics and videos.</li><li><strong>Listen</strong> contains audio lessons and transcripts.</li><li><strong>Study</strong> contains mind maps, flashcards, quizzes and study guides.</li></ul><p>For this article, start with the <strong>System Overview</strong> and <strong>Request Lifecycle</strong> packs. The most relevant code-first lessons are:</p><ul><li>Video 4 — Protocols, Adapters, and Dependency Injection.</li><li>Video 5 — Typed Runtime State, Events, and Stop Reasons.</li><li>Video 6 — The Bounded ReAct Loop and Runtime Budgets.</li><li>Video 7 — Tool Contracts, Schemas, and the Secure Registry.</li><li>Video 8 — Policy, Durable Approval, and Safe Execution.</li></ul><p>The rich-media package and installation instructions are documented in <a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/visual-learning/README.md">Visual Learning Studio</a>. Existing media is derivative, review-ready learning material generated from an earlier source revision; canonical Markdown and current source remain authoritative.</p><h3>Inspect and verify</h3><ul><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/code-walkthroughs/runtime.md">Runtime walkthrough</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/code-walkthroughs/tool-registry.md">Tool registry walkthrough</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/code-walkthroughs/policy-and-approval.md">Policy and approval walkthrough</a></li><li><a href="https://github.com/levalencia/cogentrex/blob/048706a7a9b4f29eecb631659b236ad5a672f2c6/docs/course/reference/test-map.md">Test map</a></li></ul><h3>Continue the series</h3><p>Next: <a href="https://medium.com/@luisevalencia/building-a-typed-react-loop-without-a-framework-0c7b88c1c981?postPublishedType=initial"><strong>Building a Typed ReAct Loop Without a Framework</strong>.</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a1174a98f79" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/built-an-ai-agent-harness-from-scratch-heres-what-frameworks-abstract-away-6a1174a98f79">Built an AI Agent Harness from Scratch, Here’s What Frameworks Abstract Away</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Prompt to Production 01/41-Anatomy of a Production AI Agent: What You Actually Need to Ship]]></title>
            <link>https://python.plainenglish.io/from-prompt-to-production-01-41-anatomy-of-a-production-ai-agent-what-you-actually-need-to-ship-57eff992752b?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/57eff992752b</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[foundry]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Wed, 29 Jul 2026 11:43:32 GMT</pubDate>
            <atom:updated>2026-07-30T10:51:20.473Z</atom:updated>
            <content:encoded><![CDATA[<h3>The 3 AM Refund That Changed Everything</h3><p>Here’s a number that should keep every engineering leader up at night: 88% of AI agent initiatives never reach production. Not because the models aren’t smart enough, they are. Not because the frameworks aren’t powerful , they’re incredible. But because the gap between a working demo and a production system is roughly the same as the gap between a paper airplane and a 747.</p><p>I learned this the hard way.</p><p>A financial services team I worked with deployed their customer-service agent on a Friday afternoon. By 3 AM Saturday, it had processed a “refund” request from a user who’d figured out the magic words. The agent , with no permission boundaries, no audit trail, and no guardrails, issued $47,000 in unauthorized refunds before anyone noticed. The model worked perfectly. It did exactly what it was asked. The problem was everything <em>around</em> the model.</p><p>No circuit breaker to stop after suspicious patterns. No rate limiter to throttle excessive requests. No structured audit logging with correlation IDs to trace the chain of events. No encrypted memory to protect conversation history. No tool sandboxing to prevent unauthorized actions. No ReAct loop with iteration caps to prevent runaway execution.</p><p>This is the story that opens this series. Because the agent wasn’t stupid, it was unprotected.</p><p>And this team wasn’t incompetent. They’d followed the tutorials. They’d taken the courses. The courses just didn’t teach this part.</p><p>That’s why I’m writing this series. Over the next 41 articles, I’m building a complete production AI agent from scratch, not a demo, not a tutorial, not a “hello world” chatbot. A real system with circuit breakers, distributed rate limiters, encrypted memory, PII detection, tool sandboxing, ReAct reasoning loops, multi-agent orchestration, and production deployment on Azure Foundry. Every article comes with working code you can run locally (no API keys needed), passing tests, and the engineering decisions that separate agents that demo well from agents that ship.</p><p>This is Article 1 of 41. By the end of this series, you’ll have built , and understood , every layer a production agent needs. Let’s start with the architecture.</p><h3>Why Demos Fool Us</h3><p>Every AI agent tutorial follows the same script:</p><ol><li>Import your framework of choice</li><li>Connect to an LLM API</li><li>Define some tools</li><li>Call agent.run(&quot;Do the thing&quot;)</li><li>Marvel at the output</li></ol><p>It works. It’s impressive. It’s also about 20% of what you need to ship.</p><p>The other 80%, the part that determines whether your agent survives its first week in production, is invisible in demos. Permission boundaries. Audit trails. Circuit breakers. Rate limiters. Memory management with per-conversation key derivation. Cost caps. Graceful degradation. Sliding window throttling. Path traversal prevention. Fail-open vs fail-closed decisions. The boring stuff that keeps systems alive at 3 AM when nobody’s watching.</p><p>Klarna learned this publicly when they deployed AI agents to replace 700 customer service workers, only to start quietly rehiring humans months later when edge cases overwhelmed the system. The model was fine. The production infrastructure wasn’t there.</p><h3>The 6-Layer Architecture: A Consensus View</h3><p>After studying production deployments from Anthropic, Microsoft, OpenAI, and dozens of startups that actually shipped, a clear architecture emerges. Every successful production agent has six layers, and most failures can be traced to skipping one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/438/1*HuVpxkFGykbOdR7-b3gOaw.png" /></figure><p>Let’s walk through each layer — what it does, why it matters, and what goes wrong without it.</p><h3>Layer 1: Model / Reasoning</h3><p>What it does: Connects to an LLM and gets structured responses back. Handles API communication, retry logic, and response parsing.</p><p>Why it matters: This is the brain of your agent. But critically, it should be <em>swappable</em>. Today it’s GPT-5.6sol, tomorrow it’s Claude, next month it’s a fine-tuned open-source model. If your business logic is entangled with your model client, you’re locked in.</p><p>What goes wrong without it (done right): Vendor lock-in. Silent failures when APIs change. No ability to test without burning API credits.</p><p>In our code: We use a LLMClient Protocol — a contract that any model adapter must satisfy. Our MockLLM implements it for testing. Real adapters wrap OpenAI, Azure, or any API. Swap one line, the whole system works.</p><pre>class LLMClient(Protocol):<br>    async def chat(self, messages: list[dict], max_tokens: int = 1024) -&gt; str: ...</pre><h3>Layer 2: Orchestration</h3><p>What it does: The decision loop that determines <em>how</em> the agent acts. Does it call a tool? Does it respond directly? Should it keep iterating?</p><p>Why it matters: Without orchestration, you have a chatbot , one-shot in, one-shot out. Orchestration is what makes an agent <em>agentic</em>: it can reason, use tools, observe results, and iterate. The ReAct pattern (Reason + Act) is the production standard, think, call a tool, observe the result, decide if you’re done or need another step.</p><p>What goes wrong without it: Infinite loops that burn your API budget. Agents that call the same tool 50 times. No exit conditions, no iteration caps, no circuit breakers for runaway execution.</p><p>In our code: The ProductionAgent implements a ReAct loop with a hard iteration cap (MAX_ITERATIONS = 5). It thinks, optionally calls a tool, observes the result, and decides whether to continue or respond.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kv6rzm_Qmu_7WKt-ZUKhsQ.png" /></figure><h3>Layer 3: Tools</h3><p>What it does: Gives the agent hands. Tools are registered functions with schemas, validation, and execution policies.</p><p>Why it matters: An agent without tools is just an expensive text generator. But tools without guardrails are a security nightmare , the 3 AM refund incident was a tool execution problem. Every tool call needs permission gating, input validation, timeout enforcement via asyncio.wait_for(), and structured audit logging.</p><p>What goes wrong without it: Agents can’t interact with the real world. Or worse , they interact with no boundaries. Timeout-less tools can hang forever. Unvalidated inputs lead to injection attacks. No tool sandboxing means a rogue function can access anything on the host.</p><p>In our code: The SecureToolRegistry wraps every tool call with permission checks, input validation, real timeout enforcement (asyncio.wait_for), and audit logging. No tool runs without going through the security pipeline.</p><h3>Layer 4: Memory</h3><p>What it does: Manages what the agent knows about the current conversation and past interactions. Handles context window limits, message storage, and retrieval.</p><p>Why it matters: Without memory, every interaction starts from zero. The agent can’t reference what the user said two messages ago, can’t learn from previous tool results, and can’t maintain coherent multi-turn conversations. Production memory needs per-conversation encryption with PBKDF2 key derivation, PII detection, and context compression to manage token budgets.</p><p>What goes wrong without it: Repetitive questions. Lost context. Token limit explosions when you naively stuff everything into the prompt. Inconsistent behavior across turns.</p><p>In our code: The MemoryStore Protocol with InMemoryStore for testing and EncryptedMemoryStore for production. Conversation-scoped storage with configurable retrieval limits.</p><h3>Layer 5: Guardrails</h3><p>What it does: The security layer. Permission checks, rate limiters (sliding window with Redis sorted sets), path validation, content filtering, and cost caps. Everything that constrains what the agent is <em>allowed</em> to do.</p><p>Why it matters: This is the layer that would have prevented the 3 AM refund incident. Guardrails are the difference between a helpful assistant and a liability. A circuit breaker stops your agent from hammering a dead service. A distributed rate limiter prevents thundering herd problems. Path traversal prevention with Path.resolve() stops directory escape attacks.</p><p>What goes wrong without it: Unauthorized actions. Path traversal attacks (an agent accessing /etc/passwd instead of /tmp/workspace/report.txt). Runaway costs. Data exfiltration. Cascading failures when external services go down.</p><p>In our code: SecurePermissionManager resolves paths and prevents sibling-prefix bypass attacks with the critical / suffix fix. PermissionChecker Protocol allows plugging in any authorization strategy — RBAC, ABAC, or simple allowlists.</p><pre># The critical fix: &quot;/tmp/user/documents-evil&quot; won&#39;t match &quot;/tmp/user/documents&quot;<br>base_prefix = str(self.base_dir) + &quot;/&quot;</pre><h3>Layer 6: Observability</h3><p>What it does: Structured logging, audit trails, metrics, and tracing. Everything you need to understand what happened after the fact.</p><p>Why it matters: When something goes wrong at 3 AM , and it will, observability is how you figure out what happened, why, and how to prevent it. It’s also how you prove compliance to auditors. Correlation IDs link every log entry for a single request together, so you can trace the entire chain from user input → LLM call → tool execution → permission check → response.</p><p>What goes wrong without it: Impossible debugging. No audit trail for compliance. No way to detect drift, abuse, or errors until users complain. Blind spots in cost tracking.</p><p>In our code: StructuredAuditLogger writes every action to SQLite with timestamps, correlation IDs, and security levels. Every tool call, every permission check, every error , all traceable.</p><h3>Vendor-Neutral Code, Concrete Deployment</h3><p>The code in this series is vendor-neutral. Every component depends on Python Protocols, not SDK imports. You can deploy this agent on Azure AI Foundry, AWS Bedrock, GCP Vertex, or a $5/month VPS , the agent code doesn’t change. What changes is the adapter and the deployment manifest.</p><p>Throughout this series, we use Azure AI Foundry as our concrete deployment example because it offers hosted agents with auto-scaling, managed identity, and built-in observability. But the architecture works anywhere.</p><h3>How These 6 Layers Map to Azure AI Foundry</h3><p>If you’re deploying to Azure AI Foundry (as we do in Articles 31+), here’s how each layer maps to Foundry’s architecture:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CwPstZEWMnrVKQYkr-UjOQ.png" /></figure><p>Deployment: Our agent ships as a Docker container that speaks the Responses Protocol on port 8088. Foundry handles scaling, identity, and lifecycle , you write azd deploy and it runs.</p><p>The key insight: Foundry provides hosting and infrastructure. Your agent code, the Protocols, the ReAct loop, the security checks , is yours. If tomorrow you move to AWS, you swap the deployment manifest and the LLM adapter. The agent doesn’t change.</p><h3>What Courses Miss (and Why It Matters)</h3><p>I’ve completed two comprehensive AI agent courses , covering everything from encrypted memory to Kubernetes deployment. The technical depth is impressive. But here’s what they consistently underemphasize:</p><p>Testing without API keys. Production teams need to run hundreds of tests per CI build. You can’t call GPT-4 for each one. Mock-based testing with Protocol contracts is essential , and rarely taught.</p><p>Security as architecture, not afterthought. Most courses add “security” as a late chapter. In production, permission checking and audit logging are wired into every layer from day one.</p><p>The boring infrastructure. Circuit breakers, rate limiters, graceful degradation, cost caps , the patterns that keep systems alive when things go wrong. These get a lecture at best, not hands-on implementation.</p><p>This series fills those gaps. Not by teaching theory, but by building each component, testing it, and wiring it together.</p><h3>What We’re Building: The 41-Article Roadmap</h3><p>This series takes you from a single-file agent to a production multi-agent system deployed on Azure. Here’s the arc:</p><p>Foundation (Articles 1–5): Architecture, reasoning loops (ReAct, Plan-Execute, Reflexion), prompt engineering, memory systems, tool integration — the building blocks.</p><p>Production Hardening (Articles 6–12): Security (PII detection, path traversal prevention), testing harness (MockLLM, security probes), guardrails (rate limiters, circuit breakers), observability (OpenTelemetry, correlation IDs), error handling — what separates demos from production.</p><p>Advanced Patterns (Articles 13–20): RAG pipelines (naive → agentic), multi-agent orchestration, state management, context engineering — scaling beyond single agents.</p><p>Frameworks &amp; Platforms (Articles 21–30): AutoGen, Semantic Kernel, LangGraph, CrewAI, Microsoft Agent Framework — when to use what, honest comparison.</p><p>Deployment &amp; Operations (Articles 31–41): Docker, Kubernetes, CI/CD with evaluation gates, cost optimization, monitoring, A/B testing, canary deployments — shipping and running in production on Azure AI Foundry.</p><p>Every article has:</p><ul><li>Working code (main.py) you can run locally with no API keys</li><li>Passing tests (test_*.py) using MockLLM</li><li>A run script (run.sh) for one-command execution</li></ul><p>The code compounds. Article 1’s Protocol definitions are used in Article 41’s deployment pipeline. We’re building a real framework, not isolated examples.</p><h3>Try It Now</h3><pre>git clone https://github.com/levalencia/production-ai-agents.git<br>cd production-ai-agents<br>bash scripts/setup.sh<br>cd articles/day-01-anatomy-of-production-agent<br>./run.sh</pre><p>The demo wires all six layers together and shows a MockLLM-powered agent handling a file operation request — with permission checks, audit logging, memory, and iteration tracking all visible.</p><h3>What’s Next — And a Holiday Promise</h3><p>I’m writing this from the last day before my summer holiday. ☀️</p><p>When I come back in mid-August, Article 2 drops immediately: we’ll build a ReAct agent from scratch — no frameworks, just Python — and explore how Plan-and-Execute and Reflexion patterns extend the basic loop. You’ll see exactly how an LLM decides when to think, when to act, and when to stop. Then we get into the <em>really</em> fun stuff: encrypted memory with per-conversation PBKDF2 key derivation, PII detection pipelines, circuit breakers that protect your agent from dead services, distributed rate limiters with Redis sliding windows, tool sandboxing with real timeout enforcement, and eventually multi-agent systems with A2A protocol, agentic RAG, and production deployment on Azure AI Foundry.</p><p>41 articles. 41 runnable code examples. Zero hand-waving.</p><p>Follow me here and on <a href="https://www.linkedin.com/in/levalencia/">LinkedIn</a> so you don’t miss Article 2. This is going to be a wild ride. 🚀</p><p><em>This is Article 1 of the “From Prompt to Production” series. Code, tests, and article drafts live at </em><a href="https://github.com/levalencia/production-ai-agents"><em>github.com/levalencia/production-ai-agents</em></a><em>.</em></p><h3>Before you go</h3><ul><li>Please take a moment to like the post and follow the writer!</li><li>Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute <a href="https://plainenglish.io/write-for-us">here</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57eff992752b" width="1" height="1" alt=""><hr><p><a href="https://python.plainenglish.io/from-prompt-to-production-01-41-anatomy-of-a-production-ai-agent-what-you-actually-need-to-ship-57eff992752b">From Prompt to Production 01/41-Anatomy of a Production AI Agent: What You Actually Need to Ship</a> was originally published in <a href="https://python.plainenglish.io/">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Model Availability Is Now an AI Governance Risk]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-snippet">The Anthropic Fable 5 and Mythos 5 export-control episode is a reminder that enterprise AI teams need to monitor more than latency, price&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@luisevalencia/model-availability-is-now-an-ai-governance-risk-761e8a234a92?source=rss-c3bed98a7757------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@luisevalencia/model-availability-is-now-an-ai-governance-risk-761e8a234a92?source=rss-c3bed98a7757------2</link>
            <guid isPermaLink="false">https://medium.com/p/761e8a234a92</guid>
            <dc:creator><![CDATA[Luis Valencia]]></dc:creator>
            <pubDate>Thu, 02 Jul 2026 11:29:03 GMT</pubDate>
            <atom:updated>2026-07-02T11:29:03.132Z</atom:updated>
        </item>
    </channel>
</rss>