<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Abhishek Arora on Medium]]></title>
        <description><![CDATA[Stories by Abhishek Arora on Medium]]></description>
        <link>https://medium.com/@abhishek-arora?source=rss-2ffeb9b00952------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*HxQiV3_Y46AFwhTP.jpg</url>
            <title>Stories by Abhishek Arora on Medium</title>
            <link>https://medium.com/@abhishek-arora?source=rss-2ffeb9b00952------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 00:55:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@abhishek-arora/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Taming the Orchestrator: Adding Determinism to an AI Agent That Controls Sub-Agents]]></title>
            <link>https://abhishek-arora.medium.com/taming-the-orchestrator-adding-determinism-to-an-ai-agent-that-controls-sub-agents-768fd1f5431a?source=rss-2ffeb9b00952------2</link>
            <guid isPermaLink="false">https://medium.com/p/768fd1f5431a</guid>
            <category><![CDATA[open-code]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Abhishek Arora]]></dc:creator>
            <pubDate>Wed, 25 Mar 2026 11:57:08 GMT</pubDate>
            <atom:updated>2026-03-26T09:47:18.388Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5HNrohsN8sugisuS2UCpAw.png" /></figure><p>About a couple of weeks into building our <strong>multi-agent system</strong> for coding using <strong>opencode</strong>, a routine run silently corrupted an output. The tester sub-agent had executed against half-integrated code. The planner had glossed over a required detail. Neither complained, they just kept going. By the time we caught it, the damage had cascaded across four downstream steps.</p><p>The bug wasn’t in a sub-agent. It was in the orchestrator - the AI agent responsible for deciding what to do next, in what order, and when to move on.</p><h3>The Old Flow: The Illusion of Control</h3><p>To understand the fix, you have to understand how we started. In the “Old Flow”, the system was essentially a polite conversation. The orchestrator was given a list of sub-agents and a clear <strong>Intended Workflow</strong> in its system prompt:</p><ol><li>Architecture Design</li><li>User Approval On The Design</li><li>Code Implementation</li><li>Unit &amp; Integration Testing</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WRHQyluq7d8Ue1jQnO1Jog.png" /><figcaption>Full Workflow</figcaption></figure><p><strong>The Prompt:</strong> “You are the leader. Follow the intended sequence above, talk to the sub-agents, and tell us when the project is done.”</p><p><strong>The Execution:</strong> The orchestrator would start correctly, messaging the Design Agent. The Design Agent would reply. The orchestrator would process that reply and then decide to move to the next step.</p><p><strong>The Failure:</strong> There was zero structural certainty. Even though the intended flow was present in the orchestrator’s context, there was no guarantee it would follow it. The model treated the sequence as a soft suggestion rather than a hard constraint. Whether by mistake or by choice, the system can overlook an entire stage. If the Design Agent provides a vague or incomplete specification, the orchestrator - driven by a desire to keep the project moving- often fills in the gaps with its own assumptions and prematurely instructs the Coding Agent to proceed.</p><h3>This Is Not About a Script That Calls Sub-Agents</h3><p>In framework-heavy environments like LangGraph, you often write the workflow in Python code. That is <strong>Code-First Orchestration</strong>, and it is deterministic because you are the architect.</p><p>But we are moving toward <strong>Agent-First Orchestration</strong>, using autonomous tools like <strong>Claude Code</strong> or <strong>OpenCode</strong>. In this setup, the orchestrator is an LLM given a goal and a toolbox. It reasons about the state, decides which tool to invoke, and determines when a phase is complete. Without guardrails, the whole system inherits the non-determinism of its least constrained component - the one in charge.</p><h3>The Shift: From Prompting to Enforcement</h3><p>I stopped trying to convince the orchestrator to behave and started building a system that made incorrect behaviour structurally impossible. The orchestrator retains the intelligence to reason, but it loses the authority to unilaterally change the workflow state. It can no longer just “decide” a phase is done; it must prove it through a tool call validated against hard rules.</p><h3>Phase One: Classification as the Contract</h3><p>In the Old Flow, the workflow was a suggestion. In the New Flow, we introduced a mandatory Classification Phase. Before a single sub-agent is triggered, the orchestrator must look at the task and commit to a Workflow Contract — a structured JSON artifact locked into a database.</p><pre>{<br>  &quot;phase&quot;: &quot;CLASSIFICATION&quot;,<br>  &quot;workflowType&quot;: &quot;full_workflow&quot;,<br>  &quot;requiredPhases&quot;: [&quot;CLASSIFICATION&quot;, &quot;DESIGN&quot;, &quot;DESIGN_APPROVAL&quot;, &quot;CODING&quot;, &quot;TESTING&quot;]<br>}</pre><p>Once closed, the Intended Workflow is no longer just context — it is the immutable track the system is sitting on. The system simply won’t let the orchestrator skip Architecture Design.</p><h3>A Single Source of Truth: The Workflow State</h3><p>We moved the plan out of the LLM’s shifting context window and into a persisted Workflow State Manager. In the Old Flow, if the context got too long, the orchestrator might forget it hadn’t actually finished testing. Now, if the state doesn’t say DESIGN_COMPLETE, the implementation tools remain physically locked.</p><p>Beyond correctness, the Workflow State also becomes your performance ledger. Every phase tracks when it started, when it last had activity, and when it closed. Token usage is recorded per phase and rolled up to a session total. This means you can answer questions like “how long does our design phase typically take?” or “which phase is burning the most tokens?” without scraping logs.</p><pre>{<br>  &quot;sessionId&quot;: &quot;dev_9982&quot;,<br>  &quot;startedAt&quot;: &quot;2025-03-01T09:00:00Z&quot;,<br>  &quot;updatedAt&quot;: &quot;2025-03-01T11:42:00Z&quot;,<br>  &quot;completedAt&quot;: null,<br>  &quot;totalTokensUsed&quot;: 18420,<br>  &quot;currentPhase&quot;: &quot;CODING&quot;,<br>  &quot;completedPhases&quot;: [&quot;CLASSIFICATION&quot;, &quot;DESIGN&quot;, &quot;DESIGN_APPROVAL&quot;],<br>  &quot;phaseArtifacts&quot;: {<br>    &quot;DESIGN&quot;: { &quot;doc_id&quot;: &quot;req_01&quot;, &quot;verified&quot;: true },<br>    &quot;DESIGN_APPROVAL&quot;: { &quot;approved_by&quot;: &quot;human&quot;, &quot;verified&quot;: true },<br>    &quot;CODING&quot;: null<br>  },<br>  &quot;phaseMetrics&quot;: {<br>    &quot;CLASSIFICATION&quot;: {<br>      &quot;startedAt&quot;: &quot;2025-03-01T09:00:00Z&quot;,<br>      &quot;updatedAt&quot;: &quot;2025-03-01T09:04:00Z&quot;,<br>      &quot;completedAt&quot;: &quot;2025-03-01T09:04:00Z&quot;,<br>      &quot;tokensUsed&quot;: 1020<br>    },<br>    &quot;DESIGN&quot;: {<br>      &quot;startedAt&quot;: &quot;2025-03-01T09:04:00Z&quot;,<br>      &quot;updatedAt&quot;: &quot;2025-03-01T10:15:00Z&quot;,<br>      &quot;completedAt&quot;: &quot;2025-03-01T10:15:00Z&quot;,<br>      &quot;tokensUsed&quot;: 7340<br>    },<br>    &quot;DESIGN_APPROVAL&quot;: {<br>      &quot;startedAt&quot;: &quot;2025-03-01T10:15:00Z&quot;,<br>      &quot;updatedAt&quot;: &quot;2025-03-01T11:10:00Z&quot;,<br>      &quot;completedAt&quot;: &quot;2025-03-01T11:10:00Z&quot;,<br>      &quot;tokensUsed&quot;: 210<br>    },<br>    &quot;CODING&quot;: {<br>      &quot;startedAt&quot;: &quot;2025-03-01T11:10:00Z&quot;,<br>      &quot;updatedAt&quot;: &quot;2025-03-01T11:42:00Z&quot;,<br>      &quot;completedAt&quot;: null,<br>      &quot;tokensUsed&quot;: 9850<br>    }<br>  }<br>}</pre><h3>Tool-Gated Transitions: The “No” Loop</h3><p>Every state change goes through an explicit tool call. If the orchestrator tries to complete_phase(&quot;DESIGN&quot;) without a submitted artifact, the tool returns a hard error directly into the LLM&#39;s context:</p><blockquote>“Error: Required artifact ‘Architecture_Specs’ is missing. Phase DESIGN cannot be closed.”</blockquote><p>This creates a Negative Constraint Feedback Loop. The LLM hits a wall, realises its mistake, and self-corrects. It has to prove it finished the work — through the tools.</p><h3>Phase Artifacts: Forcing Confidence, Not Just Completion</h3><p>We require every phase to produce a structured artifact. The confidence score forces self-assessment, and the assumptions field externalises implicit decisions. The system doesn’t care how “intelligent” the orchestrator sounds in the chat logs; if the JSON artifact isn’t submitted and validated, the gate stays closed.</p><pre>{<br>  &quot;phase&quot;: &quot;DESIGN&quot;,<br>  &quot;summary&quot;: &quot;Designed a three-service architecture...&quot;,<br>  &quot;confidence&quot;: 0.87,<br>  &quot;assumptions&quot;: [<br>    &quot;Downstream services support async callbacks&quot;,<br>    &quot;Database schema is stable&quot;<br>  ],<br>  &quot;risks&quot;: [<br>    &quot;Latency may spike under high fan-out&quot;<br>  ],<br>  &quot;guarantees&quot;: [<br>    &quot;Each service boundary is independently deployable&quot;<br>  ]<br>}<br>```</pre><h3>The Human Approval Gate</h3><p>Between design and coding sits a mandatory <strong>DESIGN_APPROVAL</strong> phase but only in workflow types that include a <strong>DESIGN</strong> phase. It is a first-class phase in the workflow contract, not just a tool call tucked between two other phases.</p><pre>complete_phase(&quot;DESIGN&quot;)          ← valid after artifact is submitted<br>start_phase(&quot;DESIGN_APPROVAL&quot;)<br>request_approval(&quot;DESIGN&quot;)        ← system pauses here<br>[Human reviews design artifact]<br>approve_phase(&quot;DESIGN&quot;)<br>complete_phase(&quot;DESIGN_APPROVAL&quot;)<br>start_phase(&quot;CODING&quot;)</pre><h3>Test Artifact Verification</h3><p>After coding, testing follows. The TESTING phase cannot complete without two specific artifacts: a unit test report and an end-to-end test report. The orchestrator cannot summarise its way past a failing test suite. Both artifacts must follow the phase artifact schema with the underlying test data embedded.</p><h3>The Audit Trail: Proving Progress</h3><p>The state machine constrains tools, but it cannot force the orchestrator to invoke them. This is where the Audit Trail becomes essential — a ledger of every valid transition that actually happened. Each event is timestamped, so the audit trail doubles as a precise execution timeline you can replay or diff across runs.</p><pre>{<br>  &quot;sessionId&quot;: &quot;abc123&quot;,<br>  &quot;auditTrail&quot;: [<br>    { &quot;event&quot;: &quot;phase_started&quot;,     &quot;phase&quot;: &quot;CLASSIFICATION&quot;,  &quot;timestamp&quot;: &quot;2025-03-01T09:00:00Z&quot;, &quot;tokensUsed&quot;: 0    },<br>    { &quot;event&quot;: &quot;phase_completed&quot;,   &quot;phase&quot;: &quot;CLASSIFICATION&quot;,  &quot;timestamp&quot;: &quot;2025-03-01T09:04:00Z&quot;, &quot;tokensUsed&quot;: 1020 },<br>    { &quot;event&quot;: &quot;phase_started&quot;,     &quot;phase&quot;: &quot;DESIGN&quot;,          &quot;timestamp&quot;: &quot;2025-03-01T09:04:00Z&quot;, &quot;tokensUsed&quot;: 0    },<br>    { &quot;event&quot;: &quot;artifact_submitted&quot;,&quot;phase&quot;: &quot;DESIGN&quot;,          &quot;timestamp&quot;: &quot;2025-03-01T10:10:00Z&quot;, &quot;tokensUsed&quot;: 6800 },<br>    { &quot;event&quot;: &quot;phase_completed&quot;,   &quot;phase&quot;: &quot;DESIGN&quot;,          &quot;timestamp&quot;: &quot;2025-03-01T10:15:00Z&quot;, &quot;tokensUsed&quot;: 7340 },<br>    { &quot;event&quot;: &quot;phase_started&quot;,     &quot;phase&quot;: &quot;DESIGN_APPROVAL&quot;, &quot;timestamp&quot;: &quot;2025-03-01T10:15:00Z&quot;, &quot;tokensUsed&quot;: 0    },<br>    { &quot;event&quot;: &quot;approval_granted&quot;,  &quot;phase&quot;: &quot;DESIGN_APPROVAL&quot;, &quot;timestamp&quot;: &quot;2025-03-01T11:08:00Z&quot;, &quot;tokensUsed&quot;: 190  },<br>    { &quot;event&quot;: &quot;phase_completed&quot;,   &quot;phase&quot;: &quot;DESIGN_APPROVAL&quot;, &quot;timestamp&quot;: &quot;2025-03-01T11:10:00Z&quot;, &quot;tokensUsed&quot;: 210  }<br>  ]<br>}</pre><h3>The Full Flow</h3><p>For a <strong>full_workflow</strong> , the deterministic sequence looks like this:</p><pre>start_phase(&quot;CLASSIFICATION&quot;)<br>complete_phase(&quot;CLASSIFICATION&quot;)        → [CLASSIFICATION, DESIGN, DESIGN_APPROVAL, CODING, TESTING]<br>start_phase(&quot;DESIGN&quot;)<br>submit_phase_artifact(&quot;DESIGN&quot;, { … })<br>complete_phase(&quot;DESIGN&quot;)<br>start_phase(&quot;DESIGN_APPROVAL&quot;)<br>request_approval(&quot;DESIGN&quot;)             → waits for human<br>approve_phase(&quot;DESIGN&quot;)<br>complete_phase(&quot;DESIGN_APPROVAL&quot;)<br>start_phase(&quot;CODING&quot;)<br>submit_phase_artifact(&quot;CODING&quot;, { … })<br>complete_phase(&quot;CODING&quot;)<br>start_phase(&quot;TESTING&quot;)<br>submit_phase_artifact(&quot;TESTING_UNIT&quot;, { … })<br>submit_phase_artifact(&quot;TESTING_E2E&quot;, { … })<br>complete_phase(&quot;TESTING&quot;)<br>finish_workflow</pre><h3>Conclusion: Control Beats Creativity</h3><p>I expected the enforcement layer to make the orchestrator rigid. Instead, it made it better. By moving to this “Gated Agent” approach, you get the reasoning flexibility of an autonomous agent with the structural integrity of a production system.</p><p>State transitions, not prompt instructions. That’s the difference between a system that works in demos and one that works in production.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=768fd1f5431a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>