<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arijit Dutta on Medium]]></title>
        <description><![CDATA[Stories by Arijit Dutta on Medium]]></description>
        <link>https://medium.com/@arijitdutta23?source=rss-4ff8e09a21d9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*CpfyiSuU1eiTJtNV</url>
            <title>Stories by Arijit Dutta on Medium</title>
            <link>https://medium.com/@arijitdutta23?source=rss-4ff8e09a21d9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:56:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arijitdutta23/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evals for Agent Skills: When Can the LLM Do the Job Without a Skill?]]></title>
            <link>https://generativeai.pub/evals-for-agent-skills-when-can-the-llm-do-the-job-without-a-skill-e52bb3f4e6bf?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/e52bb3f4e6bf</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[agent-skills]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Wed, 28 Oct 2026 04:31:14 GMT</pubDate>
            <atom:updated>2026-10-28T04:31:14.384Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>As frontier models become more capable, we need a systematic way to measure whether agent skills still add value or have become unnecessary overhead.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SY_b-3ATgb8ZDzoOTYKYhQ.png" /></figure><h4>TL;DR</h4><ul><li>Agent skills should not be assumed to remain useful forever. As LLMs improve, some skills may become redundant.</li><li>Every major model release should trigger an evaluation comparing the same agent performing identical tasks <strong>with and without the skill</strong>.</li><li>By measuring quality, reliability, cost, and task success, we can decide whether to retain, simplify, improve, or retire a skill.</li><li>Some skills will remain valuable because they contain private knowledge, business rules, or executable workflows that model intelligence alone cannot replace.</li></ul><h4>Skills Are a Bridge Between Model Capabilities and Specialized Tasks</h4><p>Agent skills are a practical way to make general-purpose language models perform specialized tasks reliably.</p><p>A skill typically packages instructions, procedures, examples, reference documents, and sometimes executable scripts. Instead of expecting the LLM to figure everything out independently, we provide a structured workflow.</p><p>Consider a skill that creates a financial analysis report. It might instruct an agent to validate input data, calculate specific metrics, generate charts, follow a standard reporting format, and verify the final output.</p><p>Without that skill, an older LLM might miss important steps, produce inconsistent results, or fail to generate a usable artifact.</p><p>The skill addresses those weaknesses. But what happens when a new frontier model already understands how to perform those tasks?</p><p>It might automatically validate the data, select appropriate visualizations, follow reporting conventions, and check its own output, without needing detailed procedural instructions.</p><p>At that point, the skill may no longer provide a meaningful improvement.</p><blockquote>This leads to an important question: Should skills have an expiration date determined by evaluation?</blockquote><p>Not necessarily a fixed expiration date, but they should have an expiration test.</p><h4>The Real Question: What Additional Value Does the Skill Provide?</h4><p>We generally evaluate an LLM by asking how well it performs a task.</p><p>For agent skills, we need a slightly different evaluation. We need to measure the <em>incremental value</em> of the skill over the model’s existing capabilities.</p><p>The correct comparison is not an agent with an LLM versus an agent without an LLM. It is the same LLM-based agent operating with and without a particular skill.</p><p>Suppose we have an agent that creates professional PowerPoint presentations.</p><p>With an older model, the agent might require a detailed skill covering slide structure, typography, layouts, charts, content density, and export validation.</p><p>A new model might handle most of these requirements independently. If the agent produces equally good presentations without the skill, why continue maintaining and loading it?</p><p>A skill that does not improve outcomes may introduce unnecessary complexity, token consumption, latency, and maintenance work.</p><p>However, we should not assume the skill is redundant simply because the new model appears more capable. We need evidence.</p><h4>A Four-Way Evaluation Framework</h4><p>The most useful approach is to evaluate the skill across both the existing model and the newly released model.</p><p>There are four configurations:</p><ol><li><strong>Old LLM without skill:</strong> Establish the original model’s baseline capability.</li><li><strong>Old LLM with skill:</strong> Measure how much the skill improves the older model.</li><li><strong>New LLM without skill:</strong> Measure how much the model has improved independently.</li><li><strong>New LLM with skill:</strong> Determine whether the skill still provides additional value.</li></ol><p>The central metric is:</p><p><strong>Skill Uplift = Performance Score With Skill − Performance Score Without Skill</strong></p><p>We calculate this separately for each model.</p><p>For example, consider the following hypothetical evaluation results:</p><p><strong>ConfigurationQuality scoreOld LLM without skill65/100Old LLM with skill89/100New LLM without skill90/100New LLM with skill92/100</strong></p><p><em>These numbers are illustrative, not actual benchmark results.</em></p><p>The older model gains 24 points from the skill. The new model gains only 2 points.</p><p>The skill was highly valuable for the older model, but its measured benefit has almost disappeared with the newer model.</p><p>This does not automatically mean we should delete it. A two-point difference might be meaningful for certain business-critical tasks, or it might be normal evaluation noise.</p><p>We need to examine the individual failures, operational costs, and statistical uncertainty before making that decision.</p><p>Nevertheless, the evaluation gives us a concrete signal that the skill should be reviewed.</p><h4>How to Build Evals for Agent Skills</h4><p>A reliable skill evaluation needs more than asking the agent to perform one example task and visually inspecting the result.</p><p>We need a repeatable test suite.</p><h4>1. Create a Representative Task Dataset</h4><p>Start with a collection of tasks that the skill is expected to handle.</p><p>A small initial evaluation might contain 20–50 tasks. Larger suites may be necessary when performance differences are small, or the workflow is business-critical.</p><p>For a presentation-generation skill, the tasks could include creating an executive business review, generating a technical presentation, converting a research report into slides, and producing a presentation containing charts.</p><p>The dataset should include straightforward tasks, complex tasks, ambiguous requests, and edge cases. It should also contain negative tests: situations where the skill should not activate.</p><p>Most importantly, evaluation tasks should reflect real-world usage rather than only examples for which the skill was originally designed. Keep some tasks hidden during skill development to reduce overfitting.</p><h4>2. Run Identical Tasks With and Without the Skill</h4><p>For every evaluation task, run the agent in two configurations:</p><p><strong>Configuration A: Model only</strong></p><p>The agent receives the task, its normal system instructions, required inputs, and available tools, but the specialized skill is disabled.</p><p><strong>Configuration B: Model with skill</strong></p><p>The agent receives the same task and operating environment, with the relevant skill enabled.</p><p>Both configurations should use the same model version, tool permissions, reasoning settings, and resource limits. There is an important complication here.</p><p>Some skills contain more than instructions. They may include proprietary documents, reusable code, or external resources that the base model cannot access.</p><p>If we remove those resources in the model-only configuration, we are not simply measuring whether the model needs procedural guidance. We are also measuring whether it can work without necessary information or tools.</p><p>For such skills, introduce an additional configuration that provides the same resources without the skill’s procedural instructions. This helps separate the value of the model, the instructions, and the supporting assets.</p><h4>3. Define a Consistent Scoring Rubric</h4><p>The agent’s output should be evaluated against predefined success criteria.</p><p>For example, a general-purpose artifact-generation skill could use this illustrative scoring system:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HU3vBD452IfMbwMfYbD1TA.png" /></figure><p>The weights should be adjusted for the task.</p><p>For coding skills, automated tests and working functionality might dominate. For research skills, factual accuracy, source quality, and coverage might be more important. For document-generation skills, structure, formatting, and successful file creation could carry substantial weight.</p><p>Certain requirements should be non-negotiable. A critical security violation, fabricated financial data, or unauthorized external action should not be compensated for by a high score in formatting or presentation quality.</p><p>Such failures should be recorded separately and can trigger an automatic failure. Latency, token consumption, execution cost, and unnecessary tool calls should also be measured, even if they are not included in the quality score.</p><h4>4. Combine Automated and Model-Based Grading</h4><p>Different tasks require different evaluation methods.</p><p><strong>Deterministic graders</strong> verify objectively measurable results: whether a program passes tests, whether an output file exists, whether required fields are present, or whether calculations are correct.</p><p><strong>LLM-based graders</strong> evaluate more subjective dimensions, including clarity, structure, completeness, and usefulness.</p><p><strong>Human reviewers</strong> validate important outcomes and help calibrate the LLM-based graders.</p><p>For subjective grading, reviewers should ideally evaluate anonymized outputs without knowing whether the skill was used.</p><p>This reduces the risk of favoring the skill merely because the evaluation was designed around it.</p><p>The emphasis should be on whether the agent achieved the intended outcome, rather than whether it copied a particular workflow step by step — unless that step is itself an explicit requirement.</p><h4>5. Repeat the Tests</h4><p>LLM behavior can vary between executions. An agent might complete a task correctly once and fail on the next attempt. Therefore, a single successful run is insufficient evidence.</p><p>Execute each configuration multiple times, ideally in isolated environments with reset state. For an initial evaluation, three to five trials per task may reveal instability. More trials and more tasks may be required to distinguish small performance differences confidently.</p><p>Capture the final output, tool execution trace, errors, tokens, latency, and evaluation scores. Then compare not only average quality but also task pass rates, failure frequency, and consistency.</p><p>For stronger statistical conclusions, calculate uncertainty intervals for the paired performance differences. The goal is to determine whether an observed improvement is reliable — not just numerically positive.</p><h4>Don’t Evaluate Only the Final Output</h4><p>An agent skill has at least two separate responsibilities: being selected when appropriate and improving execution after selection.</p><p>Both require evaluation.</p><p>A skill may produce excellent output when manually invoked but almost never activate during normal user requests. Alternatively, it may activate too frequently and interfere with unrelated tasks.</p><p>For this reason, evaluate:</p><ul><li><strong>Trigger accuracy:</strong> Does the agent select the skill when the task requires it?</li><li><strong>False activations:</strong> Does the agent avoid using the skill for unrelated requests?</li><li><strong>Execution quality:</strong> Does the skill improve the actual task outcome?</li><li><strong>Operational efficiency:</strong> Does it reduce or increase latency, token usage, tool calls, and cost?</li><li><strong>Reliability:</strong> Does it help the agent succeed consistently across repeated runs?</li></ul><p>Evaluate automatic skill selection separately from forced invocation. A skill that is useful only when explicitly selected may require better metadata or trigger instructions rather than retirement.</p><p>This approach aligns with OpenAI’s published guidance on systematically testing agent skills and Anthropic’s work on evaluating agent behavior.</p><h4>Make Skill Retirement an Evidence-Based Decision</h4><p>Once the evaluations are complete, each skill can receive one of four decisions.</p><h4>Retain</h4><p>The skill provides a measurable improvement in quality, reliability, safety, or efficiency.</p><p>It may also contain business-specific knowledge, required procedures, or executable assets that the model cannot independently reproduce.</p><h4>Simplify</h4><p>The newer model already handles most of the task, but still benefits from a few instructions or resources.</p><p>Instead of keeping a long procedural skill, remove unnecessary explanations and retain only the parts that produce measurable value.</p><p>For example, a 2,000-word report-writing skill might be reduced to a short description of mandatory business conventions and a validation checklist.</p><h4>Retire</h4><p>The model consistently performs the task without the skill, with no meaningful loss in quality or reliability.</p><p>The skill also provides no unique data, required workflow, or tooling advantage.</p><p>In this case, retiring it reduces maintenance and operational complexity.</p><p>Retirement should be reversible. Keep its version history and evaluation results so the decision can be revisited if performance regresses.</p><h4>Improve</h4><p>The skill is still needed, but evaluation shows that it is not delivering its intended benefit.</p><p>Perhaps its instructions are outdated, the triggering rules are unreliable, or the workflow conflicts with the new model’s strengths.</p><p>This is a reason to redesign and retest the skill rather than automatically preserve or delete it.</p><h4>Not All Skills Will Become Obsolete</h4><p>The idea that skills are temporary workarounds is useful, but it needs one important qualification.</p><p>We should distinguish between skills that compensate for model limitations and skills that provide external knowledge or operational control.</p><p>A skill that explains how to summarize a document may become redundant as models become better at summarization.</p><p>But a skill that defines an organization’s approved financial reporting procedure is different.</p><p>Even an exceptionally capable LLM cannot be expected to know a private, recently updated company policy unless that information is made available to it.</p><p>Similarly, a skill containing deterministic scripts may remain valuable because execution through tested code can offer guarantees that free-form generation cannot.</p><p>Some skills exist to improve model intelligence in a narrow domain. Others exist to encode organizational requirements.</p><p>The first category may gradually shrink as frontier models improve.</p><p>The second category may remain important regardless of model capability.</p><blockquote>The objective should not be to eliminate all skills. It should be to eliminate unnecessary skills while preserving genuinely valuable expertise and operational controls.</blockquote><h4>Evals Should Be Part of Every Model Upgrade</h4><p>Today, teams often migrate existing prompts, agents, and skills to newer models without questioning whether all those components are still needed.</p><p>That approach can preserve technical debt. A better strategy is to treat every major frontier model release as an opportunity to review the skill library.</p><p>The evaluation system should automatically run a predefined set of tasks against the current production model and the candidate replacement, both with and without each relevant skill.</p><p>A practical evaluation report should contain the model version, skill version, task coverage, overall quality, task-level failures, skill uplift, critical regressions, execution cost, and recommended action.</p><p>For highly important skills, a model upgrade should not proceed until the evaluation confirms that mandatory requirements are still satisfied.</p><p>This process should also run after significant skill changes and periodically as real user tasks evolve. An evaluation suite is not a one-time benchmark. It is part of the lifecycle of an agent system.</p><h4>The Future: Skills That Continuously Justify Their Existence</h4><p>As LLMs grow more capable, the relationship between models and skills will change.</p><p>A task that required extensive procedural guidance today may require only a few constraints tomorrow. Eventually, some tasks may require no specialized guidance at all.</p><p>Without systematic evaluation, organizations could accumulate hundreds of skills that no longer improve performance; or worse, actively constrain more capable models.</p><p>A useful long-term direction is a skill registry that stores each skill’s purpose, supported models, benchmark results, maintenance owner, and latest evaluation status.</p><p>When a new model becomes available, the evaluation suite can identify which skills still provide value.</p><p>Skills with negligible benefits become candidates for simplification or retirement. Skills that address genuine capability gaps remain active.</p><p>Over time, the library evolves alongside the models rather than becoming a collection of permanent instructions.</p><p>The key principle is simple:</p><blockquote>A skill should not remain in an agent because it was useful when it was created. It should remain because evaluations demonstrate that it is still useful.</blockquote><p>As frontier models improve, the challenge is no longer just teaching agents how to perform specialized tasks. It is knowing when we can stop teaching them and confidently let the model do the job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*VzJTCfbpIcrT9bfe.png" /></figure><p>This story is published under the <a href="https://generativeai.pub/">Generative AI</a> publication. Connect with us on <a href="https://www.linkedin.com/company/generative-ai-publication">LinkedIn</a> and follow <a href="https://www.zeniteq.com/">Zeniteq</a> to stay in the loop with the latest AI stories. Let’s shape the future of AI together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*jfeROV4JdK4xEMC_.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e52bb3f4e6bf" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/evals-for-agent-skills-when-can-the-llm-do-the-job-without-a-skill-e52bb3f4e6bf">Evals for Agent Skills: When Can the LLM Do the Job Without a Skill?</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Preserve Cursor Agent Chats Across EC2 Rebuilds]]></title>
            <link>https://medium.com/@arijitdutta23/how-to-preserve-cursor-agent-chats-across-ec2-rebuilds-7431cb27cd9b?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7431cb27cd9b</guid>
            <category><![CDATA[cursor]]></category>
            <category><![CDATA[ai-coding-agent]]></category>
            <category><![CDATA[how-to]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Thu, 08 Oct 2026 14:09:22 GMT</pubDate>
            <atom:updated>2026-10-08T14:09:22.097Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kls_qz_umeMazkopB-OJ1g.png" /></figure><h3>Problem</h3><p>One of my development environment runs on an AWS EC2 instance that is rebuilt every two weeks. Each restack/rebuild can change both the instance IP address and the project checkout path.</p><p>Cursor associates Remote SSH chats with the remote workspace identity, so connecting to a new host address or opening a different project path can make previous Agent conversations disappear from that workspace.</p><p>The goal is to keep the Cursor workspace identity stable across EC2 rebuilds, even when the underlying instance and checkout directory change.</p><p>If your EC2 instance is rebuilt regularly, you can keep Cursor attached to the same logical workspace by keeping two things stable:</p><ul><li>the SSH host alias</li><li>the project path opened in Cursor</li></ul><h3>TL;DR</h3><p>Use a permanent SSH alias such as cursor-dev and a permanent symlink such as /home/&lt;user&gt;/myrepo.</p><p>After each rebuild, change what they point to, not their names.</p><h3>Why chats seem to disappear</h3><p>Cursor stores normal Agent chat history locally on the machine running Cursor.</p><p>For Remote SSH workspaces, the workspace identity depends on the SSH host name and remote project path. Conceptually:</p><pre>vscode-remote://ssh-remote+cursor-dev/home/&lt;user&gt;/myrepo</pre><p>If you connect using a new EC2 IP directly or open a different checkout path, Cursor can see it as a different workspace.</p><p>The old chats are still local, but they may no longer appear in that window.</p><h3>One-time setup</h3><p>On your laptop, create a stable SSH alias in ~/.ssh/config:</p><pre>Host cursor-dev<br>    HostName &lt;ec2-ip&gt;<br>    User &lt;user&gt;</pre><p>On EC2, create a stable symlink to the current checkout:</p><pre>ln -sfn /home/&lt;user&gt;/work/&lt;checkout&gt; /home/&lt;user&gt;/myrepo</pre><p>Then connect to:</p><pre>cursor-dev</pre><p>and always open:</p><pre>/home/&lt;user&gt;/myrepo</pre><p>Start Agent conversations only after that folder is open.</p><h3>After every EC2 rebuild</h3><p>Update only the EC2 address in your SSH config:</p><pre>Host cursor-dev<br>    HostName &lt;new-ec2-ip&gt;<br>    User &lt;user&gt;</pre><p>Then recreate or update the symlink on the new instance:</p><pre>ln -sfn /home/&lt;user&gt;/work/&lt;new-checkout&gt; /home/&lt;user&gt;/myrepo</pre><p>Confirm it:</p><pre>ls -ld /home/&lt;user&gt;/myrepo</pre><p>Finally, reconnect to cursor-dev and open /home/&lt;user&gt;/myrepo.</p><p>Your stable identity remains:</p><pre>cursor-dev + /home/&lt;user&gt;/myrepo</pre><p>while the EC2 IP and real checkout directory can change underneath it.</p><h3>What to avoid</h3><p>Don’t connect using:</p><pre>user@&lt;new-ip&gt;</pre><p>and don’t open the real changing directory:</p><pre>/home/&lt;user&gt;/work/instanceID024<br>/home/&lt;user&gt;/work/instanceID090</pre><p>Always use:</p><pre>cursor-dev<br>/home/&lt;user&gt;/myrepo</pre><p>One important distinction: this setup preserves the <strong>workspace identity</strong>; it is not a backup. Cursor’s normal chat history still lives on your laptop, so losing or resetting that local Cursor data is a separate problem.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7431cb27cd9b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a Reliable Natural-Language-to-SQL Agent]]></title>
            <link>https://generativeai.pub/building-a-reliable-natural-language-to-sql-agent-f58eea891caf?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f58eea891caf</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Wed, 16 Sep 2026 01:11:03 GMT</pubDate>
            <atom:updated>2026-09-16T01:11:03.214Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Generating SQL is easy. Generating the SQL that matches what the business actually means is the difficult part.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1x01viqP7Zc67Q6yqt-J8A.png" /></figure><h4>TL;DR</h4><p>A reliable natural-language-to-SQL system should not send a user’s question directly to an LLM and hope for the best.</p><p>It should first retrieve the relevant schema, resolve business terminology against authoritative metadata, derive joins from documented relationships, account for cardinality and effective-dated data, generate a read-only query, and validate the result before returning it.</p><p>A few important principles are:</p><ul><li>Retrieve only the schema relevant to the question.</li><li>Map business terms to documented fields and definitions rather than model assumptions.</li><li>Prefer authoritative metadata over naming similarity.</li><li>Derive joins from documented relationships, not columns that merely look related.</li><li>Understand relationship cardinality and temporal rules before joining tables.</li><li>Treat existing application SQL as supporting evidence, not unquestioned truth.</li><li>Separate syntactic validity, safety, schema validity, and semantic correctness.</li><li>Surface ambiguity instead of silently inventing a business definition.</li><li>Keep database execution separate from SQL generation unless execution is explicitly authorized.</li></ul><p>The hard part of Text-to-SQL is not SQL syntax. It is grounding business language in the database’s actual meaning.</p><h4>A Plausible Query Can Still Be Wrong</h4><p>Suppose someone asks an AI system:</p><blockquote>Show me all active Finance employees who are eligible for remote work.</blockquote><p>At first glance, this sounds like a simple SQL-generation task. An LLM might immediately produce a query joining an EMPLOYEES table to a DEPARTMENTS table. The SQL might compile. It might return rows. It might even look completely reasonable to an experienced developer. But that still does not mean the query is correct.</p><p>The real question is not:</p><blockquote>Can the model write SQL?</blockquote><p>It is:</p><blockquote>Can the system translate the user’s business language into the database’s actual meaning?</blockquote><p>That distinction is the foundation of a reliable natural-language-to-SQL agent.</p><h4>The Problem With “Just Ask the LLM to Write SQL”</h4><p>Imagine a simplified Oracle database containing employee information.</p><p>A user asks:</p><blockquote>Find active employees in Finance who are eligible for remote work.</blockquote><p>The model sees several familiar concepts:</p><pre>employee<br>department<br>active<br>Finance<br>remote work</pre><p>It could generate something like:</p><pre>SELECT e.employee_id,<br>       e.employee_name<br>FROM   employees e<br>JOIN   departments d<br>       ON d.department_id = e.department_id<br>WHERE  d.department_name = &#39;Finance&#39;<br>AND    e.status = &#39;ACTIVE&#39;<br>AND    e.office_location IS NULL;</pre><p>The SQL looks plausible.</p><p>The problem is the last condition:</p><pre>e.office_location IS NULL</pre><p>Why should a missing office location mean that an employee is eligible for remote work?</p><blockquote>The model invented that relationship.</blockquote><p>Perhaps the real system contains a separate table:</p><pre>EMPLOYEE_WORK_POLICY<br>--------------------<br>EMPLOYEE_ID<br>REMOTE_ELIGIBLE<br>HYBRID_ELIGIBLE<br>POLICY_EFFECTIVE_DATE</pre><p>and its metadata says:</p><pre>REMOTE_ELIGIBLE<br>Indicates whether the employee is currently approved for remote work.</pre><p>In that case, REMOTE_ELIGIBLE = &#39;Y&#39; is the authoritative business definition. The first query may be syntactically correct while answering the wrong question. That is the central problem in natural-language-to-SQL systems.</p><h4>A Better Architecture</h4><p>Instead of sending the user’s question directly to an LLM, treat SQL generation as a small reasoning pipeline:</p><pre>User question<br>      ↓<br>Identify entities and business terms<br>      ↓<br>Retrieve relevant schema<br>      ↓<br>Map business terms to authoritative fields<br>      ↓<br>Determine documented relationships<br>      ↓<br>Generate one read-only SQL query<br>      ↓<br>Validate query structure<br>      ↓<br>Return SQL with schema evidence and assumptions</pre><p>Each stage prevents a different type of mistake. The language model still plays an important role, but it works from retrieved facts instead of trying to reconstruct the database from general knowledge.</p><h4>1. Start With a Read-Only Boundary</h4><p>A natural-language-to-SQL assistant should normally generate queries, not modify data.</p><p>A simple safety policy can allow:</p><pre>SELECT ...</pre><p>and:</p><pre>WITH ...<br>SELECT ...</pre><p>while rejecting operations such as:</p><pre>INSERT<br>UPDATE<br>DELETE<br>MERGE<br>CREATE<br>ALTER<br>DROP<br>TRUNCATE</pre><p>The validator should also consider less obvious cases such as:</p><pre>SELECT ... FOR UPDATE</pre><p>multiple SQL statements, stored-procedure execution, or constructs that create or modify database objects. This is important, but it solves only the safety problem. A read-only query can still be completely wrong.</p><h4>2. Retrieve Only the Schema You Need</h4><p>Large enterprise databases can contain thousands of tables, views, columns, relationships, extensions, and historical objects.</p><p>Putting the complete database dictionary into the model’s context creates several problems:</p><ul><li>unnecessary token usage;</li><li>more similarly named fields to confuse;</li><li>higher chances of choosing obsolete objects;</li><li>more opportunities to invent incorrect joins.</li></ul><p>Instead, retrieve schema incrementally.</p><p>For our employee question, the first retrieval might search for concepts such as:</p><pre>employee<br>department<br>active employee<br>remote eligible<br>remote work</pre><p>That search could return:</p><pre>EMPLOYEES<br>- EMPLOYEE_ID<br>- EMPLOYEE_NAME<br>- DEPARTMENT_ID<br>- EMPLOYMENT_STATUS</pre><pre>DEPARTMENTS<br>- DEPARTMENT_ID<br>- DEPARTMENT_NAME</pre><pre>EMPLOYEE_WORK_POLICY<br>- EMPLOYEE_ID<br>- REMOTE_ELIGIBLE<br>- POLICY_EFFECTIVE_DATE</pre><p>This is far more useful than giving the model hundreds of unrelated HR, payroll, finance, security, and audit tables.</p><p>The principle is simple:</p><blockquote>Retrieve the smallest amount of schema that can answer the question confidently.</blockquote><h4>3. Search Business Descriptions, Not Just Column Names</h4><p>A schema search should not stop at names. Business concepts often appear in metadata descriptions rather than table or column names.</p><p>Consider the phrase:</p><blockquote>active employee</blockquote><p>The database might not have a column called:</p><pre>ACTIVE_EMPLOYEE</pre><p>Instead, it might have:</p><pre>EMPLOYMENT_STATUS</pre><pre>Description:<br>Current employment status of the worker.<br>Possible values include ACTIVE, TERMINATED, and LEAVE.</pre><p>Now the agent has evidence for:</p><pre>e.employment_status = &#39;ACTIVE&#39;</pre><p>The same applies to remote eligibility.</p><p>If metadata says:</p><pre>REMOTE_ELIGIBLE</pre><pre>Description:<br>Indicates whether the employee is currently approved<br>to work remotely.</pre><p>then this:</p><pre>wp.remote_eligible = &#39;Y&#39;</pre><p>is stronger than guessing from:</p><ul><li>office location;</li><li>job title;</li><li>department;</li><li>manager;</li><li>address;</li><li>work schedule.</li></ul><p>Business descriptions are often where the real semantics live.</p><h4>4. Organization-Specific Tables Matter</h4><p>Enterprise systems rarely consist only of standard vendor tables, especially when organizations extend off-the-shelf applications.</p><p>Companies add:</p><ul><li>extension tables;</li><li>custom attributes;</li><li>reporting views;</li><li>integration tables;</li><li>application-specific metadata;</li><li>historical compatibility structures.</li></ul><p>Imagine that employee information comes from standard HR tables, while remote-work approval is stored in a company-specific table:</p><pre>CORP_EMPLOYEE_WORK_POLICY</pre><p>If the agent searches only the standard employee schema, it may never discover the field that actually answers the question.</p><p>A practical retrieval sequence can therefore look like:</p><ol><li>standard database schema;</li><li>application-specific schema;</li><li>organization-specific extensions;</li><li>metadata descriptions;</li><li>existing application SQL;</li><li>live metadata inspection, when explicitly authorized.</li></ol><p>Custom schema is not automatically less trustworthy. For some business questions, it may contain the most authoritative definition.</p><h4>5. Never Guess a Join Because the Columns Look Similar</h4><p>Joining tables is one of the easiest places for an LLM to produce convincing but incorrect SQL.</p><p>Suppose the schema documents:</p><pre>EMPLOYEES.DEPARTMENT_ID<br>References DEPARTMENTS.DEPARTMENT_ID</pre><p>That supports:</p><pre>e.department_id = d.department_id</pre><p>Likewise, if:</p><pre>EMPLOYEE_WORK_POLICY.EMPLOYEE_ID<br>References EMPLOYEES.EMPLOYEE_ID</pre><p>then the second join is clear:</p><pre>wp.employee_id = e.employee_id</pre><p>Now imagine two tables both contain:</p><pre>NAME</pre><p>That is not enough evidence to write:</p><pre>a.name = b.name</pre><p>Generic column names such as NAME, TYPE, CODE, STATUS, or ID are especially dangerous.</p><p>A reliable agent derives joins from:</p><ul><li>foreign-key metadata;</li><li>documented object references;</li><li>data-dictionary relationships;</li><li>explicit field descriptions;</li><li>trusted application queries;</li><li>verified schema documentation.</li></ul><p>A join should exist because the schema says the objects are related, not because the column names happen to resemble each other.</p><h4>6. Existing Application SQL Can Be Valuable Evidence</h4><p>Formal schemas do not always document every implementation detail. Existing code can help.</p><p>For example, suppose several production reports contain:</p><pre>JOIN employee_work_policy wp<br>  ON wp.employee_id = e.employee_id</pre><p>That does not automatically prove every future query should use the same join. But it is useful supporting evidence.</p><p>Repositories can reveal important database behavior through:</p><ul><li>reporting queries;</li><li>batch jobs;</li><li>stored queries;</li><li>integration code;</li><li>migration scripts;</li><li>test utilities.</li></ul><p>The key is to use existing SQL as evidence, not as unquestioned truth. Old code may be obsolete, application-specific, or built around assumptions that no longer apply.</p><h4>7. Separate SQL Validity From SQL Correctness</h4><p>A useful mental model is to evaluate SQL at four levels.</p><h4>Level 1: Syntactic validity</h4><p>Will Oracle accept the SQL statement?</p><h4>Level 2: Safety</h4><p>Is it a read-only query?</p><h4>Level 3: Schema validity</h4><p>Do the referenced tables and columns actually exist?</p><h4>Level 4: Semantic correctness</h4><p>Does the query answer what the user meant?</p><p>The fourth level is the hardest.</p><p>Consider:</p><pre>e.office_location IS NULL</pre><p>versus:</p><pre>wp.remote_eligible = &#39;Y&#39;</pre><p>Both may be valid Oracle SQL. Both may reference real columns. Both may return results. But only one may represent the company’s official meaning of “eligible for remote work.” A SQL parser cannot discover that distinction. Schema grounding can.</p><h4>8. The Employee Example, End to End</h4><p>Now consider the original request:</p><blockquote>Show all active Finance employees who are eligible for remote work.</blockquote><h4>Step 1: Identify the concepts</h4><p>The agent extracts:</p><pre>Entity: Employee<br>Entity: Department</pre><pre>Business terms:<br>- active employee<br>- Finance<br>- eligible for remote work</pre><h4>Step 2: Retrieve employee schema</h4><p>The system discovers:</p><pre>EMPLOYEES<br>EMPLOYEE_ID<br>EMPLOYEE_NAME<br>DEPARTMENT_ID<br>EMPLOYMENT_STATUS</pre><p>Metadata defines EMPLOYMENT_STATUS = &#39;ACTIVE&#39; as the required employment state.</p><h4>Step 3: Resolve Finance</h4><p>Schema retrieval finds:</p><pre>DEPARTMENTS<br>DEPARTMENT_ID<br>DEPARTMENT_NAME</pre><p>with a documented relationship:</p><pre>EMPLOYEES.DEPARTMENT_ID<br>    →<br>DEPARTMENTS.DEPARTMENT_ID</pre><h4>Step 4: Resolve remote eligibility</h4><p>A metadata-description search finds:</p><pre>EMPLOYEE_WORK_POLICY.REMOTE_ELIGIBLE</pre><pre>Indicates whether the employee is currently<br>approved for remote work.</pre><p>The schema also documents:</p><pre>EMPLOYEE_WORK_POLICY.EMPLOYEE_ID<br>    →<br>EMPLOYEES.EMPLOYEE_ID</pre><h4>Step 5: Generate SQL</h4><p>The grounded query becomes:</p><pre>SELECT e.employee_id,<br>       e.employee_name,<br>       d.department_name<br>FROM   employees e<br>JOIN   departments d<br>       ON d.department_id = e.department_id<br>JOIN   employee_work_policy wp<br>       ON wp.employee_id = e.employee_id<br>WHERE  e.employment_status = &#39;ACTIVE&#39;<br>AND    d.department_name = :department_name<br>AND    wp.remote_eligible = &#39;Y&#39;;</pre><p>with:</p><pre>:department_name = Finance</pre><p>The important achievement is not the SQL formatting.</p><p>It is how every important part of the query was justified:</p><ul><li>EMPLOYMENT_STATUS defines “active.”</li><li>DEPARTMENT_NAME identifies Finance.</li><li>REMOTE_ELIGIBLE defines remote-work eligibility.</li><li>both joins come from documented relationships.</li><li>the external department value uses a bind variable.</li></ul><p>That is what makes the query reviewable.</p><h4>Prototype: A Schema-Grounded SQL Skill</h4><p>The prototype turns a natural-language question into a single validated, read-only SQL query.</p><p>Instead of asking the model to generate SQL directly, the skill first retrieves only the relevant schema definitions, searches field descriptions for business meaning, derives joins from documented relationships, and validates the generated query against safety rules.</p><p>The skill should also explain the tables, columns, joins, and assumptions behind the SQL so the result can be reviewed before execution.</p><p>Database execution remains a separate step and should happen only after explicit approval.</p><h4>Prototype File Structure</h4><pre>natural-language-sql/<br>├── SKILL.md<br>├── examples.md<br>├── joins-and-business.md<br>└── scripts/<br>    ├── extract_schema.py<br>    └── assert_select_only.py</pre><h4>What Each File Does</h4><ul><li><strong>SKILL.md</strong> — Defines when the skill should be triggered, the safety rules, schema lookup order, and the overall natural-language-to-SQL workflow.</li><li><strong>examples.md</strong> — Contains worked natural-language-to-SQL examples, expected outputs, and common failure cases.</li><li><strong>joins-and-business.md</strong> — Documents how to resolve business terms, derive joins from evidence, and handle more complex relationships such as nested structures.</li><li><strong>scripts/extract_schema.py</strong> — Retrieves only the relevant tables, fields, relationships, and business descriptions needed for the current question.</li><li><strong>scripts/assert_select_only.py</strong> — Validates the generated SQL, rejects unsafe or multi-statement queries, and allows only read-only statements.</li></ul><p>The goal of this prototype is intentionally narrow: <strong>generate grounded SQL, explain why it is correct, and keep execution separate.</strong> This makes the skill easier to test, review, and improve before connecting it to a live database.</p><h4>9. What the Agent Should Return</h4><p>A production-quality SQL agent should return more than a code block. A useful response can contain:</p><p><strong>SQL — </strong>The single read-only query.</p><p><strong>Business-term mapping</strong></p><pre>&quot;active&quot;<br>→ EMPLOYEES.EMPLOYMENT_STATUS = &#39;ACTIVE&#39;</pre><pre>&quot;Finance&quot;<br>→ DEPARTMENTS.DEPARTMENT_NAME</pre><pre>&quot;eligible for remote work&quot;<br>→ EMPLOYEE_WORK_POLICY.REMOTE_ELIGIBLE = &#39;Y&#39;</pre><p><strong>Join evidence</strong></p><pre>EMPLOYEES.DEPARTMENT_ID<br>→ DEPARTMENTS.DEPARTMENT_ID</pre><pre>EMPLOYEE_WORK_POLICY.EMPLOYEE_ID<br>→ EMPLOYEES.EMPLOYEE_ID</pre><p><strong>Assumptions</strong></p><p>Any business meaning or schema relationship that could not be fully confirmed.</p><p><strong>Bind variables</strong></p><p>Values supplied by the user rather than inserted directly into the SQL string.</p><p>This makes mistakes visible instead of hiding uncertainty behind polished SQL.</p><h4>A Compact Checklist</h4><p>Before returning generated SQL, ask:</p><ul><li>Is the statement a single read-only query?</li><li>Does every referenced table come from retrieved schema?</li><li>Does every referenced column actually exist?</li><li>Were important business terms searched in metadata descriptions?</li><li>Were application-specific or custom tables considered?</li><li>Are joins backed by documented relationships?</li><li>Are stable identifiers used instead of fragile text matching where possible?</li><li>Are user-provided values represented with bind variables?</li><li>Are unresolved assumptions clearly identified?</li><li>Is execution separated from generation?</li></ul><p>If the agent cannot answer one of these confidently, it should say so rather than inventing the missing information.</p><h4>Key Takeaway</h4><p>Natural-language-to-SQL is not mainly a SQL-writing problem. Modern language models are already good at arranging known tables, columns, filters, and joins into SQL. The difficult part is supplying the right facts.</p><p>A reliable system therefore spends much of its effort on <strong>schema retrieval, business-definition mapping, relationship evidence, safety validation, and assumption tracking</strong>.</p><p>The best SQL agent is not the one that writes a query fastest. It is the one that can explain why every important table, join, and filter belongs there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*L5gX2Hm2p5WTME5e.png" /></figure><p>This story is published under the <a href="https://generativeai.pub/">Generative AI</a> publication. Connect with us on <a href="https://www.linkedin.com/company/generative-ai-publication">LinkedIn</a> and follow <a href="https://www.zeniteq.com/">Zeniteq</a> to stay in the loop with the latest AI stories. Let’s shape the future of AI together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*i5X-zF9RPimmLI4t.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f58eea891caf" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/building-a-reliable-natural-language-to-sql-agent-f58eea891caf">Building a Reliable Natural-Language-to-SQL Agent</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Pack Compiler: Give Coding Agents Less Context, Not More]]></title>
            <link>https://generativeai.pub/context-pack-compiler-give-coding-agents-less-context-not-more-0e78112dceed?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e78112dceed</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[context]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Thu, 10 Sep 2026 11:14:28 GMT</pubDate>
            <atom:updated>2026-09-10T11:14:28.235Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Instead of feeding an AI coding agent an entire repository, give it the smallest slice of the codebase needed to solve the current task.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ydzMFLQxX0nwt1jMLkB6w.png" /></figure><h4>TL;DR</h4><ul><li>AI coding agents often suffer from <strong>too much irrelevant repository context</strong>, not simply too little context.</li><li>A Context Pack Compiler searches the repository broadly but selects only the files, symbols, tests, architectural rules, and constraints that materially affect the task.</li><li>It does <strong>not eliminate repository search</strong>. Its value is turning repository exploration into a small, reusable, task-specific package that other agents can consume.</li><li>For a single capable coding agent, this may simply be a good context-management strategy. It becomes more valuable when multiple agents, reviews, retries, or handoffs are involved.</li></ul><p>AI coding agents need context to write correct code or debug an issue. The obvious response is to give them more of it. More files. More documentation. More source code. More tests. Maybe even the entire repository.</p><p>That sounds reasonable, but it creates another problem: <strong>relevant information gets buried inside irrelevant information</strong>. A better approach may be to treat context like something we compile.</p><p>Given an engineering task, search the repository, identify exactly what matters, and create a small package containing only the information required to implement that task correctly.</p><p>That is the idea behind a <strong>Context Pack Compiler</strong>.</p><h4>The problem: more context is not always better</h4><p>Imagine asking a coding agent:</p><blockquote>Add retry support to failed payment webhook processing.</blockquote><p>A large application might contain thousands of files.</p><p>Only a handful may actually matter:</p><ul><li>the webhook entry point;</li><li>the payment processing service;</li><li>an existing retry implementation;</li><li>the relevant interfaces;</li><li>configuration for background jobs;</li><li>tests around webhook failures;</li><li>an architecture decision describing how retries should work.</li></ul><blockquote>The coding agent does not need the UI components, analytics pipeline, authentication pages, unrelated database models, build scripts, and hundreds of other files.</blockquote><p>Yet repository exploration can gradually pull all of this information into the agent’s working context. The result is not necessarily better understanding. It can create noise, overload the precious context area.</p><p>The real goal should therefore not be:</p><blockquote>Give the model as much repository context as possible.</blockquote><p>It should be:</p><blockquote>Give the model the minimum context required to make the correct change.</blockquote><h4>What a Context Pack Compiler does</h4><p>A Context Pack Compiler receives an engineering task and builds a task-specific representation of the repository.</p><p>For example:</p><pre>Task:<br>Add retry support to failed payment webhook processing.</pre><p>The compiler investigates the repository and might produce:</p><pre>.context/<br>├── task.md<br>├── architecture.md<br>├── symbols.md<br>├── tests.md<br>├── constraints.md<br>└── skills.md</pre><p>Instead of handing the implementation agent hundreds of files, it might identify ten important ones.</p><p>For each file, it explains why the file matters:</p><pre>src/webhooks/payment-handler.ts</pre><pre>WHY_REQUIRED:<br>Entry point responsible for processing payment webhook events.</pre><p>Another might be:</p><pre>src/jobs/retry-policy.ts</pre><pre>WHY_REQUIRED:<br>Existing retry mechanism used by background jobs and likely<br>the preferred implementation pattern.</pre><p>The key idea is deliberate selection. Every included piece of context has to justify its presence.</p><h4>But doesn’t the compiler still search the whole repository?</h4><p>Yes. And this is an important distinction. A Context Pack Compiler does <strong>not</strong> eliminate repository search. It may still search broadly across the entire codebase.</p><p>The difference is between <strong>searching information</strong> and <strong>loading information into the agent’s working context</strong>.</p><p>A repository might contain:</p><pre>20,000 files</pre><p>The compiler can search file names, symbols, imports, references, test names, and documentation across all 20,000 files. But perhaps it deeply reads only 12.</p><p>The flow becomes:</p><pre>20,000 repository files<br>        ↓<br>search and inspect<br>        ↓<br>identify relevant code<br>        ↓<br>10-15 important files/symbols<br>        ↓<br>implementation agent</pre><p>Searching broadly is cheap compared with forcing a reasoning model to actively consider thousands of unrelated pieces of code. The compiler’s job is therefore <strong>filtering</strong>, not avoiding search.</p><h4>The compiler should think in symbols, not just files</h4><p>A basic version of this idea could simply produce a file list. That helps, but files are often too coarse. Programming systems are built from relationships between functions, classes, interfaces, services, and modules.</p><p>Consider:</p><pre>WebhookRouter.dispatch()<br>        ↓<br>PaymentWebhookHandler.handle()<br>        ↓<br>PaymentService.process()<br>        ↓<br>LedgerWriter.write()</pre><p>That relationship tells the coding agent much more than four file names. A useful context compiler should therefore build a lightweight task-specific symbol graph.</p><p>For example:</p><pre>SYMBOL: PaymentWebhookHandler.handle</pre><pre>DEFINED_IN:<br>src/webhooks/payment-handler.ts</pre><pre>CALLED_BY:<br>WebhookRouter.dispatch</pre><pre>CALLS:<br>PaymentService.process<br>RetryQueue.enqueue</pre><pre>TESTED_BY:<br>tests/webhooks/payment-handler.test.ts</pre><p>Now the downstream agent understands not only what code exists but how the relevant pieces interact. This is closer to a <strong>task-specific program slice</strong> than a file search.</p><h4>Existing patterns matter as much as target code</h4><p>One of the easiest ways for a coding agent to produce inconsistent code is to understand the target implementation but miss how the repository normally solves similar problems.</p><p>Suppose the repository already has retry handling for email jobs.</p><p>A compiler could discover:</p><pre>EmailJob<br>   ↓<br>RetryPolicy<br>   ↓<br>JobQueue</pre><p>That implementation might reveal an architectural convention:</p><blockquote>Retry behavior belongs in the shared RetryPolicy rather than individual handlers.</blockquote><p>Without this context, the coding agent might implement a completely new retry loop inside the payment webhook handler. The code could work while still violating the architecture.</p><p>This is why a useful context pack should deliberately search for <strong>analogous implementations</strong>, not only direct dependencies.</p><h4>Tests are part of the context, not an afterthought</h4><p>Tests often encode requirements that aren’t documented elsewhere.</p><p>A test may reveal that:</p><ul><li>retries stop after three attempts;</li><li>a particular error is considered permanent;</li><li>failed jobs must preserve an idempotency key;</li><li>certain exceptions must not be retried;</li><li>metrics must be emitted when retries occur.</li></ul><p>These behaviors may not be obvious from the implementation itself. A Context Pack Compiler should therefore treat relevant tests as first-class context.</p><p>Its output might say:</p><pre>tests/webhooks/payment-handler.test.ts</pre><pre>WHY_REQUIRED:<br>Defines expected webhook failure behavior and existing<br>idempotency guarantees.</pre><p>The implementation agent now starts with the behavioral contract instead of discovering it after breaking tests.</p><h4>Architecture documents can prevent locally correct but globally wrong code</h4><p>Repositories often contain knowledge outside source files:</p><pre>docs/<br>ADR/<br>README files<br>architecture guides<br>configuration files<br>team conventions</pre><p>These files can contain constraints that are invisible in the code immediately surrounding the task.</p><p>For example:</p><pre>ADR-014:<br>Network retries must run through the background job system.<br>Request handlers must not block while retrying.</pre><p>A model that only sees the webhook controller could easily produce a retry loop inside the request. Locally, the code looks reasonable. Architecturally, it is wrong.</p><p>The context compiler should actively look for these constraints before implementation begins.</p><h4>A practical context package</h4><p>A compact package could contain six sections.</p><pre>.context/<br>├── task.md<br>├── architecture.md<br>├── symbols.md<br>├── patterns.md<br>├── tests.md<br>└── constraints.md</pre><p>task.md explains what needs to change and how the compiler interprets the task.</p><p>architecture.md describes the relevant subsystem and request flow.</p><p>symbols.md maps important functions, classes, interfaces, callers, and dependencies.</p><p>patterns.md identifies similar implementations already present in the repository.</p><p>tests.md lists behavioral tests and important expected outcomes.</p><p>constraints.md captures configuration, ADRs, compatibility requirements, and architectural rules.</p><p>The implementation agent can then begin from a compact representation of the problem rather than reconstructing the entire repository architecture from scratch.</p><h4>Why this becomes more valuable with multiple agents</h4><p>For a single sophisticated coding agent, a separate compiler may not always be necessary.</p><p>The agent could simply follow good context discipline:</p><pre>Find entry point<br>↓<br>Trace dependencies<br>↓<br>Find similar implementations<br>↓<br>Find tests<br>↓<br>Read architectural constraints<br>↓<br>Implement</pre><p>In that case, Context Pack Compiler is more of a <strong>workflow rule</strong> than a separate agent. The idea becomes more interesting when several agents are involved.</p><p>Consider:</p><pre>Repository<br>    ↓<br>Context Pack Compiler<br>    ↓<br>Task Context<br>   ↙   ↓   ↘<br>Coder Reviewer Test Agent</pre><p>Without the compiler, each agent performs its own repository exploration. The coder develops one mental model. The reviewer develops another. The test agent develops another.</p><p>With a shared context package, they begin from the same architectural baseline. This can reduce duplicated exploration and make handoffs more consistent.</p><h4>It can also help when an agent fails</h4><p>Suppose the first coding agent produces an incorrect implementation. Without persistent task context, a second agent may repeat much of the same repository exploration.</p><p>With a compiled context package:</p><pre>Task<br> ↓<br>Compile context once<br> ↓<br>.context/<br> ↓<br>Agent attempt 1<br> ↓<br>Review<br> ↓<br>Agent attempt 2</pre><p>The expensive discovery step becomes reusable. That is an important distinction. The compiler isn’t necessarily reducing how much the repository must be searched.</p><p>It is reducing how often the same repository understanding must be reconstructed.</p><h4>The file limit should be a budget, not a rule</h4><p>A useful default might be:</p><blockquote>Prefer fewer than 15 files unless evidence requires more.</blockquote><p>The word <strong>prefer</strong> matters. Artificially limiting the context to 15 files could be dangerous if the change genuinely spans 25 important modules. The compiler should optimize for minimum sufficient context rather than minimum context at all costs.</p><p>A good principle is:</p><pre>Start small.<br>Expand only when evidence shows another dependency matters.</pre><p>This keeps context focused without hiding necessary complexity.</p><h4>The bigger idea: context engineering</h4><p>Context Pack Compiler is really an example of a broader concept:</p><p><strong>context engineering</strong>.</p><p>Prompt engineering asks:</p><blockquote>What instructions should I give the model?</blockquote><p>Context engineering asks:</p><blockquote>What information should the model have available while performing the task?</blockquote><p>For coding agents, that distinction matters. A brilliant prompt cannot compensate for missing architectural constraints. And a huge context window cannot guarantee that the important constraint won’t be buried under thousands of irrelevant lines.</p><p>The quality of the selected context can matter as much as the quality of the model consuming it.</p><h4>Key takeaway</h4><p>A Context Pack Compiler doesn’t magically avoid searching a large repository. It does something more practical.</p><p>It converts broad repository exploration into a <strong>small, explicit, reusable model of the current engineering task</strong>.</p><p>Instead of:</p><pre>repository → coding agent</pre><p>you get:</p><pre>repository<br>    ↓<br>search + dependency discovery<br>    ↓<br>relevance filtering<br>    ↓<br>task-specific context<br>    ↓<br>coding agent</pre><p>For a single autonomous coding agent, this might simply become part of its normal workflow.</p><p>For systems involving multiple agents, reviews, repeated attempts, expensive models, or handoffs, a reusable task context package becomes considerably more valuable.</p><p>The underlying principle is simple:</p><p><strong>Search broadly. Understand relationships. Load selectively.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*SAZPFUXfs99Ktu1Z.png" /></figure><p>This story is published under the <a href="https://generativeai.pub/">Generative AI</a> publication. Connect with us on <a href="https://www.linkedin.com/company/generative-ai-publication">LinkedIn</a> and follow <a href="https://www.zeniteq.com/">Zeniteq</a> to stay in the loop with the latest AI stories. Let’s shape the future of AI together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*0jZ-qz6FN6VftOLc.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e78112dceed" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/context-pack-compiler-give-coding-agents-less-context-not-more-0e78112dceed">Context Pack Compiler: Give Coding Agents Less Context, Not More</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Claude Code vs Cursor: How I Actually Use Them]]></title>
            <link>https://blog.stackademic.com/claude-code-vs-cursor-how-i-actually-use-them-b302aa869ba8?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/b302aa869ba8</guid>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Wed, 09 Sep 2026 00:50:19 GMT</pubDate>
            <atom:updated>2026-09-09T00:50:19.377Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ql9JMUXTfbxyeolkZHmDIQ.png" /></figure><p>I stopped comparing Claude Code and Cursor once I realized I reach for them at completely different moments.</p><p>The Claude Code vs Cursor debate is usually framed as a choice:</p><p>I think that is the wrong question.</p><p>As of September 2026, both tools can do far more than their original labels suggest. Cursor is no longer just an AI-powered code editor. It has local agents, cloud agents, parallel workflows, browser integration, and increasingly autonomous development capabilities. Claude Code is no longer simply “Claude in a terminal” either.</p><p>So I don’t choose between them based on a feature checklist.</p><p>I choose based on <strong>how I want to work on the problem</strong>.</p><p>My basic split is:</p><p><strong>Cursor when I want to stay close to the code.</strong></p><p><strong>Claude Code when I want to delegate a problem to an agent that can operate through the repo and terminal.</strong></p><p>That distinction sounds small, but in practice it changes everything.</p><h3>Cursor is where I stay in the development loop</h3><p>Cursor makes the most sense to me when I am actively shaping something.</p><p>I want to see the files.</p><p>I want to inspect the diff.</p><p>I want to jump between the implementation and the surrounding code.</p><p>I may accept an AI-generated change, modify some of it manually, ask another question on a code snippet, rename something myself, and then send the agent in a different direction.</p><p>That is an interactive development loop.</p><p>Cursor fits naturally into it because the editor itself remains the center of the experience.</p><p>This is especially useful when the task on hand is</p><ul><li>frontend components;</li><li>API handlers;</li><li>small-to-medium feature changes;</li><li>unfamiliar code I need to explore;</li><li>refactors where I want to inspect each step;</li><li>debugging where I am moving between several files;</li><li>UI work where visual feedback matters.</li></ul><p>Cursor has also moved well beyond this local-editor workflow. Its newer agent architecture supports multiple agents, handoffs between local and cloud environments, and long-running Cloud Agents that can build, test, and create pull requests remotely.</p><p>That makes the boundary less clean than it used to be.</p><p>But my mental model remains the same:</p><p><strong>Cursor starts from my coding workspace and expands outward.</strong></p><p>I still feel like the developer operating the environment, with agents working around me.</p><h3>Claude Code is where I delegate the problem</h3><p>Claude Code feels different.</p><p>Instead of thinking:</p><blockquote>Which lines should I change?</blockquote><p>I can start with:</p><blockquote>Here is the problem. Investigate the repository, figure out the cause, implement the fix, run the tests, and tell me what changed.</blockquote><p>Claude Code is designed around that agent loop. The target state feels moving more towards autonomous agents from future.</p><p>Anthropic describes it as an agent that can explore codebases, edit code, write and execute tests, use command-line tools, and work with Git. Its CLI can also be called non-interactively, piped data, resumed across sessions, constrained by tool permissions, and return structured output for automation.</p><p>That makes it particularly attractive when the task naturally belongs in the terminal.</p><p>For example:</p><ul><li>tracing a bug across a large repository;</li><li>running tests repeatedly while fixing failures;</li><li>performing a repository-wide refactor;</li><li>investigating logs;</li><li>resolving build errors;</li><li>cleaning up lint or type errors;</li><li>searching Git history;</li><li>updating several connected files;</li><li>writing scripts;</li><li>automating repetitive engineering work.</li></ul><p>I’m less interested in watching every edit happen.</p><p>I’m more interested in defining the objective/goal clearly, giving the agent enough context, and checking whether the final result is correct.</p><p>That is a different style of AI-assisted programming.</p><h3>The difference is not “IDE vs CLI”</h3><p>This distinction is important because the obvious comparison is becoming outdated.</p><p>It is tempting to say:</p><p><strong>Cursor = GUI</strong></p><p><strong>Claude Code = terminal</strong></p><p>Technically, that is increasingly incomplete.</p><p>Claude Code can operate across CLI, IDE, web, and mobile workflows, and Anthropic has added things like skills, hooks, subagents, MCP connectors, and project instructions for controlling how agents work.</p><p>Cursor, meanwhile, is pushing further into autonomous development. Its Cloud Agents can run remotely in isolated development environments, execute tasks over long periods, test changes, and open pull requests. Cursor also supports agent automations and event-driven workflows.</p><p>Cursor comes with nice CLI as well.</p><p>So both products are moving toward agents that can do real autonomous engineering work(atleast for some workflows).</p><p>The more useful distinction is <strong>interaction style</strong>.</p><p>Cursor is optimized around an environment where code, agents, diffs, browsing, and manual editing sit close together.</p><p>Claude Code is optimized around giving an agent access to the same primitives engineers already use from the terminal.</p><p>That affects how I prompt them.</p><h3>My Cursor prompts tend to be narrower</h3><p>With Cursor, I often want something like:</p><blockquote><em>Refactor this component so the loading, empty, and error states use the same pattern.</em></blockquote><p>Or:</p><blockquote><em>Trace how this value gets from the API response into this component.</em></blockquote><p>Or:</p><blockquote><em>Add this field to the form and follow the validation pattern used by the existing fields.</em></blockquote><p>The task has boundaries.</p><p>I’m already looking at the relevant part of the system, and the AI helps me move through it faster.</p><p>I might immediately inspect what it produced and continue editing.</p><p>The human and agent are working almost side by side.</p><h3>My Claude Code prompts tend to describe outcomes</h3><p>With Claude Code, I prefer to give more responsibility to the agent.</p><p>Something like:</p><blockquote>The integration tests for the billing service are failing after the latest schema change. Investigate the failure, identify the root cause, fix it without changing the intended API behavior, run the relevant tests, and summarize the changes.</blockquote><p>Notice the difference.</p><p>I am not telling it which file to edit.</p><p>Finding the file is part of the task.</p><p>Determining the cause is part of the task.</p><p>Choosing the implementation is part of the task.</p><p>Testing the result is part of the task.</p><p>That is where coding agents start becoming much more interesting than autocomplete.</p><h3>Claude Code becomes especially powerful when the terminal is part of the problem</h3><p>One reason I like the Claude Code model is that software engineering is not just editing source files.</p><p>Real development involves:</p><pre>git<br>grep<br>tests<br>linters<br>build tools<br>package managers<br>databases<br>logs<br>shell scripts<br>deployment tools<br>internal CLIs</pre><p>A coding agent that can operate those tools can reason about the <em>process</em> around the code, not just the code itself.</p><p>Claude Code can also connect to external systems through MCP, an open protocol for giving AI applications access to tools and data sources. That can extend a coding task beyond the local repository — for example, into issue trackers, monitoring systems, databases, documentation, or design tools when the appropriate integrations exist.</p><p>That is useful because many real engineering tasks start somewhere outside the code.</p><p>The ticket explains the requirement.</p><p>The logs reveal the failure.</p><p>The documentation explains the API.</p><p>The repository contains the implementation.</p><p>The tests tell you whether the fix worked.</p><p>The more of that loop the agent can traverse, the more meaningful the delegation becomes.</p><h3>But Cursor is getting much better at delegation too</h3><p>This is where the comparison gets interesting.</p><p>If Cursor had remained an AI-enhanced editor, the decision would be easy.</p><p>It hasn’t.</p><p>Cursor’s Cloud Agents can move work into remote environments where agents plan, edit code, execute commands, run tests, and produce pull requests. Cursor says these agents can continue working even after you close your laptop.</p><p>Its 2026 releases have pushed further toward persistent and event-driven agents. Cloud Agents can respond to events, follow work through pull-request feedback and CI, and participate in scheduled or subscribed workflows.</p><p>So there are now tasks I could reasonably give to either product.</p><p>At that point, I stop asking:</p><p><strong>Which one can do this?</strong></p><p>Usually, both probably can.</p><p>Instead I ask:</p><p><strong>Where do I want this work to live?</strong></p><h3>My practical decision rule</h3><p>I use a very simple heuristic.</p><h3>Use Cursor when I want to participate continuously</h3><p>If I expect to read, edit, redirect, inspect, and make decisions every few minutes, I want the editor experience.</p><p>That usually means Cursor.</p><h3>Use Claude Code when I can describe a clear objective</h3><p>If I can define the desired outcome and verification criteria, I’m more comfortable handing over more of the loop.</p><p>That usually means Claude Code.</p><h3>Use Cursor Cloud Agents when the task should run away from my machine</h3><p>For longer jobs where a remote environment and pull-request-based handoff make sense, Cursor’s cloud workflow becomes much more attractive.</p><h3>Use Claude Code when I want terminal-native automation</h3><p>Claude Code’s non-interactive CLI mode is especially interesting here.</p><p>Because it can accept prompts programmatically and produce structured output, it can participate in scripts and automation rather than requiring every interaction to happen manually inside an editor.</p><p>That moves it from “coding assistant” toward “engineering primitive.”</p><h3>I don’t obsess over which model wins</h3><p>Another reason I dont feel declaring a permanent winner is that model rankings age badly.</p><p>The underlying models change.</p><p>Agent harnesses improve.</p><p>Context handling changes.</p><p>Tool use improves.</p><p>New models arrive.</p><p>The product that performs better on one task this month may behave differently six months later.</p><p>So I care more about the workflow abstraction than a temporary benchmark result.</p><p>The useful question is not:</p><p><strong>Which AI writes better code?</strong></p><p>It is:</p><p><strong>How much responsibility can I safely delegate for this particular task, and what environment makes reviewing that work easiest?</strong></p><p>That question survives model upgrades.</p><h3>The biggest mistake is giving either tool vague work</h3><p>Neither Cursor nor Claude Code removes the need to think clearly.</p><p>“Fix the backend” is still a terrible engineering specification.</p><p>The more autonomy I give an agent, the more useful it becomes to define:</p><ul><li>what is broken;</li><li>what behavior should remain unchanged;</li><li>what constraints matter;</li><li>what files or systems may be relevant;</li><li>what counts as completion;</li><li>which tests should pass;</li><li>what the agent should not change.</li></ul><p>The better the definition of done, the easier it is to delegate.</p><p>This becomes increasingly important as coding agents work for longer periods without continuous human intervention. Anthropic’s research, for example, has documented Claude Code sessions operating autonomously for progressively longer stretches, while Cursor is explicitly building agents for long-running remote development.</p><p>More autonomy makes verification more important, not less.</p><h3>So which one should you learn?</h3><p>If you are new to AI coding tools, I would not try to master everything at once.</p><p>Start with Cursor if your priority is improving the coding workflow you already have.</p><p>It gives you a relatively natural path from:</p><p><strong>I write the code</strong></p><p>to:</p><p><strong>AI helps me write the code</strong></p><p>to:</p><p><strong>AI handles larger pieces while I supervise.</strong></p><p>Then learn a terminal-based agent like Claude Code or Cursor CLI.</p><p>That introduces the bigger conceptual shift:</p><p><strong>AI is not just generating code for me. It can operate parts of the engineering workflow for me.</strong></p><p>Understanding that transition matters more than memorizing another collection of shortcuts.</p><h3>The takeaway</h3><p>I don’t see Claude Code and Cursor as mutually exclusive.</p><p>They represent two interfaces to the same larger change in software development: we are moving from asking AI to produce snippets toward giving agents objectives and letting them perform increasingly complete engineering tasks.</p><p>Cursor is where I want the agent embedded in my development environment.</p><p>Claude Code is where I want the agent operating through the repo, terminal, and tools around the code.</p><p>And as both products become more autonomous, even that line will continue to blur.</p><p>So my rule is simple:</p><p><strong>If I want to code with the agent, I reach for Cursor.</strong></p><p><strong>If I want to hand work to the agent, I reach for Claude Code.</strong></p><p>The interesting part is that every few months, both tools get better at doing the other one’s job.</p><h3>Before you go</h3><ul><li>Please take a moment to like the post and follow the writer!</li><li>Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute <a href="https://plainenglish.io/write-for-us"><strong>here</strong></a>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b302aa869ba8" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/claude-code-vs-cursor-how-i-actually-use-them-b302aa869ba8">Claude Code vs Cursor: How I Actually Use Them</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Step-by-Step Learning Path for Artificial Intelligence]]></title>
            <link>https://ai.plainenglish.io/a-step-by-step-learning-path-for-artificial-intelligence-1876ac7daea4?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/1876ac7daea4</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Tue, 08 Sep 2026 10:23:24 GMT</pubDate>
            <atom:updated>2026-09-08T10:23:24.041Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7iv8wUFB4NLpb57BpenVPg.jpeg" /></figure><p>Artificial Intelligence can feel overwhelming because the field contains mathematics, machine learning, deep learning, large language models, agents, MLOps, reinforcement learning, and many other areas.</p><p>I believe one practical approach is not to study everything at once, but to build your knowledge layer by layer.</p><p>This is my personal AI learning path, a roadmap I’m following and still learning through. If you spot an important topic I’ve missed, feel free to share your feedback; I’d be glad to keep improving it.</p><h3>The Complete AI Learning Path</h3><p>A high-level sequence could be:</p><pre>Programming<br>↓<br>Mathematics<br>↓<br>Data Handling<br>↓<br>Classical Machine Learning<br>↓<br>Deep Learning<br>↓<br>NLP and Transformers<br>↓<br>Large Language Models<br>↓<br>Embeddings, Search, and RAG<br>↓<br>AI Agents<br>↓<br>Evaluation and Safety<br>↓<br>MLOps and Production AI<br>↓<br>Specialization<br>↓<br>Research</pre><p>A memorable way to connect the entire roadmap is to think of <strong>AI as building and running a smart factory</strong>.</p><blockquote><strong><em>Programming</em></strong><em> builds the factory, </em><strong><em>mathematics</em></strong><em> explains how the machinery works, and data is the raw material. </em><strong><em>Machine learning</em></strong><em> teaches the machines to recognize patterns, while </em><strong><em>deep learning</em></strong><em> gives them a more powerful brain. </em><strong><em>Transformers</em></strong><em> teach that brain how to understand relationships in sequences, and </em><strong><em>LLMs</em></strong><em> turn it into a language-capable intelligence. </em><strong><em>Embeddings</em></strong><em> and </em><strong><em>RAG</em></strong><em> give it a searchable library and external memory. </em><strong><em>Agents</em></strong><em> give it hands, tools, and the ability to take actions. </em><strong><em>MLOps</em></strong><em> keeps the factory running reliably in the real world, while evaluation and safety act as quality control. Specialization and research are the R&amp;D department that invents what comes next.</em></blockquote><p>A shorter version to remember is:</p><pre>Programming = Build<br>Mathematics = Understand<br>Data = Fuel<br>Machine Learning = Learn<br>Deep Learning = Brain<br>Transformers = Attention<br>LLMs = Language<br>RAG = Memory<br>Agents = Action<br>MLOps = Operate<br>Evaluation &amp; Safety = Verify<br>Research = Discover</pre><p>Or even more compactly:</p><p><strong>Build → Understand → Feed → Learn → Think → Remember → Act → Operate → Verify → Discover</strong></p><p>Now let’s go through each stage.</p><h3>Phase 1: Programming Foundations</h3><p>Before diving into AI, build a solid foundation in programming, preferably with Python.</p><h3>1. Python</h3><p>Learn:</p><ul><li>Variables</li><li>Data types</li><li>Conditions</li><li>Loops</li><li>Functions</li><li>Lists</li><li>Dictionaries</li><li>Sets</li><li>Classes</li><li>File handling</li><li>Exceptions</li><li>Virtual environments</li><li>Type hints</li></ul><p>You do not need to become a Python expert before continuing. You should simply be comfortable writing/understanding small programs without constantly fighting the language.</p><h3>2. Scientific Python</h3><p>Learn the most common libraries used in AI:</p><ul><li>NumPy</li><li>Pandas</li><li>Matplotlib</li><li>Jupyter Notebook</li></ul><p>NumPy is especially important because vectors, matrices, tensors, and numerical operations appear throughout machine learning.</p><h3>3. Software Engineering Basics</h3><p>Learn:</p><ul><li>Git</li><li>GitHub</li><li>Unit testing</li><li>pytest</li><li>REST APIs</li><li>JSON</li><li>Docker</li><li>Basic Linux commands</li><li>Clean code</li><li>Modular application design</li></ul><p>These skills become extremely important when you move from AI experiments to production AI applications.</p><p><strong>Suggested project:</strong></p><p>Take a public dataset, analyze it with Pandas, visualize the results, and expose a few results through a small REST API.</p><h3>Phase 2: Mathematics for AI</h3><p>You do not need to finish an entire university mathematics curriculum before starting machine learning.</p><p>Learn mathematics alongside practical AI development.</p><h3>4. Linear Algebra</h3><p>Learn:</p><ul><li>Scalars</li><li>Vectors</li><li>Matrices</li><li>Dot products</li><li>Matrix multiplication</li><li>Linear transformations</li><li>Eigenvalues</li><li>Eigenvectors</li><li>Singular Value Decomposition</li></ul><p>Linear algebra is the language used to represent data and neural-network computations.</p><h3>5. Probability and Statistics</h3><p>Learn:</p><ul><li>Random variables</li><li>Probability distributions</li><li>Conditional probability</li><li>Bayes’ theorem</li><li>Mean</li><li>Variance</li><li>Standard deviation</li><li>Covariance</li><li>Correlation</li><li>Hypothesis testing</li><li>Maximum likelihood estimation</li></ul><p>Probability helps us reason about uncertainty, while statistics helps us learn patterns from data.</p><h3>6. Calculus and Optimization</h3><p>Learn:</p><ul><li>Functions</li><li>Derivatives</li><li>Partial derivatives</li><li>Gradients</li><li>Chain rule</li><li>Gradient descent</li><li>Convex optimization</li><li>Non-convex optimization</li></ul><p>The chain rule and gradients become especially important when you learn backpropagation.</p><p><strong>Suggested project:</strong></p><p>Implement linear regression and gradient descent using NumPy without using a machine learning library.</p><h3>Phase 3: Data Preparation</h3><p>Machine learning models are only as useful as the data used to train them.</p><h3>7. Data Cleaning</h3><p>Learn how to handle:</p><ul><li>Missing values</li><li>Duplicate records</li><li>Incorrect values</li><li>Outliers</li><li>Class imbalance</li><li>Data leakage</li><li>Poor label quality</li></ul><h3>8. Exploratory Data Analysis</h3><p>Learn how to inspect:</p><ul><li>Distributions</li><li>Correlations</li><li>Feature relationships</li><li>Patterns</li><li>Anomalies</li><li>Possible bias in datasets</li></ul><h3>9. Feature Engineering</h3><p>Learn:</p><ul><li>Categorical encoding</li><li>Feature scaling</li><li>Normalization</li><li>Feature selection</li><li>Dimensionality reduction</li><li>Train-validation-test splits</li></ul><p><strong>Suggested project:</strong></p><p>Create a reusable preprocessing pipeline for a real-world dataset.</p><h3>Phase 4: Classical Machine Learning</h3><p>Classical machine learning teaches many of the core principles that remain important even when working with modern LLMs.</p><h3>10. Supervised Learning</h3><p>Study these algorithms:</p><ul><li>Linear regression</li><li>Logistic regression</li><li>Decision trees</li><li>Random forests</li><li>Gradient boosting</li><li>Support Vector Machines</li><li>K-Nearest Neighbours</li><li>Naive Bayes</li></ul><h3>11. Unsupervised Learning</h3><p>Study:</p><ul><li>K-Means clustering</li><li>Hierarchical clustering</li><li>DBSCAN</li><li>Principal Component Analysis</li><li>Anomaly detection</li></ul><h3>12. Model Evaluation</h3><p>Learn:</p><ul><li>Accuracy</li><li>Precision</li><li>Recall</li><li>F1 score</li><li>Confusion matrices</li><li>ROC-AUC</li><li>PR-AUC</li><li>Regression metrics</li><li>Cross-validation</li><li>Calibration</li><li>Bias-variance trade-off</li><li>Overfitting</li><li>Underfitting</li><li>Regularization</li></ul><h3>13. Machine Learning Pipelines</h3><p>Learn:</p><ul><li>Scikit-learn</li><li>Preprocessing pipelines</li><li>Hyperparameter tuning</li><li>Experiment tracking</li><li>Model reproducibility</li></ul><p><strong>Suggested project:</strong></p><p>Build an end-to-end machine learning system for customer churn, fraud detection, house-price prediction, or another real-world prediction problem.</p><h3>Phase 5: Deep Learning</h3><p>Once classical machine learning makes sense, move into neural networks.</p><h3>14. Neural Network Fundamentals</h3><p>Learn these concepts carefully:</p><ul><li>Perceptrons</li><li>Weights</li><li>Biases</li><li>Activation functions</li><li>Forward propagation</li><li>Loss functions</li><li>Backpropagation</li><li>Gradient descent</li><li>Optimizers</li><li>Weight initialization</li><li>Batch normalization</li><li>Dropout</li><li>Sigmoid</li><li>ReLU</li><li>Softmax</li></ul><p>Backpropagation is one of the most important concepts to understand in deep learning.</p><h3>15. Learn PyTorch</h3><p>Study:</p><ul><li>Tensors</li><li>Automatic differentiation</li><li>Neural-network modules</li><li>Datasets</li><li>Data loaders</li><li>Training loops</li><li>GPU training</li><li>Model checkpoints</li></ul><h3>16. Major Neural Network Architectures</h3><p>Study:</p><ul><li>Multilayer Perceptrons</li><li>Convolutional Neural Networks</li><li>Recurrent Neural Networks</li><li>LSTMs</li><li>GRUs</li><li>Attention</li><li>Transformers</li></ul><p><strong>Suggested project:</strong></p><p>Implement a tiny neural network using NumPy first. Then rebuild the same network using PyTorch.</p><h3>Phase 6: Natural Language Processing</h3><h3>17. NLP Fundamentals</h3><p>Learn:</p><ul><li>Tokenization</li><li>Stemming</li><li>Lemmatization</li><li>Bag-of-Words</li><li>TF-IDF</li><li>Word embeddings</li><li>Text classification</li><li>Sequence modelling</li></ul><h3>18. Transformers</h3><p>Study these concepts deeply:</p><ul><li>Attention</li><li>Self-attention</li><li>Queries, keys, and values</li><li>Multi-head attention</li><li>Positional encoding</li><li>Encoder architectures</li><li>Decoder architectures</li><li>BERT-style models</li><li>GPT-style models</li><li>Pretraining</li><li>Fine-tuning</li></ul><h3>19. Modern NLP Tooling</h3><p>Learn:</p><ul><li>Hugging Face Transformers</li><li>Hugging Face Datasets</li><li>Tokenizers</li><li>Sentence Transformers</li><li>Embedding models</li></ul><p><strong>Suggested project:</strong></p><p>Fine-tune a transformer for sentiment analysis, text classification, or named-entity recognition.</p><h3>Phase 7: Large Language Models</h3><p>At this stage, start learning how modern language models work and how applications are built around them.</p><h3>20. LLM Fundamentals</h3><p>Learn:</p><ul><li>Pretraining</li><li>Next-token prediction</li><li>Instruction tuning</li><li>Supervised fine-tuning</li><li>Preference optimization</li><li>Context windows</li><li>Tokens</li><li>Sampling</li><li>Temperature</li><li>Hallucinations</li><li>LLM evaluation</li></ul><h3>21. Prompt Engineering</h3><p>Study:</p><ul><li>Clear instructions</li><li>Few-shot examples</li><li>Structured outputs</li><li>Tool-use prompts</li><li>Prompt testing</li><li>Context management</li></ul><blockquote>Prompt engineering is useful, but it should be treated as one engineering technique, not the entirety of AI engineering.</blockquote><h3>Phase 8: Embeddings, Search, and RAG</h3><h3>22. Embeddings</h3><p>Learn:</p><ul><li>Vector representations</li><li>Embedding models</li><li>Cosine similarity</li><li>Semantic similarity</li><li>Vector search</li></ul><h3>23. Retrieval Systems</h3><p>Study:</p><ul><li>Chunking</li><li>Vector databases</li><li>Metadata filtering</li><li>Keyword search</li><li>Semantic search</li><li>Hybrid search</li><li>Reranking</li></ul><h3>24. Retrieval-Augmented Generation</h3><p>Learn the complete RAG pipeline:</p><pre>Documents<br>↓<br>Parsing<br>↓<br>Chunking<br>↓<br>Embeddings<br>↓<br>Vector Database<br>↓<br>User Query<br>↓<br>Retrieval<br>↓<br>Reranking<br>↓<br>Context<br>↓<br>LLM<br>↓<br>Answer</pre><p>Study:</p><ul><li>Document ingestion</li><li>Chunking strategies</li><li>Query rewriting</li><li>Metadata filtering</li><li>Retrieval</li><li>Reranking</li><li>Citation generation</li><li>Retrieval evaluation</li><li>Answer evaluation</li></ul><p><strong>Suggested project:</strong></p><p>Build a question-answering system over your own documents that returns citations for every important answer.</p><h3>Phase 9: Model Adaptation</h3><h3>25. Fine-Tuning and Optimization</h3><p>Learn:</p><ul><li>Supervised fine-tuning</li><li>LoRA</li><li>Parameter-efficient fine-tuning</li><li>Quantization</li><li>Distillation</li><li>Synthetic training data</li></ul><p>Do not automatically fine-tune whenever an LLM produces a poor answer.</p><p>First determine whether the problem is caused by prompting, retrieval, missing context, tool design, or evaluation.</p><h3>Phase 10: AI Agents</h3><p>Agents combine language models with tools, state, control flow, and external systems.</p><h3>26. Agent Fundamentals</h3><p>Learn:</p><ul><li>Models</li><li>Tools</li><li>Instructions</li><li>State</li><li>Function calling</li><li>Structured outputs</li><li>Planning</li><li>Memory</li><li>Human approval</li></ul><h3>27. Agent Workflows</h3><p>Study:</p><ul><li>Routing</li><li>Prompt chaining</li><li>Parallel execution</li><li>Evaluator-optimizer loops</li><li>State machines</li><li>Multi-agent systems</li><li>Event-driven workflows</li></ul><h3>28. Agent Engineering</h3><p>Learn:</p><ul><li>Tool design</li><li>Context engineering</li><li>Harness engineering</li><li>State management</li><li>Retries</li><li>Timeouts</li><li>Idempotency</li><li>Observability</li><li>Cost management</li><li>Latency management</li></ul><p>One important lesson is that reliable agents should not depend entirely on the LLM remembering every operational step.</p><p>Critical steps should often be enforced by deterministic workflows, validation, state machines, tests, and explicit execution policies.</p><h3>29. Agent Safety</h3><p>Study:</p><ul><li>Prompt injection</li><li>Data leakage</li><li>Tool permissions</li><li>Sandboxing</li><li>Human approval boundaries</li><li>Audit logs</li><li>Input validation</li><li>Output validation</li></ul><p><strong>Suggested project:</strong></p><p>Build a coding, support, or research agent that uses multiple tools and requires human approval before sensitive operations.</p><h3>Phase 11: MLOps and Production AI</h3><p>Building a model is only part of building an AI system.</p><h3>30. Model Serving</h3><p>Learn:</p><ul><li>FastAPI</li><li>Batch inference</li><li>Online inference</li><li>Model serialization</li><li>GPU serving</li><li>Caching</li><li>Streaming responses</li></ul><h3>31. MLOps</h3><p>Study:</p><ul><li>Data versioning</li><li>Model versioning</li><li>Experiment tracking</li><li>Model registries</li><li>CI/CD</li><li>Training pipelines</li><li>Feature stores</li></ul><h3>32. Monitoring</h3><p>Learn how to monitor:</p><ul><li>Model quality</li><li>Data drift</li><li>Concept drift</li><li>Latency</li><li>Failures</li><li>Token usage</li><li>Cost</li><li>User feedback</li></ul><h3>33. AI System Design</h3><p>Learn:</p><ul><li>Scalability</li><li>Reliability</li><li>Queue-based processing</li><li>Distributed inference</li><li>Privacy</li><li>Security</li><li>Cost-performance trade-offs</li></ul><p><strong>Suggested project:</strong></p><p>Deploy an AI service with automated tests, monitoring, model versioning, logging, and a rollback mechanism.</p><h3>Phase 12: AI Evaluation and Responsible AI</h3><h3>34. AI Evaluation</h3><p>Study:</p><ul><li>Golden datasets</li><li>Offline evaluation</li><li>Online evaluation</li><li>Human evaluation</li><li>LLM-as-judge evaluation</li><li>Regression testing</li><li>A/B testing</li><li>Error analysis</li></ul><p>Evaluation is one of the most important skills in modern AI engineering.</p><p>Without good evaluation, improving an AI application becomes guesswork.</p><h3>35. Responsible AI</h3><p>Learn:</p><ul><li>Fairness</li><li>Explainability</li><li>Privacy</li><li>Transparency</li><li>Robustness</li><li>Safety</li><li>Governance</li><li>Copyright</li><li>Data provenance</li></ul><h3>36. Adversarial Testing</h3><p>Study:</p><ul><li>Prompt-injection testing</li><li>Jailbreak testing</li><li>Red teaming</li><li>Distribution shifts</li><li>Tool misuse</li><li>Unexpected model behavior</li></ul><p><strong>Suggested project:</strong></p><p>Build an automated evaluation suite for your RAG system or AI agent.</p><h3>Phase 13: Choose a Specialization</h3><p>At this point, you should have enough background to choose one or two areas to study deeply.</p><h3>Generative AI</h3><ul><li>Diffusion models</li><li>Variational Autoencoders</li><li>Generative Adversarial Networks</li><li>Multimodal models</li><li>Image generation</li><li>Audio generation</li><li>Video generation</li></ul><h3>Computer Vision</h3><ul><li>Image classification</li><li>Object detection</li><li>Image segmentation</li><li>Vision Transformers</li><li>Vision-language models</li></ul><h3>Reinforcement Learning</h3><ul><li>Markov Decision Processes</li><li>Q-Learning</li><li>Policy gradients</li><li>Actor-critic methods</li><li>Offline reinforcement learning</li><li>Reinforcement learning from feedback</li></ul><h3>AI Systems</h3><ul><li>Distributed training</li><li>Mixed-precision training</li><li>Model parallelism</li><li>Data parallelism</li><li>Model compression</li><li>Inference optimization</li><li>GPU architecture</li></ul><h3>AI Research</h3><ul><li>Statistical learning theory</li><li>Information theory</li><li>Optimization theory</li><li>Causal inference</li><li>Probabilistic graphical models</li><li>Representation learning</li><li>Interpretability</li><li>AI alignment</li><li>AI safety</li></ul><h3>Before you go</h3><ul><li>Please take a moment to like the post and follow the writer!</li><li>Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute <a href="https://plainenglish.io/write-for-us">here</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1876ac7daea4" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/a-step-by-step-learning-path-for-artificial-intelligence-1876ac7daea4">A Step-by-Step Learning Path for Artificial Intelligence</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agent Compute Broker: Stop Using Your Best AI Model for Every Task]]></title>
            <link>https://generativeai.pub/agent-compute-broker-stop-using-your-best-ai-model-for-every-task-f511edc445c9?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f511edc445c9</guid>
            <category><![CDATA[computing]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[ai-model]]></category>
            <category><![CDATA[techonology]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Sun, 06 Sep 2026 09:47:10 GMT</pubDate>
            <atom:updated>2026-09-06T09:47:10.398Z</atom:updated>
            <content:encoded><![CDATA[<h4>The next step for AI coding agents may not be choosing a better model, but using a scheduler that decides when advanced reasoning is worth the cost and when to fall back to cheaper models or deterministic tools.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FGXoULgYF0U1JhJ1yEdF1A.png" /></figure><p>Most sophisticated AI-agent workflows have an inefficient default: once you choose a model, that model tends to handle almost everything.</p><p>The same expensive frontier model that makes a difficult API-design decision may also spend tokens listing files, searching for call sites, summarizing logs, transforming data, and formatting JSON.</p><p>The reverse problem happens too. A cheaper model may receive a genuinely difficult debugging or architecture problem, make several unsuccessful attempts, and only then get escalated to something stronger.</p><p>Both approaches waste compute.</p><p>A better architecture is an <strong>Agent Compute Broker</strong>: a scheduling layer that breaks a workflow into tasks, estimates how much reasoning each task deserves, sends it to the appropriate model or deterministic tool, verifies the result, and escalates only when necessary.</p><p>Instead of asking:</p><blockquote>Which model should run this agent?</blockquote><p>the system asks:</p><blockquote>Which parts of this job actually require expensive intelligence?</blockquote><p>That distinction becomes increasingly useful as agent platforms expose more control over reasoning effort, task budgets, subagents, tools, and context.</p><h4>The problem with one-model-for-everything agents</h4><p>Consider asking an AI coding agent to migrate a 40-module monorepo to a new framework version.</p><p>The job might contain hundreds of individual operations:</p><ul><li>inventory packages;</li><li>identify deprecated dependencies;</li><li>find every caller of a changed API;</li><li>edit configuration files;</li><li>apply repetitive syntax migrations;</li><li>compile the repository;</li><li>cluster compiler errors;</li><li>investigate obscure runtime failures;</li><li>make compatibility decisions</li><li>rerun tests.</li></ul><p>These tasks do not have equal reasoning requirements.</p><p>Finding every reference to a symbol is mostly an information-retrieval problem. If the language server can answer it exactly, asking a frontier model to reason through the repository is both slower and less reliable.</p><p>Changing 70 identical import statements is largely mechanical.</p><p>Deciding whether a framework’s new request lifecycle breaks an internal abstraction is different. That may require understanding documentation, code history, architectural constraints, backward compatibility, and several competing implementation choices.</p><p>Yet a conventional agent often treats all three operations as roughly the same kind of model invocation.</p><p>The result is an expensive form of resource allocation: <strong>premium reasoning is spent where deterministic computation would work, while difficult reasoning is not always recognized early enough.</strong></p><h4>What an Agent Compute Broker does</h4><p>The broker sits between the user’s objective and the agents that execute it.</p><p>Before execution, it converts the plan into a task graph, usually a directed acyclic graph, or DAG, where nodes represent pieces of work and edges represent dependencies.</p><p>For each node, the broker estimates attributes such as:</p><ul><li><strong>reasoning difficulty</strong> — how much judgment or inference is required;</li><li><strong>failure cost</strong> — what happens if the answer is wrong;</li><li><strong>context requirement</strong> — how much repository or conversation state must be understood;</li><li><strong>verification strength</strong> — how easily the result can be checked automatically;</li><li><strong>expected token cost</strong>;</li><li><strong>expected latency</strong>.</li></ul><p>The scheduler then chooses an execution strategy.</p><p>A simplified migration might look like this:</p><pre>repository inventory<br>        ↓<br>inexpensive agent<br>find 87 callers of changed API<br>        ↓<br>language server / programmatic search<br><br>apply mechanical edits<br>        ↓<br>mid-tier coding agent<br>decide compatibility strategy<br>        ↓<br>frontier reasoning agent<br>run build and tests<br>        ↓<br>deterministic tooling<br>classify 23 failures<br>        ↓<br>code + inexpensive model<br>investigate 2 unexplained failures<br>        ↓<br>frontier reasoning agent</pre><p>The important point is that <strong>an LLM does not need to execute every node</strong>.</p><p>Sometimes the best agent is grep. Sometimes it is a compiler. Sometimes it is a short Python program. And sometimes it is the most capable reasoning model available.</p><h4>Why this is becoming a practical need</h4><p>Claude’s current developer stack contains several pieces that make this architecture more feasible.</p><p>The Claude Agent SDK provides the agent loop and built-in tools for operations such as reading files, editing code, running shell commands, searching repositories, connecting to MCP servers, and delegating work to subagents.</p><p>Recent Claude models also expose <strong>effort controls</strong> that let developers trade additional reasoning depth and token consumption against latency and cost. Anthropic explicitly recommends recalibrating effort levels against your own evaluations instead of simply carrying settings over from an older model.</p><p>That is important for a compute broker. Model choice no longer has to be the only scheduling variable.</p><p>A broker could potentially choose:</p><pre>cheap model + low effort<br>mid-tier model + medium effort<br>strong model + normal effort<br>frontier model + very high effort</pre><p>depending on the task.</p><p>Anthropic has also introduced <strong>task budgets</strong> for newer agentic workloads. These let a model know approximately how much token budget is available across an agentic task so it can pace its work accordingly. They are different from max_tokens: task budgets guide the agent, while max_tokens remains a hard generation ceiling. Anthropic currently describes task budgets as a beta capability and recommends using them selectively rather than constraining every open-ended task.</p><p>Together, model tier, effort, and task-budget controls start looking less like ordinary inference parameters and more like resources a scheduler can allocate.</p><h4>Programmatic Tool Calling is the other half of the idea</h4><p>Model routing alone does not solve the efficiency problem.</p><p>Suppose an agent needs to:</p><ol><li>retrieve 500 compiler errors;</li><li>group them by error code;</li><li>count occurrences;</li><li>find affected packages;</li><li>return only the unusual clusters.</li></ol><p>One implementation makes hundreds of tool calls and returns every intermediate result to the model.</p><p>That is expensive because the model repeatedly wakes up, reads results, reasons about them, and decides what to call next.</p><p>But none of those steps requires deep reasoning.</p><p>Anthropic’s Programmatic Tool Calling attacks exactly this problem. Claude can write code that invokes tools, loops over results, filters data, performs aggregation, and controls which information ultimately enters the model’s context. Anthropic notes that conventional multi-tool workflows can otherwise require repeated inference passes while filling the model’s context with intermediate data.</p><p>For a compute broker, this creates three execution categories rather than two:</p><pre>deterministic software<br>        ↓<br>programmatic tool orchestration<br>        ↓<br>model reasoning</pre><p>The broker’s job is to push a task as far down that stack as safely possible.</p><p>If Python can compute it, use Python. If a smaller agent can reliably classify it, use the smaller agent. Reserve expensive reasoning for the remaining uncertainty.</p><h4>Verification makes aggressive routing possible</h4><p>There is an obvious risk. What if the broker underestimates a task?</p><p>A cheap model attempts something difficult, produces a plausible but incorrect patch, and the system silently continues. That is why routing and verification have to be designed together.</p><p>The broker should ask not only:</p><blockquote>Can a cheaper model probably perform this task?</blockquote><p>but also:</p><blockquote>Can I cheaply detect whether it failed?</blockquote><p>Consider two tasks.</p><h4>Task A: mechanical refactoring</h4><p>A cheaper coding agent modifies 40 call sites.</p><p>Verification is strong:</p><pre>compile<br>→ run unit tests<br>→ check static analysis<br>→ verify old API has zero remaining references</pre><p>If all checks pass, there may be little value in paying a frontier model to perform the edit.</p><h4>Task B: choosing an authentication architecture</h4><p>Automated verification is much weaker. Several designs may compile and pass tests while still differing substantially in security, maintainability, or compatibility. The failure cost is also higher. That task deserves stronger reasoning from the beginning.</p><p>This leads to a useful scheduling principle:</p><blockquote>The easier a result is to verify, the more aggressively you can route execution toward cheaper compute.</blockquote><h4>Escalation should be automatic</h4><p>Routing does not need to be perfect on the first attempt. The broker can treat model selection as an escalation ladder.</p><p>For example:</p><pre>mechanical edit<br>    ↓<br>mid-tier agent<br>    ↓<br>tests fail<br>    ↓<br>mid-tier agent retries with diagnostics<br>    ↓<br>tests fail again<br>    ↓<br>frontier debugging agent</pre><p>Other escalation signals might include:</p><ul><li>repeated tool errors;</li><li>contradictory evidence;</li><li>low classifier confidence;</li><li>unusually large code changes;</li><li>failed tests;</li><li>unexpected performance regressions;</li><li>unresolved ambiguity;</li><li>excessive retries;</li><li>human rejection.</li></ul><p>The system could eventually learn that particular task classes almost always escalate.</p><p>If “database migration planning” reaches the frontier model 92% of the time, there is little reason to route it cheaply first.</p><p>If “rename generated DTO fields” succeeds with the inexpensive tier 99.8% of the time, the broker can confidently keep it there.</p><p>Routing becomes an empirical optimization problem rather than a collection of hand-written assumptions.</p><h4>The broker can learn from its own workload</h4><p>The first version does not require sophisticated machine learning.</p><p>A simple classifier might use:</p><pre>task_type<br>files_touched<br>estimated_context_size<br>presence_of_test_oracle<br>historical_success_rate<br>previous_attempt_count<br>expected_failure_cost</pre><p>and produce:</p><pre>route = cheap | standard | frontier | deterministic</pre><p>Over time, execution telemetry provides training data.</p><p>For every task node, record:</p><ul><li>selected model;</li><li>effort level;</li><li>input and output tokens;</li><li>latency;</li><li>tool-call count;</li><li>retries;</li><li>verification outcome;</li><li>escalation;</li><li>human intervention;</li><li>final success.</li></ul><p>Now the broker can answer useful questions.</p><ul><li>Which tasks are consistently over-provisioned?</li><li>Which tasks fail when routed cheaply?</li><li>Does higher reasoning effort actually improve success for debugging tasks?</li><li>At what point does a retry cost more than immediate escalation?</li><li>Which verification methods allow safe use of smaller models?</li></ul><p>The optimization objective does not have to be “minimize tokens.”</p><p>It could be:</p><pre>minimize cost<br>subject to ≥ 95% success rate</pre><p>or:</p><pre>minimize elapsed time<br>subject to a fixed compute budget</pre><p>or even:</p><pre>maximize expected task success<br>subject to $5 total cost<br>and &lt; 10 minute latency</pre><p>That is why “broker” is a more useful mental model than “router.” A router chooses a model. A broker allocates a constrained resource.</p><h4>A realistic prototype</h4><p>A first implementation could deliberately stay small.</p><p>Support four task classes:</p><ol><li>search;</li><li>mechanical edit;</li><li>debugging;</li><li>architecture.</li></ol><p>Use only two model tiers. Add deterministic tools for search, builds, and tests.</p><p>Then define a simple policy:</p><pre>search<br>→ tools or cheap model</pre><pre>mechanical edit<br>→ cheap model<br>→ escalate after verification failure</pre><pre>debugging<br>→ cheap model if strong tests exist<br>→ frontier after two failed attempts</pre><pre>architecture<br>→ frontier model immediately</pre><p>Next, evaluate it against a baseline:</p><blockquote><strong>Baseline:</strong> run every task with the strongest model.</blockquote><p>Use 30 historical engineering tasks and measure:</p><ul><li>final task success;</li><li>total token cost;</li><li>wall-clock time;</li><li>number of model calls;</li><li>number of retries;</li><li>number of escalations;</li><li>human interventions.</li></ul><p>A compelling result does not require dramatic intelligence gains. If the broker retains roughly 95% of the strongest-model baseline’s task success while substantially reducing compute spend, that is already valuable. Then add more sophistication only where the data justifies it.</p><h4>The biggest challenge is classification, not orchestration</h4><p>Building the scheduler is relatively straightforward. Knowing when <em>not</em> to economize is harder.</p><p>A five-line change can hide a difficult semantic decision. A 300-file refactor can be almost completely mechanical. Task size is therefore a poor proxy for reasoning difficulty.</p><p>The broker needs signals about uncertainty, reversibility, and verification. It also needs to avoid spending more compute deciding where to spend compute than the original task would have consumed. The classifier itself should be cheap.</p><p>For obvious cases, rules may be better than models:</p><pre>symbol-reference search → LSP<br>test execution → shell<br>JSON transformation → code</pre><p>Only ambiguous routing decisions need model-based classification.</p><h4>Model selectors are for users. Compute schedulers are for systems.</h4><p>Most AI coding tools expose model choice as a user preference.</p><p>Pick the fast model. Pick the smart model. Pick the expensive one when the task looks difficult. That works when a “task” means one chat request. It becomes less sensible when one request expands into 300 heterogeneous operations.</p><p>The human does not know in advance which four nodes of that workflow will require exceptional reasoning. The agent discovers that during execution. So the natural place to make the decision is inside the runtime.</p><p>The engineer should be able to specify something closer to:</p><pre>Quality target: high<br>Maximum budget: $8<br>Latency preference: balanced</pre><p>The broker should decide how to spend that budget. That turns model intelligence into something resembling compute infrastructure.</p><p>We already schedule CPUs, GPUs, memory, database queries, and distributed jobs according to workload characteristics. Agentic systems may need the same abstraction for reasoning.</p><p>The important question may no longer be:</p><blockquote>Which model should power this agent?</blockquote><p>It may become:</p><blockquote>Where, inside this workflow, is expensive intelligence actually worth buying?</blockquote><h4>Sources</h4><ul><li>Anthropic, <em>Claude Agent SDK overview</em> — current SDK capabilities including built-in tools, subagents and MCP.</li><li>Anthropic, <em>Prompting best practices</em> — current guidance on adaptive thinking, effort calibration and agent behavior.</li><li>Anthropic, <em>Migration guide for current Claude models</em> — effort levels, task budgets, and cost/latency recalibration guidance.</li><li>Anthropic Engineering, <em>Introducing advanced tool use on the Claude Developer Platform</em> — Tool Search and Programmatic Tool Calling.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*SHSlgKw0rwjhKbpF.png" /></figure><p>This story is published under the <a href="https://generativeai.pub/">Generative AI</a> publication. Connect with us on <a href="https://www.linkedin.com/company/generative-ai-publication">LinkedIn</a> and follow <a href="https://www.zeniteq.com/">Zeniteq</a> to stay in the loop with the latest AI stories. Let’s shape the future of AI together!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Zy6gqF_VZKrSkLrE.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f511edc445c9" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/agent-compute-broker-stop-using-your-best-ai-model-for-every-task-f511edc445c9">Agent Compute Broker: Stop Using Your Best AI Model for Every Task</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Few Tips to Cut Claude Code Token Costs]]></title>
            <link>https://pub.towardsai.net/a-few-tips-to-cut-claude-code-token-costs-7e58af77e5b0?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e58af77e5b0</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[claude-code-tips]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Mon, 31 Aug 2026 18:01:01 GMT</pubDate>
            <atom:updated>2026-08-31T18:01:01.854Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ETeZgVaBrEFCTMyrH_x4A.png" /></figure><p>Pull up /usage after what felt like a light Claude Code session, and the numbers can be surprising. Your prompts were short. The task seemed simple. So where did all those tokens come from?</p><p>Often, not from what you typed, but from everything Claude Code had to process alongside it.</p><p>Each request can include the system prompt, project instructions, tool definitions, and relevant conversation history. Prompt caching makes repeated content much cheaper, but as a session grows, that background context can still account for a large share of the tokens associated with each turn.</p><p>A bloated CLAUDE.md, a long-running session, or changes that invalidate the prompt cache can make an otherwise simple request noticeably more expensive without changing the task itself.</p><p>In this post I have tried to walk through few key areas. What’s actually in each request, which parts grow fastest, and how to control them, from caching and compaction to thinking budgets and the stale-session trap.</p><h3>How Claude Code Structures Every Request</h3><p>Think of every request as three stacked layers:</p><ol><li><strong>System prompt</strong> — Claude Code’s own instructions, plus any MCP tool definitions</li><li><strong>Project context</strong> — your CLAUDE.md, scoped rules, and any loaded skills</li><li><strong>Conversation</strong> — the full message history from this session</li></ol><p>Claude Code uses <a href="https://code.claude.com/docs/en/prompt-caching">prompt caching</a> to avoid re-billing the same content repeatedly. Cached input tokens cost roughly 10% of the standard input token rate. That’s a real discount, but it only kicks in when the cached content stays stable. Because prompt caching depends on matching prefixes, changes near the beginning of the prompt can invalidate cached content that follows. Some modern Claude Code features, such as deferred MCP tool loading, are specifically designed to reduce unnecessary cache invalidation.</p><p>Some changes still create a cold cache. Switching models or effort levels does, and enabling fast mode for the first time in a conversation can as well. MCP changes are more nuanced because Claude Code can defer tool definitions, allowing some server changes without invalidating the existing cached prefix.</p><p>The highest-leverage optimizations target what stays the same across turns, not what changes. Keep the early layers small and stable, and the cache does the work.</p><h3>Layer 1: Tighten Your Project Context</h3><p>CLAUDE.md loads at every session start. It&#39;s the single most overlooked token sink in a typical Claude Code setup, because it grows organically as teams add instructions over time.</p><p>Anthropic’s <a href="https://code.claude.com/docs/en/costs">official cost guidance</a> recommends keeping CLAUDE.md under 200 lines.</p><p>The rule: CLAUDE.md should contain only what&#39;s true for every task in this project. Anything workflow-specific belongs in a <strong>skill</strong>.</p><p>Unlike CLAUDE.md, a skill’s full instructions load only when the skill is invoked. Its short description may remain in the base context so Claude knows when to use it. A &quot;PR review checklist&quot; or &quot;database migration procedure&quot; sitting in CLAUDE.md loads on every session. The same content as a named skill loads only when you call it. For a project with five specialized workflows, that gap compounds quickly.</p><p><strong>Path-scoped rules</strong> give you similar control at a finer grain. Rules without a paths: frontmatter key load at every session start regardless of what you&#39;re working on. Add a paths: key and the rule only enters context when Claude touches a matching file:</p><pre>---<br>paths:<br>  - &quot;src/api/**&quot;<br>---<br>Always validate request bodies against the Zod schema before processing.</pre><p>This rule’s body doesn’t consume context unless Claude works with a matching file, such as one under src/api/. Audit your rules directory, not just CLAUDE.md.</p><h3>Layer 2: Control What Tools and Servers Inject</h3><p>MCP tools still add context overhead, but modern Claude Code defers many MCP tool definitions by default. Connecting or disconnecting a deferred server generally preserves the existing cached prefix. A cache rebuild becomes relevant when those tool definitions are loaded into the prompt upfront.</p><p>Disabling servers you won’t use in a session is cleaner. Run /mcp and disconnect what doesn&#39;t apply.</p><p>Where you have a choice, prefer CLI tools over MCP equivalents. Running gh or aws from Bash adds zero per-definition overhead to the system prompt.</p><p>Model and mode stability matters here too. Switching models or changing effort level causes a cache miss. Enabling fast mode for the first time in a conversation also causes a cache miss, though later fast-mode toggles can preserve the cache. Per the <a href="https://code.claude.com/docs/en/prompt-caching">prompt caching docs</a>, these are explicit invalidation triggers. Pick your model at the top of a session and stick with it. If you use a pattern like Opus for planning and Sonnet for execution, know that each switch starts a cold cache, so the output token savings come with a re-read cost on every transition.</p><h3>Layer 3: Prune the Conversation as You Go</h3><p>The conversation layer always grows. In a long session, it’s often what dominates the bill.</p><p>Three commands manage it:</p><ul><li><strong>/clear</strong>: wipes the conversation entirely. Use it between unrelated tasks. Carrying context from a completed bug fix into a new feature is pure waste.</li><li><strong>/compact</strong> [instructions]: replaces the conversation with a summary and reduces the context carried into subsequent turns. Compaction has an upfront summarization cost, but when the existing cache is still warm, Claude can reuse that cached prefix while producing the summary. Compacting after the cache has already expired is considerably more expensive. Use custom instructions to preserve what matters: /compact Focus on code samples and API usage. You can also set a permanent default with a # Compact instructions block in CLAUDE.md.</li><li><strong>/rewind</strong>: truncates back to an already-cached prefix. No rebuild cost. Use it when Claude goes off track and you want to abandon a bad path cleanly.</li></ul><p>One caveat on compaction: don’t compact constantly. The rebuild turn is real money. The savings only pay off if the session continues long enough afterward. Compact at genuine task boundaries, not reflexively.</p><h3>The stale session trap</h3><p>Walk away from a session long enough and the cache can expire. When you return, the old prefix can no longer be served at the cheaper cache-read rate, so Claude must reprocess the context and re-establish the cache. That makes the first turn back substantially more expensive than a normal warm-cache turn. A single return to a stale long session can be the most expensive request you send all day.</p><p>If you know you’re leaving a long session and expect to return later, consider running /compact while the cache is still warm. That leaves a much smaller conversation to restore when you come back. You pay the compact cost once, and the next turn back starts from a much smaller cached prefix. Source: <a href="https://code.claude.com/docs/en/prompt-caching">Claude Code prompt caching docs</a>.</p><h3>Two 3rd Party Skills Worth taking a look</h3><h4>/graphiphy — for large codebases</h4><p>When you’re doing research or read-only exploration across a big codebase, use /graphiphy. It builds a lightweight graph of your codebase structure so Claude navigates by index rather than reading every file. Though if you have a write heavy repo, you might find yourself with outdated graph and need to update the graph often.</p><p>Github Project: <a href="https://github.com/safishamsi/graphify">https://github.com/safishamsi/graphify</a></p><h4>/caveman — for readable output</h4><p>This one isn’t just about tokens. It’s about comprehension.</p><p>LLM output is verbose by default. Long paragraphs, hedged sentences, qualifications on qualifications. /caveman forces Claude to respond in short, direct sentences. No filler. No AI prose.</p><p>The token saving could be real for some users as fewer output tokens per response. But the bigger win is speed: you read faster, catch mistakes faster, and course-correct faster. When agents talks like a caveman, you spend less time parsing and more time building.</p><p>Github Project: <a href="https://github.com/juliusbrussee/caveman">https://github.com/juliusbrussee/caveman</a></p><blockquote>Both are third-party tools rather than Anthropic products. Review their code, permissions, maintenance status, and security implications before installing them.</blockquote><p>The /usage command shows usage, cache behavior, and estimated cost. Press d for the 24-hour view or w for the 7-day view. Run it when your token use or bill surprises you. To inspect what is actually occupying the context window — such as tools, project instructions, and conversation history — use /context.</p><h3>Stop Paying for Output You Didn’t Need</h3><p>Two categories of expensive output have structural fixes.</p><h3>Extended thinking</h3><p>Reserve high-effort mode for tasks that need deep reasoning: architectural decisions, debugging subtle concurrency issues, evaluating complex tradeoffs. For “add a null check to this function,” a low thinking budget is fine and the quality difference is negligible.</p><p>Lower the effort level with /effort command</p><pre>/effort low</pre><h3>Verbose tool output</h3><p>Claude can also burn tokens on tool output you never needed. A test command that dumps thousands of log lines into the conversation can consume a large amount of context even if only a handful of lines contain useful errors.</p><p>The simplest fix is to reduce noisy output at the command itself:</p><pre>npm test 2&gt;&amp;1 | grep -A 5 -E &#39;(FAIL|ERROR|error:)&#39; | head -100</pre><p>For repeated workflows, you can automate this with a PreToolUse hook. The hook runs before a tool call executes and receives the proposed tool input as JSON. It can inspect the Bash command and return a modified updatedInput before execution.</p><p>For example, a hook script could append an output filter to selected test commands:</p><pre>#!/bin/bashINPUT=$(cat)<br>COMMAND=$(echo &quot;$INPUT&quot; | jq -r &#39;.tool_input.command&#39;)<br>if echo &quot;$COMMAND&quot; | grep -qE &#39;^(npm test|pytest|go test)&#39;; then<br>  FILTERED_COMMAND=&quot;$COMMAND 2&gt;&amp;1 | grep -A 5 -E &#39;(FAIL|ERROR|error:)&#39; | head -100&quot;<br>  jq -n \<br>    --arg command &quot;$FILTERED_COMMAND&quot; \<br>    &#39;{<br>      &quot;hookSpecificOutput&quot;: {<br>        &quot;hookEventName&quot;: &quot;PreToolUse&quot;,<br>        &quot;permissionDecision&quot;: &quot;allow&quot;,<br>        &quot;updatedInput&quot;: {<br>          &quot;command&quot;: $command<br>        }<br>      }<br>    }&#39;<br>fi</pre><p>You can then register that script as a PreToolUse hook for Bash commands.</p><p>Be careful with aggressive filtering: removing too much output can hide information Claude needs to diagnose a failure. A safer pattern is to filter commands where the useful signal is predictable, such as test failures or repetitive build logs.</p><blockquote>Another option is to delegate log-heavy exploration to a subagent. That keeps large intermediate outputs out of the main conversation context while letting the subagent return only the useful findings.</blockquote><h3>The Moves That Cost Nothing in Quality</h3><p>Some optimizations are purely mechanical, with no quality tradeoff at all:</p><ul><li><strong>Write specific prompts.</strong> “Add input validation to login in auth.ts&quot; runs without scanning the entire codebase. &quot;Improve the code&quot; can touch anything.</li><li><strong>Use plan mode before complex tasks.</strong> Catching a wrong direction in the plan phase is far cheaper than unwinding file edits.</li><li><strong>Press Escape early.</strong> If a response is going the wrong direction, abort it. Use /rewind to restore to the last good state rather than paying for a full rollback conversation.</li></ul><h3>Measure First, Then Cut</h3><p>Most token waste in Claude Code comes from what you pay to <em>load</em>, not from what you ask Claude to <em>do</em>. An oversized CLAUDE.md, unscoped rules, default thinking budgets, and mid-session model switches all generate charges before you type a single word.</p><p>Start here: use /usage to understand token consumption, cache behavior, and cost, then use /context to inspect what is occupying the context window. If tool definitions take up significant space, review your MCP connections and tool configuration. If project instructions are large, audit CLAUDE.md against the 200-line guideline and move specialized workflows into skills or scoped rules. If conversation history dominates, consider clearer task boundaries, /clear, or strategic use of /compact.</p><p>One honest caveat: skills, hooks, and scoped rules add maintenance surface. For a solo developer on a small codebase with a modest monthly spend, the setup overhead may not be worth it. These tools pay off at team scale and on projects where sessions regularly run long. Know your situation before optimizing for it.</p><p>The numbers in /usage tell you whether you have a real problem. Start there.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e58af77e5b0" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/a-few-tips-to-cut-claude-code-token-costs-7e58af77e5b0">A Few Tips to Cut Claude Code Token Costs</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LangGraph Agents: A Practical Guide to Building Stateful AI Workflows]]></title>
            <link>https://pub.towardsai.net/langgraph-agents-a-practical-guide-to-building-stateful-ai-workflows-cbbcd30310e8?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/cbbcd30310e8</guid>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Sun, 30 Aug 2026 00:01:01 GMT</pubDate>
            <atom:updated>2026-08-30T00:01:01.464Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>How state, nodes, edges, tools, persistence, interrupts, and deterministic control fit together in a reliable AI-agent architecture.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y7AddCEPnnnAeawRYOgGjQ.png" /></figure><p>A useful AI application often needs much more than “send a prompt, receive an answer.” It may need to classify a request, retrieve trusted data, call APIs, ask for approval, remember progress, retry after failure, and decide what should happen next.</p><p><strong>LangGraph</strong> is a low-level orchestration framework for building these stateful workflows and agents as a graph of steps. Those steps can branch, loop, persist progress, call tools, stream intermediate events, and pause for human input.</p><blockquote><strong><em>The simplest mental model is: state is what the system knows, nodes do the work, and edges decide what runs next.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*QgWy2CVPMFzst2k4LfrMDQ.png" /></figure><h3>Why orchestration matters</h3><p>Consider a customer asking: <em>“My order arrived damaged. Can I get a refund?”</em></p><p>A production system may need to:</p><ul><li>understand the request,</li><li>extract or locate an order ID,</li><li>load the real order record,</li><li>apply refund policy,</li><li>route large refunds to a human,</li><li>perform the refund exactly once, and</li><li>explain the verified result to the customer.</li></ul><p>If all of this logic is hidden inside one large function or one opaque model loop, it becomes difficult to test, observe, resume, and control. LangGraph makes the control flow explicit while still allowing model-driven decisions where they are useful.</p><h3>A delivery-hub analogy</h3><p>Imagine a parcel moving through a delivery hub:</p><ul><li><strong>State</strong> is the parcel’s label and contents: information carried through the process.</li><li><strong>Nodes</strong> are stations: inspect address, weigh parcel, request payment, load truck.</li><li><strong>Edges</strong> are conveyor choices: they determine which station comes next.</li><li><strong>Checkpoints</strong> are scanner logs: they record how far the parcel got.</li><li><strong>Interrupts</strong> are supervisor stops: work pauses until a human approves or corrects something.</li></ul><p>The analogy is useful because it makes execution steps and transitions visible. It is not exact: software nodes can run in parallel, invoke probabilistic models, revisit earlier nodes, and update only selected fields of shared state.</p><h3>The core LangGraph model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/777/1*1eX2C6bhMQGfhsc1Et1lcw.png" /></figure><h3>1. State: the shared working data</h3><p><strong>State</strong> is the current snapshot of information available to graph steps. A state schema defines which fields exist and, in Python, usually their types.</p><pre>from typing_extensions import TypedDict<br>class CounterState(TypedDict):<br>    count: int</pre><p>If the current state is {&quot;count&quot;: 4}, a node can read that value and return an update such as {&quot;count&quot;: 5}. A node does not need to return every state field.</p><p>A useful rule is to keep state limited to the information required to coordinate the workflow. Treat it as shared working data, not as a dump of every temporary local variable.</p><h3>2. Nodes: units of work</h3><p>A <strong>node</strong> is executable logic. It may call a model, run deterministic code, validate output, query a service, perform a tool action, or request human input.</p><pre>def increment(state: CounterState):<br>    return {&quot;count&quot;: state[&quot;count&quot;] + 1}</pre><p>If count is 4, this node returns 5. Returning an explicit update gives the runtime a clean boundary for state management.</p><p>This boundary is also valuable for software engineering: small nodes let you test routing separately from database access, tool execution, validation, or an LLM call.</p><h3>3. Edges: control flow</h3><p>An <strong>edge</strong> determines which node executes next. A normal edge always follows the same transition. A conditional edge calls routing logic and chooses a destination based on state.</p><pre>from typing import Literal<br>def route(state: CounterState) -&gt; Literal[&quot;again&quot;, &quot;done&quot;]:<br>    return &quot;again&quot; if state[&quot;count&quot;] &lt; 3 else &quot;done&quot;</pre><p>Conditional routing is especially useful for exact business rules. If ordinary code can determine the answer precisely, there is usually no benefit in asking an LLM to guess it.</p><h3>A minimal graph</h3><p>With the Graph API, StateGraph is a builder. You add nodes and edges, then call compile() to produce an executable graph.</p><pre>from langgraph.graph import StateGraph, START, END<br>builder = StateGraph(MyState)<br>builder.add_node(&quot;work&quot;, work_node)<br>builder.add_edge(START, &quot;work&quot;)<br>builder.add_edge(&quot;work&quot;, END)<br>graph = builder.compile()</pre><p>START and END are special markers rather than business-logic nodes. START represents where input enters; END represents termination.</p><h3>The agent loop: model, tool, model</h3><p>A tool-capable model can produce a structured request containing a tool name and arguments. The application, not the model itself, performs the real operation.</p><pre>messages -&gt; model -&gt; tool request?<br>                    | no  -&gt; finish<br>                    |<br>                    | yes<br>                    v<br>                   tool -&gt; tool result -&gt; model -&gt; ...</pre><p>The model chooses only from the tools you expose. Tool descriptions and schemas therefore become part of the agent’s interface design. A vague tool description can lead the model to call the right capability incorrectly.</p><p>This is also where an important distinction appears: a <strong>workflow</strong> follows predetermined code paths, while an <strong>agent</strong> dynamically chooses actions and tool usage. Strong systems often combine the two: deterministic workflow structure around a smaller agentic region.</p><h3>Do not make everything agentic</h3><p>One of the most practical design lessons in the source material is that an LLM does not have to control everything.</p><p>A robust refund workflow might divide responsibility like this:</p><ul><li><strong>LLM:</strong> understand a free-form customer request.</li><li><strong>Backend service:</strong> load trusted order data.</li><li><strong>Deterministic code:</strong> apply exact refund-policy rules.</li><li><strong>Human:</strong> approve a high-value refund.</li><li><strong>Refund API:</strong> move money.</li><li><strong>LLM:</strong> turn the verified outcome into a clear customer-facing response.</li></ul><blockquote><strong><em>Use models where flexible judgment adds value. Keep exact rules, permissions, and irreversible actions constrained.</em></strong></blockquote><h3>create_agent or custom LangGraph?</h3><p>The section distinguishes a higher-level agent harness from lower-level orchestration. As of the current LangChain documentation, LangChain’s create_agent is the recommended higher-level option for a familiar model-plus-tools loop and is built on LangGraph.</p><pre>from langchain.agents import create_agent<br>def get_weather(city: str) -&gt; str:<br>    &quot;&quot;&quot;Return weather data for a city.&quot;&quot;&quot;<br>    return f&quot;Weather lookup for {city}&quot;<br>agent = create_agent(<br>    model=&quot;openai:gpt-5.5&quot;,<br>    tools=[get_weather],<br>    system_prompt=&quot;Use tools when needed, then answer concisely.&quot;,<br>)<br>result = agent.invoke({<br>    &quot;messages&quot;: [<br>        {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: &quot;Weather in Kolkata?&quot;}<br>    ]<br>})</pre><p>The expected behavior is that the model can choose the weather tool, receive its returned value, and then answer the user. Exact wording depends on the configured model and provider.</p><p><strong>Use </strong><strong>create_agent first when:</strong></p><ul><li>your architecture is mainly the standard model-plus-tools loop,</li><li>you do not need to own every graph transition, and</li><li>the prebuilt harness already matches the workflow you need.</li></ul><p><strong>Use custom LangGraph when you need:</strong></p><ul><li>custom state,</li><li>nontrivial branching or loops,</li><li>explicit multi-stage orchestration,</li><li>specialized human review,</li><li>subgraphs or parallel fan-out, or</li><li>fine control over persistence and execution.</li></ul><h3>Reducers and message state</h3><p>A <strong>reducer</strong> defines how an existing state value combines with a new update. Without a special reducer, a new update commonly replaces the previous value. Some fields instead need accumulation.</p><p>For example, if a list field uses concatenation, conceptually:</p><pre>new list = old list + incoming items</pre><p>Here + means list concatenation, not numeric addition. [&quot;a&quot;] + [&quot;b&quot;] becomes [&quot;a&quot;, &quot;b&quot;].</p><p>For chat-style agents, MessagesState provides a standard convenience state for maintaining a chronological sequence of user messages, model replies, tool requests, and tool results.</p><p>Reducers matter even more when multiple branches update the same key. Parallel execution creates a state-design problem: simultaneous updates must have intentional merge behavior.</p><h3>Persistence: checkpoints and stores are not the same thing</h3><p>LangGraph separates two kinds of persistence</p><ul><li><strong>Checkpointer:</strong> saves snapshots of one thread’s graph state.</li><li><strong>Store:</strong> persists application-defined data that may be shared across threads.</li></ul><p>A checkpointer supports conversation continuity, resumption, human-in-the-loop flows, debugging or time-travel behavior, and fault recovery. A thread_id identifies which thread state to load.</p><pre>from langgraph.checkpoint.memory import InMemorySaver<br>checkpointer = InMemorySaver()<br>graph = builder.compile(checkpointer=checkpointer)<br>config = {<br>    &quot;configurable&quot;: {<br>        &quot;thread_id&quot;: &quot;user-42-chat-1&quot;<br>    }<br>}<br>result = graph.invoke(input_state, config=config)</pre><p>InMemorySaver is useful for examples, but it loses data when the process restarts. A production system that needs durable recovery should use a persistence backend suited to its deployment.</p><p>A store solves a different problem: durable application information that should not be tied to only one conversation thread.</p><h3>Human-in-the-loop with interrupts</h3><p>An <strong>interrupt</strong> pauses graph execution and surfaces a JSON-serializable payload to the caller. This is useful when the workflow needs an approval or other external decision before continuing.</p><pre>from langgraph.types import interrupt<br>def approval_node(state):<br>    approved = interrupt({<br>        &quot;question&quot;: &quot;Approve refund?&quot;,<br>        &quot;amount&quot;: state[&quot;refund_amount&quot;],<br>    })<br>    return {&quot;approved&quot;: bool(approved)}</pre><p>To continue, the same thread is resumed with a Command(resume=...) value.</p><p>The key implementation detail is easy to miss: when execution resumes, the interrupted node starts again from its beginning. That means side effects performed before interrupt() should be idempotent, or better, moved after approval.</p><h3>Streaming and control primitives</h3><p>Streaming lets a caller observe intermediate execution instead of waiting only for the final result. Depending on the API and stream mode, the caller may observe message chunks, state snapshots or updates, custom events, and interrupts.</p><p>Streaming does not make the underlying work faster; it makes progress visible sooner and can make long-running agent execution easier to debug.</p><p>LangGraph also provides Command, which can combine a state update with a routing decision from inside a node. It is useful when a node naturally produces both data and control flow. The source cautions against using it everywhere: ordinary edges keep simple flows easier to read.</p><p>For dynamic parallel fan-out, Send can create downstream work items with different inputs. Subgraphs can package reusable workflows, teams of agents, or specialized stages behind a clear input/output contract.</p><h3>A worked architecture: customer-support refund agent</h3><p>The PDF’s integrated example combines the main ideas into a refund workflow. It is presented as an original teaching example rather than an official policy.</p><p><strong>State might include:</strong></p><ul><li>messages: conversation messages,</li><li>order_id: parsed order identifier,</li><li>order: trusted backend order data,</li><li>refund_amount: requested or computed amount,</li><li>eligible: deterministic policy result,</li><li>approved: human decision when needed, and</li><li>refund_id: backend identifier proving the refund occurred.</li></ul><pre>START -&gt; understand_request -&gt; load_order -&gt; policy_check<br>                                             /    |    \<br>                                    ineligible  small  large<br>                                        |         |      |<br>                                     explain    refund  approval<br>                                        |         |      |<br>                                       END      reply &lt;- refund<br>                                                  |<br>                                                 END</pre><h3>Step 1: understand the request</h3><p>A model can turn free text such as “Please refund order A123; the item arrived damaged” into structured fields. This is a good model task because natural language varies.</p><h3>Step 2: load trusted order data</h3><p>A deterministic service call should fetch the real order record. Do not let the model invent price, payment status, delivery state, or whether the order exists.</p><h3>Step 3: apply policy in code</h3><pre>def policy_check(state):<br>    order = state[&quot;order&quot;]<br>    eligible = (<br>        order[&quot;delivered&quot;]<br>        and order[&quot;days_since_delivery&quot;] &lt;= 30<br>        and not order[&quot;already_refunded&quot;]<br>    )<br>    return {&quot;eligible&quot;: eligible}</pre><p>If the order was delivered eight days ago and has not already been refunded, all three Boolean conditions are true and eligible becomes True.</p><h3>Step 4: route by risk</h3><p>In the teaching example, eligible refunds at or below Rs. 2,000 can proceed automatically, while larger refunds require human review.</p><pre>def refund_route(state):<br>    if not state[&quot;eligible&quot;]:<br>        return &quot;reject&quot;<br>    if state[&quot;refund_amount&quot;] &lt;= 2000:<br>        return &quot;auto_refund&quot;<br>    return &quot;human_review&quot;</pre><p>For eligible=True and refund_amount=1500, the route is &quot;auto_refund&quot;. For Rs. 3,500, the route is &quot;human_review&quot;.</p><h3>Step 5: request approval when required</h3><p>The graph pauses with an interrupt containing only the information a reviewer needs: order ID, amount, reason, and relevant policy facts. The same thread_id must be reused when execution resumes.</p><h3>Step 6: make the side effect safe</h3><p>The refund API should accept an idempotency key such as refund:&lt;order_id&gt;:&lt;case_id&gt; when the external service supports that pattern. Idempotent means that retrying the same logical operation does not create a duplicate refund.</p><p>The returned refund_id should be stored in state so later nodes work from the verified outcome rather than from a model&#39;s assumption.</p><h3>Step 7: compose the customer response</h3><p>The model can turn the authoritative result into a friendly explanation. It should receive the actual status and refund identifier rather than decide whether money moved.</p><blockquote><strong><em>The agentic part is language understanding or flexible tool selection. The deterministic part should include eligibility rules, permission checks, monetary side effects, and idempotency.</em></strong></blockquote><h3>Production engineering lessons</h3><p>The source recommends a practical implementation sequence:</p><ol><li>Define the state schema before writing nodes.</li><li>Mark which facts are trusted business data versus model-generated guesses.</li><li>Make each node do one testable job.</li><li>Put exact rules in code; use models for interpretation, ranking, drafting, and flexible decisions.</li><li>Add persistence before introducing interrupts or long-running resumable work.</li><li>Make external side effects idempotent and record their identifiers.</li><li>Stream useful progress to the UI without exposing hidden model reasoning.</li><li>Trace, evaluate, and load-test representative paths before production rollout.</li></ol><h3>Tests that catch real failures</h3><ul><li><strong>Node unit tests:</strong> given fixed state, assert the returned update.</li><li><strong>Routing tests:</strong> verify every boundary condition, especially exact thresholds.</li><li><strong>Graph path tests:</strong> simulate common end-to-end routes without real payments.</li><li><strong>Tool contract tests:</strong> verify schemas, authorization, timeouts, and error mapping.</li><li><strong>Resume tests:</strong> interrupt, restart if feasible, then resume using the same thread.</li><li><strong>Idempotency tests:</strong> repeat a side-effect node and confirm only one real operation occurs.</li><li><strong>Evaluation tests:</strong> for model-driven steps, use a curated dataset and measurable pass/fail criteria.</li></ul><h3>Common implementation mistakes</h3><ul><li>One giant node that hides parsing, retrieval, validation, and side effects.</li><li>Storing secrets or huge raw objects in graph state without need.</li><li>Letting an LLM enforce exact policy or authorization rules.</li><li>Using in-memory persistence in production and assuming it survives restarts.</li><li>Performing a payment, email, or database side effect before an interrupt without idempotency.</li><li>Infinite or expensive loops with no termination strategy.</li><li>Tool descriptions that omit units, allowed values, or failure semantics.</li></ul><h3>Seven common LangGraph misunderstandings</h3><h3>1. “LangGraph is an LLM.”</h3><p>It is not. LangGraph orchestrates execution. A graph can call models from different providers, or it can contain no model at all.</p><h3>2. “Every node is an AI agent.”</h3><p>A node is simply a function or runnable step. It may be deterministic code, a service call, a model call, or a human-input step.</p><h3>3. “State means chat history.”</h3><p>Messages are one possible state field. State is the graph’s broader working snapshot: IDs, flags, structured results, counters, approvals, trusted data, and more.</p><h3>4. “A checkpointer and a store are the same memory mechanism.”</h3><p>They solve different problems. A checkpointer persists thread-scoped graph snapshots; a store holds application-defined information that may be shared across threads.</p><h3>5. “create_agent replaced LangGraph.&quot;</h3><p>create_agent is a higher-level LangChain harness built on LangGraph. Custom LangGraph remains useful when lower-level orchestration control is required.</p><h3>6. “An interrupt resumes from the exact Python instruction after interrupt().&quot;</h3><p>The interrupted node starts again from its beginning when resumed. That is why side-effect placement and idempotency matter.</p><h3>7. “More autonomy always means a better agent.”</h3><p>More freedom is not automatically better. Give the model flexibility only where it creates value; constrain exact rules, permissions, and irreversible actions.</p><h3>A memory technique: SNER</h3><ul><li><strong>State</strong> = what the system currently knows.</li><li><strong>Nodes</strong> = work that uses or changes that knowledge.</li><li><strong>Edges</strong> = what runs next.</li><li><strong>Runtime</strong> = persistence, streaming, interrupts, retries, and execution behavior.</li></ul><p>Picture a railway map: the train carries state, stations are nodes, tracks are edges, switches are conditional routes, station logs are checkpoints, and a red signal is an interrupt.</p><h3>Final takeaway</h3><p>LangGraph is useful when agent behavior must be explicit and operationally reliable. Its value is not that every step becomes “AI.” Its value is that you can combine probabilistic model decisions with deterministic software engineering in one visible execution model.</p><p>Start with the smallest abstraction that fits. If a simple model-plus-tools loop is enough, a higher-level agent harness may be the better choice. If you need custom state, nontrivial branching, durable execution, human approval, carefully controlled side effects, or multi-stage orchestration, custom LangGraph gives you the lower-level control to design those behaviors directly.</p><blockquote><strong><em>State remembers. Nodes work. Edges route. Checkpoints save. Interrupts ask humans. Tools act on the outside world.</em></strong></blockquote><h3>References</h3><ul><li><a href="https://docs.langchain.com/oss/python/langgraph/overview">LangGraph overview</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cbbcd30310e8" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/langgraph-agents-a-practical-guide-to-building-stateful-ai-workflows-cbbcd30310e8">LangGraph Agents: A Practical Guide to Building Stateful AI Workflows</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Peripheral Brain: The Cold Memory Layer for My Coding Agents]]></title>
            <link>https://pub.towardsai.net/my-peripheral-brain-the-cold-memory-layer-for-my-coding-agents-b1e331cdcf2a?source=rss-4ff8e09a21d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1e331cdcf2a</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Arijit Dutta]]></dc:creator>
            <pubDate>Sun, 21 Jun 2026 12:01:01 GMT</pubDate>
            <atom:updated>2026-06-21T12:01:01.019Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*asm-CRKP6BAF_IjaPJvYuA.png" /><figcaption>AI generated illustration</figcaption></figure><h3><strong>The Problem</strong></h3><p>Everyone is building their second brain for their coding agents and rightly so. In the agentic development era, I also found it particularly difficult to keep track of all my in-progress ideas and projects, sometimes too many things are moving at once. Some projects have completed development but are pending thorough testing. Low-priority items keep getting pushed aside to make way for burning items. And somewhere in between, few ideas get lost.</p><p>The loss is not dramatic. You do not forget a project exists. You just forget where you left it. You forget what you had decided about the implementation, which edge case you were stuck on, or why you had deprioritized it in the first place. By the time you return, you spend 20 minutes reconstructing context that should have been one read away.</p><p>Pre-agentic notetaking approach doesn’t work anymore. Notes created manually, outside the agent context, break the loop. The problem was never the format. Saving happened in one place, work happened in another. Retrieval meant launching a search expedition. Neither felt worth the interruption, so I skipped both. Which meant the context existed nowhere except my head, which is the least reliable storage layer I have.</p><h3>TL;DR</h3><p>Coding agents lose context when sessions end. I built a cold memory layer using a Claude Code skill that automatically creates a Jira ticket and saves a structured design doc to an Obsidian vault synced via Google Drive. Save happens at the end of every planning session, one command. Retrieval happens from inside any future agent session, one prompt. Both ends live inside the workflow so neither gets skipped.</p><h3>What I Actually Needed</h3><p>I wanted a non-volatile, persistent cold memory layer, something that lives outside the context window but is reachable from within any agent session. A place where I could park:</p><ul><li>In-progress projects with their current status</li><li>Design for critical items I am currently working on</li><li>Deprioritized ideas waiting for their turn</li><li>Debugging gotchas and hard-won insights</li></ul><p>So whenever I need implementation details for one of my projects, the status of any item, or I need to pick something up from this pool, I have all the information at my fingertips inside the agent session. No searching across artifacts and notes, no digging through chat history, no trying to remember what I named that .md file three weeks ago.</p><h4><strong>The Design constraint I kept returning to</strong></h4><p>The memory system had to be native to the agent workflow. Both ends of it had to live inside the session.</p><p>If saving required me to switch tools, copy context manually, and structure a note on my own, I would skip it when I was in flow. I know this about myself. Every note-taking system I have tried collapsed for exactly this reason. The friction at save time is invisible until it accumulates, and then one day you check your notes and they are two months out of date.</p><p>The retrieval side had a symmetric problem. If pulling context meant leaving the agent session, searching a separate app, copying something back in, and then re-orienting the agent, the cost was high enough that I would often just try to reconstruct from memory instead. Which meant I was back to the original problem.</p><p>Both steps needed to be prompt-shaped. Something I could trigger from inside a Claude Code session without breaking the work.</p><h3><strong>My Workflow</strong></h3><p>The workflow came together around a simple trigger: as soon as I am happy with the planning phase for a work item, I ask Claude Code to create a Jira ticket for it with a problem statement and proposed solution baked in. <strong>That ticket becomes the canonical record for the work item from day one.</strong></p><p>Alongside the ticket, Claude saves the design document or the RCA document for bugs as a .md file and stores it in a Google Drive synced Obsidian vault. Every work item now has two things: a trackable Jira ticket and a human-readable artifact living in my vault.</p><p>The repetitive prompt that kicks all of this off create ticket, write doc, save to vault is wrapped into a skill. Once my planning is done, I run the skill instead of re-typing the same instructions every time.</p><p>The retrieval side is just as simple. Later in any coding session, I can ask the agent to refer to my Obsidian vault as a brain dump. It fetches the relevant design doc, checks the status, picks up where I left off. The vault acts as the cold memory layer I wanted outside the context window, but fully reachable from within any agent session.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fdz5xk5vhDkaauapaym1-Q.png" /></figure><h3>The Stack i am using</h3><ul><li><strong>Jira</strong> — tracks the work item with problem statement and proposed solution</li><li><strong>Obsidian </strong>— stores the design doc or RCA as a .md artifact</li><li><strong>Google Drive</strong> — syncs the vault in Cloud, making the brain portable</li><li><strong>Claude Code skill</strong> — Custom skill which automates the save workflow</li><li><strong>Agent retrieval</strong> — claude code or any coding agengt session can query the vault on demand</li><li><strong><em>Prerequisite:</em></strong><em> Some of the above tools require Claude Code MCP(e.g. </em>Atlassian MCP for Jira<em>) to be configured. The agent retrieval and save workflow will not function without it.</em></li></ul><p>Jira tracks the work item with problem statement and proposed solution. Obsidian stores the design doc or RCA as a .md artifact. Google Drive syncs the vault to the cloud, making the brain portable across machines. A Claude Code skill wraps the save workflow into a single command. Agent retrieval means any Claude Code session can query the vault on demand.</p><blockquote>Note: The architecture is tool-agnostic. What matters is that the save side is structured enough for the retrieval side to work cleanly. Tools shown above (Jira, Obsidian, Google Drive) are for reference only. You can use any similar tool of your preference.</blockquote><h4><strong>What This Looks Like in Practice</strong></h4><p>I finish a planning conversation with the agent about a new feature. Instead of ending the session, I tell Claude Code to run the memory skill. It creates the Jira ticket, writes the design doc with the decisions we just made, and saves it to the vault. The session ends with the context preserved, not lost.</p><p>Three weeks later, I pick up some enahancements on the mentioend feature. I start a new session and ask the agent to pull the design doc for that work item from the vault. It comes back with the full context. The edge cases I already knew about. The approach I decided on and why. The Jira ticket link. I have started implentation within five minutes, not excavating.</p><p>For bugs, the pattern is the same. I hit a production issue, investigate, fix it. Before closing the session, the RCA goes to the vault. Next time I see similar symptoms, make changes it the fix or add a new test case, I ask the agent to search the vault for anything related. It finds the RCA from three months ago. That kind of retrieval used to require remembering which Slack thread or which commit message had the relevant note.</p><h4><strong>What Gets Stored and How</strong></h4><p>The design doc is not a transcript of the conversation. It is a structured artifact the agent writes at the end of the planning phase, capturing the key decisions, rejected alternatives, open questions, and next steps.</p><p>For feature work, the doc typically includes a one-paragraph problem statement, the proposed solution with the specific approach I chose, any edge cases already identified, and the Jira ticket ID. For bugs, the RCA doc captures the symptom, the root cause, the fix, and any related production context that took effort to discover. That last category is the most valuable one in the long run.</p><p>The structure matters because retrieval has to work without me guiding it. If I later ask the agent to search for anything related to rate limiting or anything related to the payment processor timeout, it needs enough signal in the doc to find the right file. Vague notes do not retrieve cleanly. Structured ones do.</p><h4><strong>The Real Test</strong></h4><p>The only thing that matters in a memory system is whether you actually use it. Most note-taking workflows collapse because the save step is manual and the retrieval step is disconnected from where you work. This one survives because both steps happen inside the agent session.</p><p>The vault is not a parallel system I maintain separately. It is something the agent reads in the same session where I am doing the work. The distance between “I need context” and “I have context” is one prompt.</p><p>Most people will build a notes system and abandon it because the friction is too high on either end. The ones who keep it running are the ones who made both ends live inside the workflow itself.</p><p>If you have a similar setup or a completely different approach to agent memory, drop it in the comments. I’m genuinely curious what others have landed on.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1e331cdcf2a" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/my-peripheral-brain-the-cold-memory-layer-for-my-coding-agents-b1e331cdcf2a">My Peripheral Brain: The Cold Memory Layer for My Coding Agents</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>