<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Klaus Helbig on Medium]]></title>
        <description><![CDATA[Stories by Klaus Helbig on Medium]]></description>
        <link>https://medium.com/@klaus.helbig?source=rss-ed5922f5c062------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*VUj9qUvC9aluH4L462zIMA.png</url>
            <title>Stories by Klaus Helbig on Medium</title>
            <link>https://medium.com/@klaus.helbig?source=rss-ed5922f5c062------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 04:32:11 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@klaus.helbig/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to Actually Test Your Claude Code Skills and Agents]]></title>
            <link>https://medium.com/@klaus.helbig/how-to-actually-test-your-claude-code-skills-and-agents-ff2e85c04b03?source=rss-ed5922f5c062------2</link>
            <guid isPermaLink="false">https://medium.com/p/ff2e85c04b03</guid>
            <dc:creator><![CDATA[Klaus Helbig]]></dc:creator>
            <pubDate>Fri, 24 Apr 2026 06:05:07 GMT</pubDate>
            <atom:updated>2026-04-24T06:07:09.569Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kPuhGrwdHHQzvZxSgOm_gQ.png" /></figure><h3>A practical evaluation playbook, built around a security-reviewer agent</h3><h4><strong><em>TL;DR</em></strong></h4><ul><li>Skills and subagents are stochastic software. One good run proves nothing. Measure pass^k, not just pass@1.</li><li>Write three evals <em>before</em> writing the body: positive, refusal, edge/scope-boundary.</li><li>Per scenario, declare both <em>expectations</em> (must say) and <em>prohibitions</em> (must not claim). A triggered prohibition fails the case instantly.</li><li>Never grade your own v1-vs-v2 comparison. Use a blind judge or a fresh Claude session.</li><li>Form drives style. Skill descriptions need quoted triggers; subagent descriptions need &lt;example&gt; blocks. The evaluation rigor underneath is identical.</li></ul><h3>What’s in this article</h3><ol><li>The Ship-and-Mute Story</li><li>Why “It Worked When I Tried It” Proves Nothing</li><li>Vocabulary First</li><li>Our Worked Example: security-reviewer</li><li>Writing the Three Evals</li><li>Running the Loop: pass@1, pass^k, train/test</li><li>Scoring: Rubric + Coverage + Structural</li><li>The Blindness Principle</li><li>Documenting and Shipping</li></ol><ul><li>Appendix A — Skill or Subagent?</li><li>Appendix B — Checklist You Can Paste Into Your Wiki</li></ul><h3>1. The Ship-and-Mute Story</h3><p>You ship a reviewer agent. Tidy description, long instruction body, four evals green. You tag v1.0.0 and move on.</p><p>A few weeks later, two incidents land on the same afternoon. The reviewer blocks a clean PR because the file it touches also contains a long-standing, backlog-tracked vulnerability on lines it never modified. On the next PR, it flags a test fixture full of obviously synthetic dummy credentials as a hardcoded-credential violation.</p><p>Two false positives in one day. The team starts muting the reviewer in their heads. A week later a real vulnerability slips past elsewhere, and now nobody trusts any of the reviewers. What you hadn’t tested were the <em>boundaries</em>, and you hadn’t run the same scenarios more than once.</p><blockquote><strong><em>Skills and subagents are stochastic software, and stochastic software needs stochastic tests.</em></strong></blockquote><p>This article is the playbook I wish I’d had. Our worked example is security-reviewer, a subagent in a spec-driven review pipeline. The mechanics apply identically whether you package the logic as a <strong>Skill</strong> sharing the main session or a <strong>Subagent</strong> in its own context: both route off a description, both produce output against expectations, both need to behave consistently across runs.</p><p>Stop treating skills and subagents as <em>documentation you wrote for an AI</em>. They are code. They have behavior, and behavior can be wrong in both directions. Miss the real vulnerability and it ships. Flag the clean thing and your team stops reading your output. Neither failure is catchable by <em>“it worked when I tried it.”</em></p><h3>2. Why “It Worked When I Tried It” Proves Nothing</h3><p>Skills and subagents share the same runtime mechanic: Markdown with YAML frontmatter plus a body of instructions. Only the frontmatter is preloaded into Claude’s context. The body loads on demand, once Claude decides the component is relevant. That routing decision happens on the description alone.</p><p>Two things make this genuinely hard to test.</p><p><strong>Routing is probabilistic.</strong> Claude doesn’t regex your description. It semantically decides whether the component applies, and that decision changes run to run. Anthropic has publicly acknowledged that skills and subagents tend to <em>undertrigger</em>. The reverse failure — <em>over-triggering</em> on near-miss prompts — is just as common and less discussed.</p><p><strong>One good run proves nothing.</strong> A single pass is a data point of one on a noisy signal. Components that look rock-solid on a single-shot eval routinely drift to much lower reliability when the same task is run repeatedly.</p><p>You cannot spot-check your way to a reliable production gate.</p><h3>3. Vocabulary First</h3><p>The rest of the article leans on a small set of terms: four actors that run during an eval, two judge signals, and a handful of reliability metrics. Quick definitions first, then we put them to work.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W6nBdZVKz98usAiBQmrSbg.png" /><figcaption><em>Executor and Grader run every time. Comparator and Analyzer run when you’re iterating on a new version.</em></figcaption></figure><h3>The four actors</h3><ul><li><strong>Executor</strong> — runs the agent against a case’s prompt and records the transcript, timing, and tokens. Typically a claude -p --agent &lt;name&gt; subprocess per case.</li><li><strong>Grader</strong> — scores the transcript. In practice the grader is split into two LLM judges (rubric + coverage, below) plus a cheap deterministic structural check.</li><li><strong>Comparator</strong> — blind-compares v1 and v2 of a component. Critical during iteration; hands labels A/B to a grader that doesn’t know which is which.</li><li><strong>Analyzer</strong> — reads a failing run and explains <em>why</em>. Human or LLM; the point is to turn a red cell into a next change.</li></ul><h3>The two judge signals</h3><ul><li><strong>Rubric</strong> — agent-wide quality criteria (“correct verdict”, “cites CWEs”, “never modifies code”). Lives in evals/agents/&lt;agent&gt;/rubric.md. Each criterion has a weight: <strong>hard-fail</strong> (below threshold = instant fail, no exceptions), <strong>critical</strong> (core correctness; below threshold = case fails), <strong>high</strong> (important, slightly more forgiving), <strong>medium</strong> (counts toward the composite, no threshold). Scored 0–4.</li><li><strong>Coverage</strong> — case-specific assertions. Two lists per scenario: expected_behavior (outcomes that <em>must</em> appear in the transcript) and forbidden_claims (outcomes that <em>must not</em>). A single triggered forbidden claim fails the case instantly.</li></ul><p>Rubric asks “does the agent do its job?” Coverage asks “for <em>this specific case</em>, does the output cover the right things and avoid the wrong ones?” Same transcript, two different questions.</p><h3>Reliability vocabulary</h3><ul><li><strong>pass@1</strong> — probability a single run of the case passes. The classic benchmark number. Details in §6.</li><li><strong>pass^k</strong> — probability <em>k consecutive runs</em> all pass. Under independence, pass^k = (pass@1)^k. Details in §6.</li><li><strong>Train set vs test set</strong> — a split of your scenarios. The <em>train set</em> is what you use to iterate on a description; the <em>test set</em> is held out and only scored at the end. Without a split, you tune against the prompts you were looking at — which looks like progress and isn’t.</li></ul><h3>4. Our Worked Example: security-reviewer</h3><p>security-reviewer is a subagent that sits at the reviewer stage of a spec-driven pipeline. After code is written, a fan of reviewer agents runs in parallel, each emitting a verdict. security-reviewer produces one of two outcomes — SECURE or FINDINGS — walking five phases of OWASP/CWE checks.</p><p>Its v1 description is third-person and names the five check categories. That is the minimum. It does <em>not</em> name explicit trigger phrases, and it does <em>not</em> carve out anti-triggers. We’ll come back to why that matters for routing in §6.</p><p>We’ll keep following this agent through the rest of the article: it provides the YAML example in §5, the pass^k math in §6, the composite scoring example in §7, and the blind v1-vs-v2 comparison in §8.</p><h3>5. Writing the Three Evals</h3><p>Before writing another line of a skill or agent body, write three evaluation scenarios. Anthropic’s own guidance is blunt: <em>build evals before writing documentation.</em> Without evals, you’re documenting imagined requirements.</p><h3>The floor: positive, refusal, edge</h3><p>Your floor is three, each for a distinct reason:</p><ol><li><strong>A positive case.</strong> The agent should trigger and do its primary job. For security-reviewer: a fixture with planted vulnerabilities, expect FINDINGS, with specific critical issues correctly called out.</li><li><strong>A negative or refusal case.</strong> Either the component should not trigger at all (Skill routed by natural-language prompt) or it was correctly routed but must refuse an out-of-role request (Subagent spawned by an orchestrator). A reviewer asked to <em>fix</em> what it finds is the classic shape: routing was right, the action is not its role.</li><li><strong>An edge or scope case.</strong> Unusual but valid inputs where a naive implementation fabricates findings, or where the agent must hold a sharp boundary. A comment-only diff. A file whose touched lines are clean but whose unmodified lines contain a known, tracked, out-of-scope vulnerability. Edge cases are where reviewers become nuisance tools.</li></ol><p>Three is the floor, not the ceiling. Five is the comfortable minimum: add one paraphrased positive and one near-miss negative.</p><blockquote><strong><em>“Three evals feels like overkill for a small skill.”</em></strong><em> Three is the floor. Below it, regressions land invisibly. Writing them costs hours. Muting a noisy reviewer costs weeks of trust.</em></blockquote><h3>Source scenarios from reality, not imagination</h3><p>Your own terminal scrollback is better material than your head. Before you wrote the component, you did its job by hand. What did you actually type? That’s your positive prompt. What did you type that looked like it should trigger but shouldn’t? That’s your negative.</p><p>For domain-specific failure modes, list the weird inputs before writing a line of code: every OWASP category, credential-handling mistakes, deserialization of untrusted input, test fixtures with synthetic credentials, comment-only diffs, pre-existing vulnerabilities outside a scoped delta.</p><h3>Outcome-grade, not step-grade</h3><p>Agents regularly find valid paths you didn’t anticipate. Grading on procedures penalizes creativity.</p><p>Describe <em>outcomes</em>: “reads the diff before issuing a verdict”, “distinguishes test-fixture dummies from real secrets”, “returns a single clear verdict block, not per-line commentary”. Whether the agent greps, parses an AST, or asks a sub-judge for a second opinion is its problem.</p><h3>Declare both expectations and prohibitions</h3><p>This is the lever most eval harnesses miss. A rubric scores <em>general</em> agent quality. For each scenario you also want <em>case-specific</em> expectations and prohibitions: a list of key points the output must hit, and a list of claims the output must not make. A declared prohibition like claims no SQL injection exists fails the case the instant the agent denies the planted vulnerability.</p><p>A scenario file doesn’t need to be elaborate. Something like this per case is enough:</p><pre># evals/agents/security-reviewer/scope_boundary_untouched_vuln.yaml<br>name: scope_boundary_untouched_vuln<br>kind: edge<br>fixture: fixtures/code/pr_with_tracked_debt.diff<br>expect:<br>verdict: SECURE<br>must_mention:<br>    - &quot;touched lines are clean&quot;<br>    - &quot;pre-existing issue is out of scope&quot;<br>must_not_claim:<br>    - &quot;SQL injection present on new lines&quot;<br>    - &quot;test-fixture credentials are hardcoded secrets&quot;<br>rubric_overrides:<br>scope_discipline: critical</pre><p>The rubric_overrides: scope_discipline: critical line pins one rubric criterion as <strong>critical</strong> for this case. In the weight system from §3, that means: if scope_discipline scores below threshold, the case fails regardless of the composite.</p><h3>Avoid these anti-patterns</h3><ul><li><strong>Vague success criteria</strong> — “produces a good review” lets an LLM judge invent its own definition and grade inconsistently.</li><li><strong>Over-strict exact-string checks</strong> — a clean input correctly returns SECURE and fails the assertion for the wrong reason.</li><li><strong>Non-hermetic inputs</strong> — live databases, wall-clock time, network calls. Flaky evals get ignored.</li><li><strong>Opaque identifiers</strong> — test_07 means nothing when it regresses six months later. positive_sql_injection_on_raw_query tells you what broke.</li></ul><h3>By hand or generated?</h3><p>Three realistic ways evals come into existence, and they stack rather than compete:</p><ul><li><strong>By hand, from scrollback and known failure modes.</strong> This is the default for your <em>first</em> three evals. You have to understand them yourself before you can judge anything downstream.</li><li><strong>Claude-generated, then reviewed.</strong> Anthropic’s own skill-creator / agent-development flows can propose candidate cases from a description and a rubric. Good for broadening coverage after the hand-written floor is in place. You stay the reviewer — every generated case gets read, tweaked, and committed only if it actually pins a behavior you care about.</li><li><strong>Backfilled from real incidents.</strong> When something breaks in production, write the eval first, fix second. Slowest to accumulate, highest-leverage over time: every past failure becomes a regression test.</li></ul><p>The rule: <strong>the first three evals you write yourself. Everything above that, Claude may propose, you decide.</strong></p><h3>6. Running the Loop: pass@1, pass^k, train/test</h3><p>Your eval suite passes on your dev machine. You ship. A week later a vulnerability slips through. On that one PR the reviewer happened to fail. But every PR runs this thing.</p><h3>What a run looks like</h3><p>One command should drive the whole loop. Your harness invocation will end up looking something like:</p><pre>python3 evals/run-eval.py agent --agent security-reviewer --runs 3</pre><p>Per case, three Claude processes spawn: one executor and two judges (rubric, coverage). A structural overlay adds cheap deterministic checks — exit code, required keywords, artifact presence. The harness then computes a composite score and a verdict, and writes a JSON result plus a human-readable Markdown summary next to the transcripts.</p><h3>pass@1 vs pass^k</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q9FhOHQxlJOLKPQwGOWkAA.png" /><figcaption><em>Two metrics, two different questions. Don’t confuse them.</em></figcaption></figure><p>pass@1 is <em>capability</em> — can the agent do X at all? Right number for benchmarks. pass^k is <em>reliability</em> — does it do X every time the scenario comes up? Right number for anything that runs more than once in production.</p><p>Under independence, the math is trivial: pass^k = (pass@1)^k. In reality runs are <em>not</em> fully independent — the same prompt tends to trigger correlated failures — so the true reliability is usually a bit worse. Treat pass^k as an optimistic upper bound.</p><p><strong>A concrete example.</strong>security-reviewer scored pass@1 = 0.95 on its positive case. Sounds solid. It sees roughly 40 PRs per week, so: pass^40 = 0.95^40 ≈ 0.13. In about 87% of weeks the reviewer misses the real issue at least once. The team notices by Friday.</p><p>The negative case is worse. A single-run pass rate of 90% sounds fine, but over 20 near-misses in a day pass^20 ≈ 0.12. The reviewer hijacks nearly every interaction it shouldn&#39;t have triggered on. Not a reviewer: a nuisance.</p><p><strong>Rule of thumb.</strong> For capability evals track pass@1. For production evals track pass^k, where k is roughly how often the scenario appears in a day of real use. A component with high pass@1 but low pass^3 is not production-ready, no matter how green your dashboard. For safety gates, the <em>negative</em>pass^k is usually the number that decides whether your team trusts the component at all.</p><blockquote><strong><em>“We ship once a week. Do we really need </em></strong><strong><em>pass^k?&quot;</em></strong><em> Yes. </em><em>k is how many times the scenario is exposed, not how often you deploy. If the reviewer sees 40 PRs a week, </em><em>pass^40 is your real reliability number.</em></blockquote><h3>Train set vs test set</h3><p>In classical ML, train/test is obvious. Here it’s less so, so: the <strong>train set</strong> is the prompts you use to tune a description — generate candidates, score, pick a winner. The <strong>test set</strong> is a held-out slice of prompts you score the final pick on, <em>once</em>. Without a split, every tweak to the description drifts toward the exact prompts you were looking at while tweaking, and the dashboard goes up while the real reliability stays flat or drops.</p><h3>Description optimization — where do the candidates come from?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nt8EkmZmOre6qMoU4XDiQA.png" /><figcaption><em>The harness scores every candidate description on the held-out test set, never on the train set it was tuned against.</em></figcaption></figure><p>Three common sources, roughly in the order you’d use them:</p><ol><li><strong>By hand.</strong> You write v1 from your understanding of the agent. This is the baseline.</li><li><strong>Claude-generated, briefed on train-set failures.</strong> You hand Claude the failing transcripts from the train set and ask for a revised description. This is what skill-creator does internally, and it&#39;s the pattern Anthropic describes in <a href="https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills"><em>Improving skill-creator</em></a>.</li><li><strong>Hill-climbing.</strong> Take the current best description, feed in the latest batch of train-set failures, ask for the next candidate, measure on held-out test, keep it only if the test score goes up. Stop when you plateau.</li></ol><p>Across all three, optimization converges on the same shape: pushier, more explicit about boundaries, anchored on concrete phrasing. Pick the winner by <strong>test</strong> score, never <strong>train</strong>.</p><h3>7. Scoring: Rubric + Coverage + Structural</h3><p>Pass rates alone don’t capture whether a component is any good. You want a single number plus a breakdown. Four signals feed that number.</p><h3>The rubric judge</h3><p>Scores the output against the agent-wide rubric criteria — “correct verdict”, “detection accuracy”, “report structure” — each weighted hard-fail / critical / high / medium and scored 0–4 with a pass threshold per weight. Certain criteria (&quot;never modifies code&quot;, &quot;never invents rules&quot;) are declared hard-fail: below threshold and the case fails regardless of anything else.</p><h3>The coverage judge</h3><p>Scores the same transcript against the case-specific expected_behavior and forbidden_claims from the scenario file. Every expected item met contributes. Every triggered prohibition forces the case to fail.</p><h3>The structural overlay</h3><p>The non-LLM layer: exit codes, protocol keywords like BLOCKER, required artifacts on disk. If the process crashed or violated a protocol contract, no judge verdict matters.</p><h3>The composite</h3><p>A weighted blend of the two judge signals:</p><pre>composite = 0.5 × rubric + 0.5 × coverage</pre><p>General quality and case-specific correctness are both essential, and they catch different failure modes. Equal weight is a sensible default for small teams.</p><h3>The consistency floor</h3><p>One hard gate on top of the composite: a component with a great average but a low pass^k fails regardless of its mean. For safety gates, raise that bar on both critical-positive and scope-boundary scenarios. Missing a real vulnerability erodes trust. Blocking a clean PR erodes trust faster.</p><blockquote><strong><em>Run the matrix on at least two model sizes.</em></strong><em> Before picking your production executor, measure. Pattern-heavy checks usually hold on a smaller model. Cross-file reasoning, protocol discipline, and the judgment that tells a real secret from a test-fixture dummy are where smaller models drift. A run like </em><em>--models sonnet,haiku --runs 3 gives you the comparison cheaply.</em></blockquote><h3>8. The Blindness Principle</h3><p>One more thing, and it’s the hardest to do emotionally. When you compare v1 and v2 of your component:</p><blockquote><strong><em>You cannot be the judge.</em></strong></blockquote><p>You’ll pick v2 because you want v2 to be better. Your grader has to be an LLM, fed two outputs labeled A and B without knowing which is old. Only after a verdict do you flip the labels.</p><blockquote><strong><em>Example.</em></strong><em> You “improve” the </em><em>security-reviewer description. Side-by-side, v2 reads tighter and more deliberate. Blind comparator, labels hidden: v1 wins 11 of 20. You thought you were shipping progress; you were shipping regression.</em></blockquote><p>With a blind comparator in place it is surprisingly common to discover a “new and improved” version is actually losing head-to-head. Without blindness, that regression ships with a clean conscience. Same principle beyond version comparisons: if you can, let a fresh Claude session with no context do the grading.</p><blockquote><strong><em>“Can’t I just grade v2 vs v1 myself? I wrote them.”</em></strong><em> That is precisely why you can’t. Authorship is a bias, not a credential.</em></blockquote><h3>9. Documenting and Shipping</h3><p>Scores only matter if you can see them over time. The difference between “we test our components” and “we know our components work” is version control.</p><p>Every plugin gets an evals/ directory that mirrors the component tree. Per component: scenarios, rubric, expectations, fixtures, a growing scorecard. Per run: machine-readable JSON with transcripts, timings, tokens, cost, rubric scores, coverage evidence, composite — plus a Markdown scorecard you actually look at first. Files are deterministically named with a timestamp-plus-component prefix, so any run is findable forever.</p><p>CI integration is what makes this work. Wire a gate that runs the suite on every PR touching the component or its eval assets, post the composite as a PR comment, and block merge on a composite drop beyond a declared threshold — unless the PR body explicitly justifies it. This single policy prevents slow death-by-tuning, where every PR trades a bit of quality for a feature nobody asked for.</p><h3>Closing</h3><p>If you take one thing away:</p><blockquote><strong><em>Write the three evals — positive, boundary, refusal — before you write the body of the skill or agent.</em></strong></blockquote><p>Everything else flows from that. You end up with shorter, more precise components, because you know exactly what problem you’re solving: the one your evals describe. And the next time a clean PR lands against a file with long-forgotten debt, your reviewer stays quiet — because you measured.</p><h3>Appendix A — Skill or Subagent?</h3><p>The evaluation machinery is identical whether your component is a Skill or a Subagent. Routing mechanic: same. Three-evals floor: same. pass^k matters just as much. The differences are operational.</p><p><strong>Skill</strong> — shares the main session’s context, routed by Claude against natural-language prompts, best for mid-conversation single-task assistance. Description shape: third person, explicit quoted trigger phrases, plus an explicit “Not for…” anti-trigger clause.</p><p><strong>Subagent</strong> — runs in its own isolated context, routed by an orchestrator choosing among available descriptions, best for end-to-end slices in a pipeline, CI gates, parallel fan-out of reviewers. Description shape: &lt;example&gt; blocks covering a direct positive, a scope-boundary case, and a refusal. Three blocks minimum — the pattern Anthropic uses in its own plugin-authoring agents, and it carries far more routing signal than a bare verb list.</p><p>Whichever you choose, write the description to match. A Skill with a subagent-style description buried in &lt;example&gt; blocks will under-trigger in conversation. A Subagent with a Skill-style quoted-trigger list will look thin to an orchestrator choosing between agents. Form drives style. Evaluation rigor is identical.</p><h3>Appendix B — Checklist You Can Paste Into Your Team Wiki</h3><ol><li>Write three evals first: positive, negative/refusal, edge. For reviewer-class components add a scope-boundary eval.</li><li>Source scenarios from real sessions, failure modes, and near-misses. Not imagination.</li><li>Outcome-grade, not step-grade. Describe results, not procedures.</li><li>Declare both expected behavior and forbidden claims per case. Rubric scores generic quality. Coverage scores case-specific correctness.</li><li>Write descriptions to match the form. Skills: quoted triggers + explicit “Not for…”. Subagents: &lt;example&gt; blocks covering direct positive, scope-boundary, and refusal.</li><li>Run each eval multiple times. Report pass@1<em>and</em>pass^k. Hold out a fraction of your prompts as a test set; optimize on train, score on test.</li><li>Blind-compare v1 and v2 when iterating. Never grade your own outputs.</li><li>Compute a composite from rubric + coverage + structural signals; gate ship-readiness on a threshold. For safety gates, raise the consistency bar.</li><li>Check in evals/: scenarios, rubrics, fixtures, results. Wire a CI gate that blocks composite regressions unless the PR body explicitly justifies them.</li><li>Test on at least two model sizes before downgrading for cost.</li></ol><p><strong>Further reading</strong></p><ul><li><a href="https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills">Improving skill-creator: Test, measure, and refine Agent Skills</a> — Anthropic</li><li><a href="https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices">Skill authoring best practices</a> — Claude API Docs</li><li><a href="https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents">Demystifying evals for AI agents</a> — Anthropic Engineering</li><li><a href="https://deepeval.com/docs/metrics-tool-correctness">Tool Correctness</a> — DeepEval</li></ul><p><em>Tags: #ClaudeCode #AIAgents #Testing #LLMOps #CodeReview #DevSecOps #DevTools</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff2e85c04b03" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>