<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Raphael Pantaleão on Medium]]></title>
        <description><![CDATA[Stories by Raphael Pantaleão on Medium]]></description>
        <link>https://medium.com/@raphaelpanta?source=rss-4fe4510de473------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Owqko3kHeOiFUKgydOr3-A.jpeg</url>
            <title>Stories by Raphael Pantaleão on Medium</title>
            <link>https://medium.com/@raphaelpanta?source=rss-4fe4510de473------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 21:15:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@raphaelpanta/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Escrevi uma constituição para um agente de IA. Eis o que ela conseguiu e o que não conseguiu impor.]]></title>
            <link>https://medium.com/@raphaelpanta/escrevi-uma-constitui%C3%A7%C3%A3o-para-um-agente-de-ia-eis-o-que-ela-conseguiu-e-o-que-n%C3%A3o-conseguiu-impor-31332eadc192?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/31332eadc192</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Fri, 16 Oct 2026 14:03:59 GMT</pubDate>
            <atom:updated>2026-10-16T14:03:59.462Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Oito princípios, 55 MUSTs, um número de versão semântica, uma checagem que roda duas vezes por feature e o PASS que acabou sendo falso.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*RPIvJm0zwHThtFSskFLW6w.png" /></figure><blockquote><em>“Imperativos do tipo definido neste memorando devem ser usados com cuidado e com parcimônia.” — RFC 2119, </em>Key words for use in RFCs to Indicate Requirement Levels<em> (1997)</em></blockquote><p>O documento no centro desta série tem 268 linhas, mora em .specify/memory/constitution.md e usa a palavra MUST 48 vezes e MUST NOT sete. Ele se chama de constituição. Tem número de versão, 1.1.1, data de ratificação, um procedimento de emenda com versionamento semântico e uma cláusula de governança que diz que ele &quot;se sobrepõe a todas as outras práticas, templates e preferências pessoais.&quot;</p><p>Foi escrito em 2 de outubro de 2026, numa sessão com um agente de IA, antes de existir qualquer código que ele governa. Toda feature desde então teve de prestar contas a ele duas vezes, por escrito. As <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Partes 1 a 5</a> desta série mostraram o que ele produziu: as specs, os hooks, o gate de mutação, duas execuções guiadas por goal. Esta sexta e última parte é sobre o documento em si: o que é uma constituição para um agente, o que ela consegue impor, o que não consegue, e o único lugar em que uma checagem contra ela disse PASS e estava errada.</p><p><em>As citações de fontes em inglês estão em tradução livre; os links levam ao original. As palavras-chave da RFC 2119 (MUST, SHOULD) e os nomes dos artefatos ficam em inglês, como no repositório.</em></p><h3>O que ela é</h3><p>O comando /speckit-constitution do Spec Kit &quot;cria ou atualiza a constituição do projeto&quot;, que a referência descreve como &quot;os princípios orientadores contra os quais toda fase posterior é avaliada&quot; (<a href="https://github.github.io/spec-kit/reference/agentic-sdd.html#speckitconstitution">referência de comandos</a>). Rode uma vez, atualize quando os princípios mudarem. O template que ele preenche tem espaços para princípios, restrições, fluxo de trabalho e governança.</p><p>A instância deste repositório tem oito princípios. Em ordem: monorepo Kotlin idiomático; arquitetura hexagonal com DDD; segurança por design (marcado NON-NEGOTIABLE); funcional e não bloqueante em primeiro lugar; um contrato de testes em camadas (NON-NEGOTIABLE); fronteiras de microsserviços; o frontend em TypeScript e React; e harness engineering orientada a hooks e econômica em tokens. Depois, uma seção de stack tecnológica com restrições duras, uma seção de fluxo de trabalho com os três quality gates, e governança.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*_k_HLDgfxWcm_4C5HcJTAA.png" /></figure><p>Cada princípio tem o mesmo formato: um título, bullets escritos com as palavras-chave da <a href="https://www.rfc-editor.org/rfc/rfc2119">RFC 2119</a> e um parágrafo de justificativa. A RFC 2119 diz que MUST “significa que a definição é um requisito absoluto da especificação”, e SHOULD significa que “podem existir razões válidas em circunstâncias particulares para ignorar um item específico”. Este documento tem 55 MUSTs e nenhum SHOULD. Toda regra é absoluta. O conselho da própria RFC, de que tais imperativos “devem ser usados com cuidado e com parcimônia”, não é seguido, e eu volto a se isso importa.</p><p>Os parágrafos de justificativa são a parte que eu defenderia com mais força. O do Princípio V diz: “testes de propriedade encontram os casos que humanos deixam passar, Testcontainers e stubs pegam falhas reais de integração, o Pact evita quebras distribuídas, e Cucumber no nível de comportamento mantém a suíte de aceitação estável entre refatorações.” Uma regra com motivo pode ser aplicada a uma situação que a regra não previu; uma regra nua não pode. A justificativa é também o que um humano lê ao decidir se emenda.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*bbto90trtsR4JUMPEI6HyQ.png" /></figure><p>Contar os imperativos por princípio diz algo sobre onde estava a ansiedade do autor. Segurança tem nove MUSTs e um MUST NOT. O princípio do harness também, com dois MUST NOTs. O documento é tão prescritivo sobre como o agente trabalha quanto sobre como o software é protegido, e as Partes 2 e 3 são a consequência.</p><h3>O que “a constituição vence” significa, e o que não significa</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*VcirW_0gD0rblyiCv6Yfhw.png" /></figure><p>A seção de governança declara uma ordem de autoridade: “Onde um template, skill ou CLAUDE.md conflitar com ela, a constituição vence e o documento conflitante MUST ser emendado.&quot; Isso é uma frase sobre documentos. Não é um mecanismo.</p><p>A documentação do próprio Claude Code é cuidadosa com a diferença. Arquivos CLAUDE.md e a memória automática são “carregados no início de toda conversa”, e então: “o Claude os trata como contexto, não como configuração imposta. Para bloquear uma ação independentemente do que o Claude decidir, use um hook de PreToolUse” (<a href="https://code.claude.com/docs/en/memory">How Claude remembers your project</a>). A constituição nem sequer é carregada no início da sessão; ela é lida porque o template do /speckit-plan manda o agente lê-la, e porque o Princípio VIII diz que agentes &quot;MUST ler esta constituição e o CLAUDE.md antes de planejar&quot;. Isso é uma instrução para ler uma instrução.</p><p>Então nada na constituição é imposto por estar na constituição. Cada regra é imposta exatamente onde algo determinístico a reafirma:</p><ul><li>“Nenhum tipo de framework em módulos domain&quot; é imposto pela divisão de módulos do Gradle (19 módulos, três por serviço) e por regras Konsist, não pelo Princípio II.</li><li>“ktlint e detekt MUST passar” é imposto pelo check e pelo hook por arquivo, não pelo Princípio I.</li><li>“O score de mutação… MUST NOT diminuir” é imposto pelo mutation-gate.sh no gate do PR, não pelo Princípio VIII.</li><li>“Steps MUST NOT referenciar endpoints, verbos HTTP, tabelas, classes, seletores” é imposto por nada. Um revisor tem de ler os feature files.</li></ul><p>A distinção de Böckeler da Parte 2 se aplica: a constituição é um guia, feedforward, inferencial. Os hooks são sensores. O verdadeiro trabalho da constituição é ser o único lugar de onde os sensores derivam, e o único documento a que um plano precisa prestar contas. Isso é muito, e não é imposição.</p><h3>A checagem que roda duas vezes</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*hBuWDPLIBs7FKUkPHh_Lnw.png" /></figure><p>O mecanismo que faz da constituição mais do que um prompt longo é uma tabela em todo plan.md. O template a chama de Constitution Check e a marca como &quot;GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.&quot; Uma linha por princípio, duas colunas de veredito. Existem catorze dessas tabelas nas sete features.</p><p>Quando um plano não consegue passar uma linha limpa, ela vai para uma segunda tabela, Complexity Tracking: “Preencha SOMENTE se o Constitution Check tiver violações que precisem ser justificadas.” Cada linha nomeia a violação, por que é necessária e a alternativa mais simples que foi rejeitada. Nas sete features há 24 linhas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Jb7l7x2eiXFzrPVJhXwrzw.png" /></figure><p>O plano da 004 tem cinco. Dezenove módulos Gradle em vez de sete, porque “um import de framework no domínio compila até um teste rodar” só com Konsist. Flyway sobre JDBC bloqueante na inicialização, porque “não há ferramenta madura de migração não bloqueante”. Um gateway escrito em Kotlin em vez de configurado no Kong, porque suas regras de segurança “precisam ser testadas por unidade, contrato e mutação como qualquer outro código Kotlin”. Cada uma é um architecture decision record numa linha. O argumento de Michael Nygard, de 2011, a favor dos ADRs (<a href="https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions">Documenting Architecture Decisions</a>) é que “é melhor evitar tanto a aceitação cega quanto a reversão cega”, e esta tabela é onde isso acontece: a regra é quebrada, por escrito, com a alternativa nomeada, para que um leitor futuro possa revertê-la de propósito.</p><p>Acho que essa tabela é a melhor ideia de toda a configuração. Ela dá à constituição uma válvula de escape, e faz da válvula de escape um documento.</p><h3>O PASS que estava errado</h3><p>Agora a parte que eu preferiria não escrever.</p><p>A linha VIII, harness econômico em tokens, passou nas catorze checagens. A célula do plano da 004 diz: “Hooks em .claude/hooks/* ativos; Gradle silencioso; subagentes usados para os artefatos deste plano; mutação no gate de fim de tarefa.&quot; Tudo verdade. O Princípio VIII também diz: &quot;A saída de CLI MUST ser filtrada por wrappers economizadores de tokens (rtk, tokensave) e resumida.&quot; A célula não os menciona, e a checagem passou mesmo assim.</p><p>Em 13 de outubro rodei os relatórios das duas ferramentas na máquina de desenvolvimento. rtk gain -p: dois comandos, 577 tokens de entrada, 577 de saída, &quot;Tokens saved: 0 (0.0%)&quot;, e um aviso: &quot;No hook installed — run rtk init -g for automatic token savings.&quot; tokensave cost: US$ 356 de uso em sete dias, 22 mil tokens de entrada e 1,3 milhão de saída, e &quot;Savings 697 tokens (0% efficiency)&quot;. O proxy foi instalado e nunca ligado; o grafo de código foi construído, 25.792 símbolos, e quase nunca consultado. O bloco do CLAUDE.md que manda o agente confiar na saída condensada do rtk estava mandando confiar numa condensação que não estava acontecendo.</p><p>Os hooks, a configuração silenciosa do Gradle e os limites de 40 linhas da Parte 2 fizeram a economia real de tokens. As duas ferramentas que o princípio cita pelo nome não fizeram nada. E o Constitution Check, escrito pelo agente, avaliou o princípio pelo que o agente escreveu na célula, não pelo que estava rodando na máquina.</p><p>Esse é o limite de uma checagem autoavaliada, dito da forma mais clara que consigo. A tabela é uma afirmação. As 95 recusas do hook de Stop na Parte 2 eram vereditos. Um PASS numa tabela Markdown é evidência de que o agente acreditou nele, e nada mais.</p><h3>Emendando</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*SANUx-NNwBLwoFjCOiX9pw.png" /></figure><p>Uma constituição que não pode mudar é um prompt com nome pomposo. Esta tem um procedimento: um pull request que edita o arquivo, inclui um Sync Impact Report, declara a justificativa e lista os passos de migração; aprovado pelos mantenedores; em vigor no merge. E uma política de versionamento emprestada do software: MAJOR quando um princípio é removido ou redefinido, MINOR quando um é adicionado ou materialmente expandido, PATCH para redação.</p><p>Foi emendada uma vez. Commit 534197f, no mesmo dia da ratificação: 1.1.0 para 1.1.1, PATCH, &quot;apenas a redação da Technology Stack mudou (Spring Boot 3.x → major GA mais recente, 4.1 na emenda); sem mudanças de princípio, seção ou template.&quot; O Sync Impact Report são três linhas na mensagem de commit, que é o que deveria ser.</p><p>Uma emenda em onze dias ou é sinal de que o documento estava certo ou é sinal de que ninguém tentou mudá-lo. Acho que é o segundo. Dois PRs sem spec, uma catraca nunca girada e duas ferramentas de token nunca ligadas são todas coisas que uma emenda poderia ter registrado. Nenhuma registrou.</p><h3>O caso contra</h3><p><strong>Cinquenta e cinco MUSTs e nenhum SHOULD.</strong> A RFC 2119 foi escrita para especificações de protocolo, onde ambiguidade quebra interoperabilidade. Um documento para guiar um agente não é um protocolo. Toda regra absoluta é uma regra que o agente vai ou seguir quando não deveria, ou reinterpretar em silêncio. A tabela de Complexity Tracking existe porque o autor sabia disso; um SHOULD teria custado menos que uma tabela.</p><p><strong>São 2.272 palavras, lidas em toda sessão de planejamento.</strong> A orientação da Anthropic é buscar “o conjunto mínimo de informações que descreva por completo o comportamento esperado” (<a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">Effective context engineering</a>), e a <a href="https://code.claude.com/docs/en/best-practices">página de boas práticas</a> do Claude Code avisa que, se um arquivo de instruções for longo demais, “o Claude ignora metade dele porque regras importantes se perdem no ruído”. Os parágrafos de justificativa que tornam o documento bom de ler são também a parte de que um modelo não precisa para aplicar os MUSTs. Uma versão de 400 palavras só com MUSTs para execuções de implement, mantendo esta para humanos e emendas, seria uma mudança MINOR que eu não fiz.</p><p><strong>Autoavaliação.</strong> Tudo da seção acima. O Constitution Check é escrito pela parte que está sendo checada. O Radar da Thoughtworks colocou spec-driven development em Assess em parte porque PRDs e checagens gerados têm um público pouco claro; um PASS que ninguém verifica não tem público nenhum. A correção não é uma tabela mais esperta; é um segundo agente, num contexto novo, avaliando a tabela, o que a página de boas práticas do Claude Code chama de revisão adversarial e o que este repositório não faz.</p><p><strong>“Criar regras detalhadas à mão para a IA, no fim, não escala.”</strong> A linha do Radar, de novo. Oito princípios para um desenvolvedor em onze dias funcionaram. A pergunta que o documento não consegue responder sobre si mesmo é como ele fica no princípio vinte, com um time, depois de um ano de emendas PATCH.</p><p><strong>E sem comparação.</strong> Eu não construí esta plataforma sob um CLAUDE.md de 100 palavras para ver o que a constituição comprou. A base de código de um milhão de linhas da OpenAI (<a href="https://openai.com/index/harness-engineering/">Harness engineering</a>) usou um AGENTS.md como mapa e uma árvore de docs como sistema de registro, sem constituição e sem checagem autoavaliada, e impôs a arquitetura “mecanicamente por meio de linters customizados… e testes estruturais.” Os dois repositórios acabaram confiando nos linters. Um deles também escreveu uma constituição.</p><h3>O que eu manteria</h3><ol><li><strong>Um documento a que o plano precisa prestar contas.</strong> Não porque seja imposto, mas porque é a fonte única de onde os sensores derivam.</li><li><strong>Justificativa em toda regra.</strong> O MUST é para a checagem; o motivo é para a situação que o MUST não previu.</li><li><strong>Proibições além de mandatos.</strong> “Cobertura MUST NOT ser critério de merge” moldou mais deste repositório do que qualquer regra positiva.</li><li><strong>A tabela de Complexity Tracking.</strong> Quebre a regra por escrito, nomeie a alternativa. É um ADR com uma coluna obrigatória.</li><li><strong>Versionamento semântico e um Sync Impact Report.</strong> Mesmo que a primeira emenda seja uma mudança de redação, o hábito está lá.</li><li><strong>Um segundo avaliador para o Constitution Check.</strong> Contexto novo, lê a tabela e a máquina, não o autor.</li><li><strong>Marcar quais regras são impostas e por quê.</strong> Uma coluna na constituição: hook, build, teste, revisor, nada. As linhas “nada” são o trabalho.</li></ol><h3>Para se aprofundar</h3><p>O resto da série, um por vez:</p><ul><li><strong>Parte 1 — Sete specs, 620 tarefas.</strong> Spec-driven development testado numa plataforma de microsserviços real: o loop, clarify, converge e os dados que ninguém tem. <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Leia a Parte 1 →</a></li><li><strong>Parte 2 — O hook de Stop disse não 95 vezes.</strong> Harness engineering: três gates, um guarda de loop e dez dias de dados reais de tempo. <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Leia a Parte 2 →</a></li><li><strong>Parte 3–3.380 bugs de propósito.</strong> Mutation testing como critério de merge: Pitest e Stryker a 80 %, sobreviventes justificados no PR. <a href="https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781">Leia a Parte 3 →</a></li><li><strong>Parte 4 — Um número em vez de uma spec.</strong> /goal para performance: AOT cache do JDK 25, compact object headers, imagens nativas, de 4,3 GiB para 1,9. <a href="https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e">Leia a Parte 4 →</a></li><li><strong>Parte 5 — O Mac estava mentindo.</strong> /goal para correção: SELinux, virtiofs, um timestamp de build e um runner self-hosted que finalmente funcionou no Linux. <a href="https://medium.com/@raphaelpanta/meu-runner-de-ci-funcionava-no-meu-mac-o-mac-estava-mentindo-02e8e8ac6a9c">Leia a Parte 5 →</a></li></ul><h3>Leitura adicional</h3><ul><li>GitHub Spec Kit, /speckit.constitution — <a href="https://github.github.io/spec-kit/reference/agentic-sdd.html#speckitconstitution">referência de comandos</a>; o <a href="https://github.com/github/spec-kit/blob/main/templates/constitution-template.md">template de constituição</a></li><li>S. Bradner, RFC 2119, <em>Key words for use in RFCs to Indicate Requirement Levels</em> (1997) — <a href="https://www.rfc-editor.org/rfc/rfc2119">rfc-editor.org</a></li><li>Michael Nygard, <em>Documenting Architecture Decisions</em> (2011) — <a href="https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions">cognitect.com</a></li><li><em>How Claude remembers your project</em> (CLAUDE.md, precedência, “context, not enforced configuration”) — <a href="https://code.claude.com/docs/en/memory">code.claude.com/docs/en/memory</a>; <em>Best practices for Claude Code</em> — <a href="https://code.claude.com/docs/en/best-practices">code.claude.com/docs/en/best-practices</a></li><li>Anthropic, <em>Effective context engineering for AI agents</em> — <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">anthropic.com/engineering</a></li><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com</a></li><li>Thoughtworks Technology Radar, <em>Spec-driven development</em> — <a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">thoughtworks.com/radar</a></li><li>Ryan Lopopolo, <em>Harness engineering: leveraging Codex in an agent-first world</em> — <a href="https://openai.com/index/harness-engineering/">OpenAI</a></li><li>Semantic Versioning 2.0.0 — <a href="https://semver.org/lang/pt-BR/">semver.org</a></li><li>O repositório: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constitution.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/commit/534197f">a emenda, commit 534197f</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/templates/plan-template.md">template de plano</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/plan.md">plan.md da 004</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/005-storefront-dev-bootstrap/plan.md">plan.md da 005</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">template de PR, “Constitution gate”</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/CLAUDE.md">CLAUDE.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.rtk/filters.toml">.rtk/filters.toml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.tokensave/config.json">.tokensave/config.json</a></li></ul><p><em>As contagens (MUST/MUST NOT por princípio, linhas de Complexity Tracking, linhas de Constitution Check) são do repositório no commit </em><em>9870792. Os números de </em><em>rtk gain -p e </em><em>tokensave cost foram capturados na máquina de desenvolvimento em 13 de outubro de 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31332eadc192" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Wrote a Constitution for an AI Agent. Here’s What It Could and Couldn’t Enforce.]]></title>
            <link>https://medium.com/@raphaelpanta/i-wrote-a-constitution-for-an-ai-agent-heres-what-it-could-and-couldn-t-enforce-da0e48c3b968?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/da0e48c3b968</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[claude-code]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Fri, 16 Oct 2026 14:02:52 GMT</pubDate>
            <atom:updated>2026-10-16T14:02:52.328Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Eight principles, 55 MUSTs, a semantic version number, a check that runs twice per feature, and the PASS that turned out to be false.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*OMMN-YjJYog5kFJwjvu-UA.png" /></figure><blockquote><em>“Imperatives of the type defined in this memo must be used with care and sparingly.” — RFC 2119, </em>Key words for use in RFCs to Indicate Requirement Levels<em> (1997)</em></blockquote><p>The document at the centre of this series is 268 lines long, lives at .specify/memory/constitution.md, and uses the word MUST 48 times and MUST NOT seven. It calls itself a constitution. It has a version number, 1.1.1, a ratification date, an amendment procedure with semantic versioning, and a governance clause that says it &quot;supersedes all other practices, templates and personal preferences.&quot;</p><p>It was written on 2 October 2026, in a session with an AI agent, before any of the code it governs existed. Every feature since has had to answer to it twice in writing. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Parts 1 to 5</a> of this series showed what it produced: the specs, the hooks, the mutation gate, two goal-driven runs. This sixth and last part is about the document itself: what a constitution for an agent is, what it can enforce, what it cannot, and the one place where a check against it said PASS and was wrong.</p><h3>What it is</h3><p>Spec Kit’s /speckit-constitution command &quot;creates or updates the project constitution,&quot; which its reference describes as &quot;the guiding principles that every later phase is evaluated against&quot; (<a href="https://github.github.io/spec-kit/reference/agentic-sdd.html#speckitconstitution">command reference</a>). Run it once, update it when your principles change. The template it fills in has placeholders for principles, constraints, workflow and governance.</p><p>This repository’s instance has eight principles. In order: Kotlin-idiomatic monorepo; hexagonal architecture with DDD; security by design (marked NON-NEGOTIABLE); functional and non-blocking first; a layered test contract (NON-NEGOTIABLE); microservice boundaries; the TypeScript and React frontend; and token-efficient, hook-driven harness engineering. Then a technology-stack section with hard constraints, a workflow section with the three quality gates, and governance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Aj_piPBQOolUcTaFGi52Gg.png" /></figure><p>Each principle has the same shape: a title, bullets written in <a href="https://www.rfc-editor.org/rfc/rfc2119">RFC 2119</a> keywords, and a rationale paragraph. RFC 2119 says MUST “mean[s] that the definition is an absolute requirement of the specification”, and SHOULD means “there may exist valid reasons in particular circumstances to ignore a particular item”. This document has 55 MUSTs and no SHOULDs at all. Every rule is absolute. The RFC’s own advice, that such imperatives “must be used with care and sparingly”, is not followed, and I will come back to whether that matters.</p><p>The rationale paragraphs are the part I would defend hardest. Principle V’s reads: “property tests find the cases humans miss, Testcontainers and stubs catch real integration faults, Pact prevents distributed breakage, and behaviour-level Cucumber keeps the acceptance suite stable across refactors.” A rule with a reason can be applied to a situation the rule did not foresee; a bare rule cannot. The rationale is also what a human reads when deciding whether to amend.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*f7iAN_ygrYnnOvVDx2HTkQ.png" /></figure><p>Counting the imperatives per principle says something about where the author’s anxiety was. Security has nine MUSTs and one MUST NOT. So does the harness principle, with two MUST NOTs. The document is as prescriptive about how the agent works as about how the software is secured, and Parts 2 and 3 are the consequence.</p><h3>What “the constitution wins” means, and what it doesn’t</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*fgbJ_O1tpvFck5E_oFHXlA.png" /></figure><p>The governance section declares an order of authority: “Where a template, skill or CLAUDE.md conflicts with it, the constitution wins and the conflicting document MUST be amended.&quot; That is a sentence about documents. It is not a mechanism.</p><p>Claude Code’s own documentation is careful about the difference. CLAUDE.md files and auto memory are “loaded at the start of every conversation”, and then: “Claude treats them as context, not enforced configuration. To block an action regardless of what Claude decides, use a PreToolUse hook instead” (<a href="https://code.claude.com/docs/en/memory">How Claude remembers your project</a>). The constitution is not even loaded at session start; it is read because /speckit-plan&#39;s template tells the agent to read it, and because Principle VIII says agents &quot;MUST read this constitution and CLAUDE.md before planning&quot;. That is an instruction to read an instruction.</p><p>So nothing in the constitution is enforced by being in the constitution. Each rule is enforced exactly where something deterministic restates it:</p><ul><li>“No framework types in domain modules&quot; is enforced by the Gradle module split (19 modules, three per service) and Konsist rules, not by Principle II.</li><li>“ktlint and detekt MUST pass” is enforced by check and the per-file hook, not by Principle I.</li><li>“Mutation score… MUST NOT decrease” is enforced by mutation-gate.sh in the PR gate, not by Principle VIII.</li><li>“Steps MUST NOT reference endpoints, HTTP verbs, tables, classes, selectors” is enforced by nothing. A reviewer has to read the feature files.</li></ul><p>Böckeler’s distinction from Part 2 applies: the constitution is a guide, feedforward, inferential. The hooks are sensors. The constitution’s real job is to be the single place from which the sensors are derived, and the single document a plan has to answer to. That is a lot, and it is not enforcement.</p><h3>The check that runs twice</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*QOBqGg4-S51jD2aupUyBYA.png" /></figure><p>The mechanism that makes the constitution more than a long prompt is a table in every plan.md. The template calls it the Constitution Check and marks it &quot;GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.&quot; One row per principle, two verdict columns. Fourteen such tables exist across the seven features.</p><p>When a plan cannot pass a row cleanly, it goes to a second table, Complexity Tracking: “Fill ONLY if Constitution Check has violations that must be justified.” Each row names the violation, why it is needed, and the simpler alternative that was rejected. Across seven features there are 24 rows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Mt9tJ1rs2RWTtiIYVurz8A.png" /></figure><p>The 004 plan has five. Nineteen Gradle modules instead of seven, because “a framework import in the domain compiles until a test runs” with Konsist alone. Flyway on blocking JDBC at start-up, because there is “no mature non-blocking migration tool”. A gateway written in Kotlin instead of configured in Kong, because its security rules “must be unit-, contract- and mutation-tested like any other Kotlin code”. Each one is an architecture decision record in a row. Michael Nygard’s 2011 case for ADRs (<a href="https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions">Documenting Architecture Decisions</a>) is that “it’s better to avoid either blind acceptance or blind reversal,” and this table is where that happens: the rule is broken, in writing, with the alternative named, so a later reader can reverse it on purpose.</p><p>I think this table is the best idea in the whole setup. It gives the constitution an escape valve, and it makes the escape valve a document.</p><h3>The PASS that was wrong</h3><p>Now the part I would rather not write.</p><p>Row VIII, token-efficient harness, passed in all fourteen checks. The 004 plan’s cell reads: “Hooks in .claude/hooks/* active; quiet Gradle; subagents used for this plan&#39;s artifacts; mutation in end-of-task gate.&quot; All true. Principle VIII also says: &quot;CLI output MUST be filtered through token-saving wrappers (rtk, tokensave) and summarised.&quot; The cell does not mention them, and the check passed anyway.</p><p>On 13 October I ran the two tools’ own reports on the development machine. rtk gain -p: two commands, 577 tokens in, 577 out, &quot;Tokens saved: 0 (0.0%)&quot;, and a warning: &quot;No hook installed — run rtk init -g for automatic token savings.&quot; tokensave cost: $356 of usage over seven days, 22k input and 1.3M output tokens, and &quot;Savings 697 tokens (0% efficiency)&quot;. The proxy was installed and never wired in; the code graph was built, 25,792 symbols, and almost never queried. The CLAUDE.md block that tells the agent to trust rtk&#39;s condensed output was telling it to trust condensing that was not happening.</p><p>The hooks, the quiet Gradle configuration and the 40-line caps from Part 2 did the real token saving. The two tools the principle names by name did nothing. And the Constitution Check, written by the agent, graded the principle on what the agent wrote in the cell, not on what was running on the machine.</p><p>That is the limit of a self-assessed check, stated as plainly as I can. The table is a claim. The Stop hook’s 95 refusals in Part 2 were verdicts. A PASS in a Markdown table is evidence that the agent believed it, and no more.</p><h3>Amending it</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*YAsM-GA-8KgunMj6rkqj-g.png" /></figure><p>A constitution that cannot change is a prompt with a grand name. This one has a procedure: a pull request that edits the file, includes a Sync Impact Report, states the rationale and lists migration steps; approved by the maintainers; effective on merge. And a versioning policy borrowed from software: MAJOR when a principle is removed or redefined, MINOR when one is added or materially expanded, PATCH for wording.</p><p>It has been amended once. Commit 534197f, the same day it was ratified: 1.1.0 to 1.1.1, PATCH, &quot;only the Technology Stack wording changed (Spring Boot 3.x → latest GA major, 4.1 at amendment); no principle, section or template changes.&quot; The Sync Impact Report is three lines in the commit message, which is what it should be.</p><p>One amendment in eleven days is either a sign the document was right or a sign nobody tried to change it. I think it is the second. Two spec-less PRs, a ratchet never turned and two token tools never switched on are all things an amendment could have recorded. None did.</p><h3>The case against</h3><p><strong>Fifty-five MUSTs and no SHOULDs.</strong> RFC 2119 was written for protocol specifications, where ambiguity breaks interoperability. A document for guiding an agent is not a protocol. Every absolute rule is a rule the agent will either follow when it should not, or quietly reinterpret. The Complexity Tracking table exists because the author knew this; a SHOULD would have cost less than a table.</p><p><strong>It is 2,272 words, read every planning session.</strong> Anthropic’s guidance is to strive for “the minimal set of information that fully outlines your expected behavior” (<a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">Effective context engineering</a>), and the Claude Code <a href="https://code.claude.com/docs/en/best-practices">best-practices page</a> warns that if an instruction file is too long “Claude ignores half of it because important rules get lost in the noise”. The rationale paragraphs that make the document good to read are also the part a model does not need in order to apply the MUSTs. A 400-word MUST-only version for implement runs, with this one kept for humans and amendments, would be a MINOR change I have not made.</p><p><strong>Self-assessment.</strong> Everything in the section above. The Constitution Check is written by the party being checked. Thoughtworks’ Radar put spec-driven development in Assess partly because generated PRDs and checks have an unclear audience; a PASS nobody verifies has no audience at all. The fix is not a smarter table; it is a second agent in a fresh context grading the table, which the Claude Code best-practices page calls an adversarial review and which this repository does not do.</p><p><strong>“Handcrafting detailed rules for AI ultimately doesn’t scale.”</strong> The Radar’s line, again. Eight principles for one developer in eleven days worked. The question the document cannot answer about itself is what it looks like at principle twenty, with a team, after a year of PATCH amendments.</p><p><strong>And no comparison.</strong> I did not build this platform under a 100-word CLAUDE.md to see what the constitution bought. OpenAI’s million-line codebase (<a href="https://openai.com/index/harness-engineering/">Harness engineering</a>) used an AGENTS.md as a map and a docs tree as the system of record, with no constitution and no self-assessed check, and enforced architecture “mechanically via custom linters… and structural tests.” Both repositories ended up trusting the linters. One of them also wrote a constitution.</p><h3>What I would keep</h3><ol><li><strong>One document the plan has to answer to.</strong> Not because it is enforced, but because it is the single source the sensors are derived from.</li><li><strong>Rationale on every rule.</strong> The MUST is for the check; the reason is for the situation the MUST did not foresee.</li><li><strong>Bans as well as mandates.</strong> “Coverage MUST NOT be a merge criterion” shaped more of this repository than any positive rule.</li><li><strong>The Complexity Tracking table.</strong> Break the rule in writing, name the alternative. It is an ADR with a required column.</li><li><strong>Semantic versioning and a Sync Impact Report.</strong> Even if the first amendment is a wording change, the habit is there.</li><li><strong>A second grader for the Constitution Check.</strong> Fresh context, reads the table and the machine, not the author.</li><li><strong>Mark which rules are enforced and by what.</strong> A column in the constitution: hook, build, test, reviewer, nothing. The “nothing” rows are the work.</li></ol><h3>Go deeper</h3><p>The rest of the series, one piece at a time:</p><ul><li><strong>Part 1 — Seven specs, 620 tasks.</strong> Spec-driven development tested on a real microservices platform: the loop, clarify, converge, and the data nobody has. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Read Part 1 →</a></li><li><strong>Part 2 — The Stop hook said no 95 times.</strong> Harness engineering: three gates, a loop guard, and ten days of real timing data. <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Read Part 2 →</a></li><li><strong>Part 3–3,380 bugs on purpose.</strong> Mutation testing as the merge criterion: Pitest and Stryker at 80 %, survivors justified in the PR. <a href="https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8">Read Part 3 →</a></li><li><strong>Part 4 — A number instead of a spec.</strong> /goal for performance: JDK 25 AOT cache, compact object headers, native images, 4.3 GiB to 1.9. <a href="https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66">Read Part 4 →</a></li><li><strong>Part 5 — The Mac was lying.</strong> /goal for fixing: SELinux, virtiofs, a build-time timestamp, and a self-hosted runner that finally worked on Linux. <a href="https://medium.com/@raphaelpanta/my-ci-runner-worked-on-my-mac-the-mac-was-lying-5eece22825ce">Read Part 5 →</a></li></ul><h3>Further reading</h3><ul><li>GitHub Spec Kit, /speckit.constitution — <a href="https://github.github.io/spec-kit/reference/agentic-sdd.html#speckitconstitution">command reference</a>; the <a href="https://github.com/github/spec-kit/blob/main/templates/constitution-template.md">constitution template</a></li><li>S. Bradner, RFC 2119, <em>Key words for use in RFCs to Indicate Requirement Levels</em> (1997) — <a href="https://www.rfc-editor.org/rfc/rfc2119">rfc-editor.org</a></li><li>Michael Nygard, <em>Documenting Architecture Decisions</em> (2011) — <a href="https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions">cognitect.com</a></li><li><em>How Claude remembers your project</em> (CLAUDE.md, precedence, “context, not enforced configuration”) — <a href="https://code.claude.com/docs/en/memory">code.claude.com/docs/en/memory</a>; <em>Best practices for Claude Code</em> — <a href="https://code.claude.com/docs/en/best-practices">code.claude.com/docs/en/best-practices</a></li><li>Anthropic, <em>Effective context engineering for AI agents</em> — <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">anthropic.com/engineering</a></li><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com</a></li><li>Thoughtworks Technology Radar, <em>Spec-driven development</em> — <a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">thoughtworks.com/radar</a></li><li>Ryan Lopopolo, <em>Harness engineering: leveraging Codex in an agent-first world</em> — <a href="https://openai.com/index/harness-engineering/">OpenAI</a></li><li>Semantic Versioning 2.0.0 — <a href="https://semver.org/">semver.org</a></li><li>The repository: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constitution.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/commit/534197f">the amendment, commit 534197f</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/templates/plan-template.md">plan template</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/plan.md">004 plan.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/005-storefront-dev-bootstrap/plan.md">005 plan.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">PR template, “Constitution gate”</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/CLAUDE.md">CLAUDE.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.rtk/filters.toml">.rtk/filters.toml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.tokensave/config.json">.tokensave/config.json</a></li></ul><p><em>Counts (MUST/MUST NOT per principle, Complexity Tracking rows, Constitution Check rows) are from the repository at commit </em><em>9870792. The </em><em>rtk gain -p and </em><em>tokensave cost figures were captured on the development machine on 13 October 2026.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da0e48c3b968" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Meu runner de CI funcionava no meu Mac. O Mac estava mentindo.]]></title>
            <link>https://medium.com/@raphaelpanta/meu-runner-de-ci-funcionava-no-meu-mac-o-mac-estava-mentindo-02e8e8ac6a9c?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/02e8e8ac6a9c</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 12:11:53 GMT</pubDate>
            <atom:updated>2026-10-16T14:10:04.557Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Uma execução de correção guiada por goal num runner self-hosted do GitHub Actions: SELinux e um socket, virtiofs e 3.000 arquivos pequenos, um timestamp que derrotou o cache do Gradle e uma checagem do Pact que perguntava cedo demais.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*k5jlXAOzLH6MOk0vaFa4_Q.png" /></figure><blockquote><em>“Um sistema distribuído é aquele em que a falha de um computador que você nem sabia que existia pode tornar o seu próprio computador inutilizável.” — Leslie Lamport, 1987</em></blockquote><p>O runner de CI desta plataforma é um container Linux. Ele roda dentro de uma máquina virtual Fedora CoreOS. A VM roda num Mac. O script que configura tudo isso também roda no Mac, em bash 3.2, e por uma semana ele vinha dimensionando o runner pelas CPUs do Mac, colocando o diretório de trabalho do runner no disco do Mac e confiando na ideia que o Mac tinha do que docker significava. Três computadores, cada um mentindo um pouco para o seguinte.</p><p>A <a href="https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e">Parte 4</a> foi uma execução guiada por goal que fez um número diminuir. Esta é o outro tipo de goal: fazer uma coisa quebrada funcionar, e depois fazê-la ficar rápida. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">pull request #4</a> tocou 44 arquivos e 1.390 linhas e, como o PR #3, não tem spec. Sua primeira frase é o diagnóstico: “A stack do runner self-hosted foi montada em torno do Mac de desenvolvimento. Na VM do Podman onde ela de fato roda, os jobs de CI falhavam ou eram lentos por motivos de ambiente.”</p><p>Aqui estão os motivos, um de cada vez, com os números.</p><p><em>As citações de fontes em inglês estão em tradução livre; os links levam ao original.</em></p><h3>Três máquinas fingindo ser uma</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*ExccgJ5I9AOvoJ-T-JVYyQ.png" /></figure><p>A <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">pesquisa do MVP, seção 4</a>, escolheu um runner self-hosted porque o workflow da plataforma sobe a stack inteira (cerca de 8 GB de containers) e roda Testcontainers na camada de integração de cada serviço, e porque as imagens vão para um registry privado no host do runner. A stack do runner mora em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/platform/ci-runner">platform/ci-runner</a>: um arquivo Compose, um script <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/scripts/run-ephemeral.sh">run-ephemeral.sh</a> e um README que agora abre com um parágrafo que eu acho que todo desenvolvedor macOS rodando containers Linux deveria ler uma vez:</p><blockquote><em>Os runners são sempre containers Linux, seja qual for a máquina: no macOS, a VM Linux do engine (Podman machine, Docker Desktop, Colima) é o host desta stack.</em></blockquote><p>Tudo o que vem a seguir é consequência de esquecer isso.</p><h3>Bug 1: o SELinux e o socket</h3><p>Todo passo docker de todo job falhava com permission denied. Todo teste com Testcontainers falhava com Could not find a valid Docker environment. O container do runner monta o socket do engine, /var/run/docker.sock, e no Mac isso sempre tinha funcionado, porque o Mac não é onde o runner roda.</p><p>O Fedora CoreOS, que é o que o podman machine inicializa, roda com SELinux em modo enforcing. Um container confinado não consegue abrir o socket do engine, digam o que disserem as permissões do arquivo. A correção é uma linha no <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/docker-compose.yml">arquivo Compose</a>, security_opt: label=disable, que o <a href="https://docs.podman.io/en/latest/markdown/podman-run.1.html">podman-run</a> documenta como &quot;Desliga a separação por label para o container.&quot;</p><p>Isso é uma redução real de isolamento, e o PR diz isso nas notas para revisores: ele “remove a separação do SELinux apenas para os containers do runner. Esses containers já têm o socket do engine, equivalente a root.” Um container que consegue falar com o engine consegue subir um container privilegiado no host; os labels do SELinux não eram o que protegia o host dele. A seção de salvaguardas abaixo é.</p><h3>Bug 2: 3.000 arquivos pequenos</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*ej6L1_TpjaXE-aCf0P8i9w.png" /></figure><p>RUNNER_WORKDIR apontava para dentro de /Users/…. Isso é uma pasta do Mac, compartilhada com a VM por virtiofs. Um checkout, um build Gradle e um npm install são todos milhares de arquivos pequenos, e o compartilhamento é o lugar errado para eles por um fator que o README mediu: 3.000 arquivos pequenos escritos em 2,8 s no compartilhamento e em 0,09 s no disco da própria VM. Trinta e uma vezes. O README arredonda para &quot;30 a 50 vezes mais lento&quot; nas cargas que testou.</p><p>run-ephemeral.sh agora coloca o diretório de trabalho de cada sessão num volume do engine, em disco nativo, e os requisitos de host do <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/README.md">README</a> dizem em negrito para nunca usar uma pasta compartilhada como diretório de trabalho. A mesma mudança corrigiu o dimensionamento: CPUs e memória agora vêm do docker info, que responde pela VM (10 CPUs, 14 GiB no Mac de desenvolvimento), e não da máquina onde o script por acaso roda. --print-config imprime o que ele resolveu, que é o tipo de flag que você adiciona depois de errar sobre isso uma vez.</p><h3>Bug 3: um timestamp, todo build</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*O1Y-YmYleKZKYu0lg5oMtQ.png" /></figure><p>Este não é sobre o Mac, e vinha custando todo build desde a feature 004.</p><p>O plugin Gradle do Spring Boot gera META-INF/build-info.properties para o endpoint /info do actuator. Por padrão ele inclui build.time. A referência do Spring Boot diz, num parágrafo fácil de pular, exatamente o que isso custa (<a href="https://docs.spring.io/spring-boot/gradle-plugin/integrating-with-actuator.html">Integrating with Actuator</a>):</p><blockquote><em>Um efeito colateral disso é que a task nunca estará up-to-date. Como resultado, os builds vão demorar mais, porque mais tasks, incluindo os testes do projeto, terão de ser executadas. Outro efeito colateral é que a saída da task sempre vai mudar e, portanto, o build não será de fato repetível.</em></blockquote><p>O guia de build cache do Gradle chama a mesma coisa de entrada instável de task, sob o título “Stable task inputs” (<a href="https://docs.gradle.org/current/userguide/build_cache_concepts.html">Build Cache Concepts</a>): um timestamp significa que “as entradas da task jar serão diferentes a cada execução do build&quot;, e entradas instáveis &quot;sempre resultarão num cache miss.&quot;</p><p>O timestamp fica nos resources principais. Os resources principais são entrada de toda task de teste, do Pitest e do boot jar. Então, para as oito aplicações Boot, toda camada de teste, toda execução do Pitest e todo jar perdiam o cache em todo build, no CI e no notebook. A correção são três linhas na convenção <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-boot-app.gradle.kts">kotlin-boot-app</a>:</p><pre>springBoot {<br>    buildInfo {<br>        excludes.set(setOf(&quot;time&quot;))<br>    }<br>}</pre><p>Com isso, o README do runner pode dizer o que diz agora: com o cache aquecido, “um pipeline de serviço depois do verify (ou o contrário) pega do cache a compilação, os testes unitários, de integração e de aceitação e o Pitest.&quot; As camadas de contrato ainda rodam, de propósito, porque a saída delas vai para o broker do Pact e não para o Gradle.</p><p>Eu não teria encontrado isso lendo o build. Foi encontrado porque uma execução guiada por goal que recebe “deixe o CI rápido” olha por que o cache falha, e o cache disse por quê.</p><h3>Bug 4, e meio: cache pela rede</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*V8w9CkKIbsyibpgEYnepfA.png" /></figure><p>Os workflows tinham sido escritos do jeito que a documentação do GitHub ensina, para runners hospedados pelo GitHub: actions/setup-gradle e setup-node com o serviço de cache do GitHub ligado. Num runner hospedado, esse serviço é o único armazenamento que sobrevive a um job, e vale o que custa. Num runner self-hosted, ele significa zipar o Gradle user home, fazer upload, download e deszipar em todo job, num disco que já o tem. O serviço também é limitado: &quot;por padrão, o limite é de 10 GB por repositório&quot;, com remoção &quot;por ordem de data do último acesso&quot; (<a href="https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows">Caching dependencies</a>). E o actions/checkout apaga o workspace, levando junto o configuration cache do Gradle e os caches de ferramentas do frontend.</p><p>Então o PR #4 moveu tudo para o host do runner. setup-gradle roda com cache-disabled: ${{ runner.environment == &#39;self-hosted&#39; }}, para que os workflows continuem funcionando num runner hospedado. setup-node roda sem cache:. npm ci --prefer-offline lê um cache em npm_config_cache. Um novo <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/ci-cache.sh">ci-cache.sh</a> cria links dos diretórios que o checkout destruiria, o configuration cache do Gradle e node_modules/.cache e .vite, para um volume indexado pelo caminho do workspace. O Gradle valida seu configuration cache contra toda entrada a partir da qual ele foi calculado, então uma entrada velha nunca é usada, só reverificada (<a href="https://docs.gradle.org/current/userguide/configuration_cache.html">Configuration Cache</a>). O JDK e o Node que setup-java e setup-node instalam caem num segundo volume montado em /opt/hostedtoolcache, que é onde essas actions procuram.</p><p>A entrada sutil dessa tabela é a do Stryker. O gate da loja e o job de mutação do gate de PR rodam ambos o Stryker sobre o mesmo commit de merge, cerca de 16 minutos cada. ci-cache.sh stryker save guarda o stryker-incremental.json da primeira execução; restore o entrega à segunda, que não roda de novo nada que já rodou. Um pull request sem resultado para o seu commit recebe o anterior mais recente, e o modo incremental do Stryker roda de novo só os mutantes do que mudou.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*G0fANBtNDlbOjD6qEmbbDQ.png" /></figure><p>Os “primeiros números do runner reiniciado” do PR, medidos com os workflows antigos na stack nova: npm ci de 58 s para 10 s, lint de 319 s para 54 s, Vitest de 139 s para 11 s, setup-java de 15 s para 1 s, e o check completo do Gradle para cart passou com Testcontainers em 184 s, o que ele não conseguia de jeito nenhum antes de o bug 1 ser corrigido.</p><h3>Bug 5: uma checagem correta na hora errada</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*AJLhj0QYFjyCKwQ3j2wBCw.png" /></figure><p>Todo pipeline de consumidor termina com o can-i-deploy do Pact, que pergunta ao broker se esta versão tem &quot;um resultado de verificação bem-sucedido&quot; contra os provedores com que ela fala (<a href="https://docs.pact.io/pact_broker/can_i_deploy">docs.pact.io</a>). Em runners hospedados pelo GitHub, os pipelines dos provedores rodam em paralelo com o do consumidor e normalmente terminam antes. Num único runner Gradle self-hosted, eles entram na fila atrás dele. A loja perguntou ao broker sobre o gateway enquanto o job do gateway ainda esperava pelo runner, e o broker respondeu &quot;no such version exists&quot;.</p><p>A correção, em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/service-ci.yml">service-ci.yml</a> e <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/storefront.yml">storefront.yml</a>, mudou a ordem e não a checagem. Os gates de consumidor agora verificam seus próprios pacts nos provedores, no mesmo commit, antes de perguntar. E um <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/gateway/src/contractTest/kotlin/com/ecommerce/gateway/GatewayBrokerVerificationTest.kt">GatewayBrokerVerificationTest</a> foi adicionado porque o gateway nunca tinha verificado seu pact da loja a partir do broker, então o can-i-deploy da loja jamais poderia ter encontrado um resultado para ele, fosse qual fosse a ordem.</p><p>Depois, o segundo runner. O runner-light, 2 CPUs e 1,5 GiB, carrega o label ecommerce-light e assume os jobs de build de imagem, scans, publish, aggregate e testes de hook. Esses jobs sobem containers no engine, fora do teto do runner leve, então o teto não limita o trabalho, só o processo do runner. Um build de imagem não espera mais um verify do Gradle terminar.</p><p>Um detalhe menor do mesmo PR de que eu gosto: em main, a imagem construída fica no engine compartilhado e o job de publish confere o ID dela, em vez de ser empacotada num artefato e desempacotada de novo. IMAGE_HANDOVER=artifact restaura a ida e volta para quem tiver runners em mais de um engine.</p><h3>Quanto custa, e quem pode rodar isso</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*yq7AXiZJ_jUli8FZhr3BLg.png" /></figure><p>Um runner self-hosted num repositório público é uma decisão que o GitHub diz para você não tomar: “runners self-hosted quase nunca devem ser usados em repositórios públicos, porque qualquer usuário pode abrir pull requests contra o repositório e comprometer o ambiente” (<a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions">Security hardening for GitHub Actions</a>). Este repositório é público, e roda um. O README lista sete salvaguardas sob o título “Public-repository safeguards” e diz que elas “só funcionam juntas”:</p><ol><li>Nenhum pull request de fork no runner. Todo job carrega if: github.event_name == &#39;push&#39; || github.event.pull_request.head.repo.full_name == github.repository; nada de pull_request_target, nada de workflow_run.</li><li>“Exigir aprovação para todos os colaboradores externos” nas configurações do repositório. O guarda no workflow é uma segunda barreira, porque um fork controla a própria cópia do arquivo de workflow.</li><li>Actions fixadas pelo SHA completo do commit; imagens de ferramentas por tag e digest.</li><li>Runners efêmeros: o GitHub “atribui apenas um job a um runner” registrado com --ephemeral (<a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners">Autoscaling with self-hosted runners</a>); um container novo por sessão, o diretório de trabalho esvaziado depois.</li><li>Privilégio mínimo: permissions: contents: read, checkout sem credenciais persistidas, segredos só nos passos que os usam, nunca no check do Gradle.</li><li>Imagens escaneadas por Trivy e osv-scanner antes do push; só main publica.</li><li>O socket é root no host, então o host é uma VM dedicada que não guarda mais nada, tratada como descartável.</li></ol><p>E então a frase que eu colocaria no topo de qualquer README desses: “Se isso não puder ser cumprido, rode as checagens de pull request em runners hospedados pelo GitHub e deixe este runner para builds de imagem pós-merge.”</p><h3>O caso contra, e o que esta execução não diz</h3><p><strong>label=disable não é nada.</strong> É justificado pela montagem do socket, e a montagem do socket é justificada pelo Testcontainers, e a cadeia inteira termina em &quot;VM dedicada e descartável&quot;. É um argumento correto e também um argumento longo. A alternativa, Podman rootless com um socket por usuário, é mencionada no README e não foi adotada. Um revisor que pare em &quot;SELinux desligado para o CI&quot; não está errado em parar aí.</p><p><strong>Os primeiros números são primeiros números.</strong> npm ci de 58 → 10 s e lint de 319 → 54 s vêm de um reinício, com os workflows antigos, numa máquina que também é um notebook. Não são um benchmark. São consistentes com escritas de arquivos pequenos 30× mais lentas deixando de existir, que é a afirmação, e nada mais.</p><p><strong>Dois runners num engine continuam sendo um engine.</strong> O runner-light tira os jobs de imagem e publish da fila do Gradle; ele não adiciona CPUs. Pitest e Testcontainers continuam disputando as mesmas 10, e é por isso que o mesmo PR adicionou o <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/com/ecommerce/build/HeavyTaskLimit.kt">HeavyTaskLimit</a>, um build service do Gradle que dimensiona o número de JVMs de teste e Pitest concorrentes a partir da memória que o build enxerga. O comentário nesse arquivo é o diário de novo: &quot;Seis GiB ainda deixavam o daemon do Gradle do runner de CI (3 a 4 GB de heap) morto por OOM sob seu teto de 10 GB com três tasks pesadas.&quot;</p><p><strong>As premissas em formato de Mac eram tanto do agente quanto minhas.</strong> Os scripts originais do runner foram escritos pelo mesmo tipo de sessão que os corrigiu. Um goal que diz “faça funcionar na VM” encontrou os bugs de fronteira; nada no harness da Parte 2 conseguiria, porque todo gate rodava no notebook, onde as premissas valiam. Uma máquina diferente é um sensor, e este repositório tinha só um.</p><p><strong>E sem spec de novo.</strong> Dois PRs sem spec seguidos. A pasta specs/ agora descreve um runner de CI que não existe. Argumentei nas Partes 1 e 4 que essa é a troca certa para tuning e correção. Não argumentei, porque não posso, que as specs serão atualizadas.</p><h3>O que eu manteria</h3><ol><li><strong>Dimensionar pelo engine, não pelo host.</strong> docker info diz a verdade sobre a máquina que vai rodar o job.</li><li><strong>Nunca colocar um diretório de trabalho numa pasta compartilhada.</strong> 31× não é um problema de tuning.</li><li><strong>Excluir </strong><strong>build.time.</strong> Uma linha no plugin de convenção do Boot; toda task a jusante vira cacheável.</li><li><strong>Manter os caches onde os dados já estão.</strong> Num runner self-hosted, o serviço de cache do GitHub é uma ida e volta pela rede até você mesmo.</li><li><strong>Fazer o gate sobre a condição que o runner torna verdadeira, não sobre a que runners hospedados por acaso tornavam verdadeira.</strong> Verifique seus próprios pacts antes de perguntar ao broker.</li><li><strong>Escrever as salvaguardas, como lista, com a condição de saída.</strong> “Se isso não puder ser cumprido, não faça.”</li><li><strong>Uma flag </strong><strong>--print-config.</strong> Para todo script que infere o seu ambiente.</li></ol><h3>Para se aprofundar</h3><p>O resto da série, um por vez:</p><ul><li><strong>Parte 1 — Sete specs, 620 tarefas.</strong> Spec-driven development testado numa plataforma de microsserviços real: o loop, clarify, converge e os dados que ninguém tem. <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Leia a Parte 1 →</a></li><li><strong>Parte 2 — O hook de Stop disse não 95 vezes.</strong> Harness engineering: três gates, um guarda de loop e dez dias de dados reais de tempo. <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Leia a Parte 2 →</a></li><li><strong>Parte 3–3.380 bugs de propósito.</strong> Mutation testing como critério de merge: Pitest e Stryker a 80 %, sobreviventes justificados no PR. <a href="https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781">Leia a Parte 3 →</a></li><li><strong>Parte 4 — Um número em vez de uma spec.</strong> /goal para performance: AOT cache do JDK 25, compact object headers, imagens nativas, de 4,3 GiB para 1,9. <a href="https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e">Leia a Parte 4 →</a></li><li><strong>Parte 6 — Uma constituição para um agente de IA.</strong> Oito princípios, 55 MUSTs, uma checagem que roda duas vezes por feature e o PASS que acabou sendo falso. <a href="https://medium.com/@raphaelpanta/escrevi-uma-constitui%C3%A7%C3%A3o-para-um-agente-de-ia-eis-o-que-ela-conseguiu-e-o-que-n%C3%A3o-conseguiu-impor-31332eadc192">Leia a Parte 6 →</a></li></ul><h3>Leitura adicional</h3><ul><li>GitHub Docs, <em>Security hardening for GitHub Actions</em> — <a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions">hardening para runners self-hosted</a>; <em>Autoscaling with self-hosted runners</em>, runners efêmeros — <a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners">docs.github.com</a>; <em>Caching dependencies to speed up workflows</em> — <a href="https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows">docs.github.com</a></li><li>Spring Boot Gradle Plugin, <em>Integrating with Actuator</em> (geração de build information) — <a href="https://docs.spring.io/spring-boot/gradle-plugin/integrating-with-actuator.html">docs.spring.io</a></li><li>Gradle, <em>Build Cache Concepts</em> (stable task inputs) — <a href="https://docs.gradle.org/current/userguide/build_cache_concepts.html">docs.gradle.org</a>; <em>Configuration Cache</em> — <a href="https://docs.gradle.org/current/userguide/configuration_cache.html">docs.gradle.org</a></li><li>Podman, <em>podman-run</em>, --security-opt label=disable — <a href="https://docs.podman.io/en/latest/markdown/podman-run.1.html">docs.podman.io</a>; <em>podman-machine-init</em> — <a href="https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html">docs.podman.io</a>; Fedora CoreOS — <a href="https://docs.fedoraproject.org/en-US/fedora-coreos/">docs.fedoraproject.org</a></li><li>Pact, <em>can-i-deploy</em> — <a href="https://docs.pact.io/pact_broker/can_i_deploy">docs.pact.io</a></li><li>npm, npm ci e --prefer-offline — <a href="https://docs.npmjs.com/cli/v11/commands/npm-ci">docs.npmjs.com</a></li><li>Leslie Lamport, e-mail ao DEC SRC, 28 de maio de 1987 — <a href="https://lamport.azurewebsites.net/pubs/distributed-system.txt">lamport.azurewebsites.net/pubs/distributed-system.txt</a></li><li>O repositório: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">PR #4</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/README.md">platform/ci-runner/README.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/docker-compose.yml">docker-compose.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/scripts/run-ephemeral.sh">run-ephemeral.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/ci-cache.sh">ci-cache.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-boot-app.gradle.kts">kotlin-boot-app.gradle.kts</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/com/ecommerce/build/HeavyTaskLimit.kt">HeavyTaskLimit.kt</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/service-ci.yml">service-ci.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/gateway/src/contractTest/kotlin/com/ecommerce/gateway/GatewayBrokerVerificationTest.kt">GatewayBrokerVerificationTest.kt</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/ci-cd.md">docs/ci-cd.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">specs/004 research.md §4</a></li></ul><p><em>Os números vêm da descrição do PR #4 e de </em><em>platform/ci-runner/README.md no commit </em><em>9870792. O host do runner na época era a Podman machine do Mac de desenvolvimento (Fedora CoreOS sob libkrun, 10 CPUs, 14 GiB).</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=02e8e8ac6a9c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My CI Runner Worked on My Mac. The Mac Was Lying.]]></title>
            <link>https://medium.com/@raphaelpanta/my-ci-runner-worked-on-my-mac-the-mac-was-lying-5eece22825ce?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/5eece22825ce</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[github-actions]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 12:10:49 GMT</pubDate>
            <atom:updated>2026-10-16T14:07:18.599Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A goal-driven fixing run on a self-hosted GitHub Actions runner: SELinux and a socket, virtiofs and 3,000 small files, one timestamp that defeated the Gradle cache, and a Pact check that asked too early.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*KZ5MbZ42V6p_MZ1MQRTP6g.png" /></figure><blockquote><em>“A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable.” — Leslie Lamport, 1987</em></blockquote><p>The CI runner for this platform is a Linux container. It runs inside a Fedora CoreOS virtual machine. The VM runs on a Mac. The script that sets it all up runs on the Mac too, in bash 3.2, and for a week it had been sizing the runner from the Mac’s CPUs, putting the runner’s work directory on the Mac’s disk, and trusting the Mac’s idea of what docker meant. Three computers, each lying a little to the next.</p><p><a href="https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66">Part 4</a> was a goal-driven run that made a number smaller. This is the other kind of goal: make a thing that is broken work, and then make it fast. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">Pull request #4</a> touched 44 files and 1,390 lines, and like PR #3 it has no spec. Its first sentence is the diagnosis: “The self-hosted runner stack was set up around the development Mac. On the Podman VM it actually runs on, CI jobs failed or were slow for environmental reasons.”</p><p>Here are the reasons, one at a time, with the numbers.</p><h3>Three machines pretending to be one</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*61XFvD1LhdzJWuAe9qfm8g.png" /></figure><p>The MVP’s <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">research, section 4</a> chose a self-hosted runner because the platform workflow starts the whole stack (about 8 GB of containers) and runs Testcontainers in every service’s integration layer, and because images go to a private registry on the runner host. The runner stack lives in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/platform/ci-runner">platform/ci-runner</a>: a Compose file, a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/scripts/run-ephemeral.sh">run-ephemeral.sh</a> script, and a README that now opens with a paragraph I think every macOS developer running Linux containers should read once:</p><blockquote><em>The runners are always Linux containers, whatever the machine: on macOS the engine’s Linux VM (Podman machine, Docker Desktop, Colima) is the host of this stack.</em></blockquote><p>Everything that follows is a consequence of forgetting that.</p><h3>Bug 1: SELinux and the socket</h3><p>Every docker step in every job failed with permission denied. Every Testcontainers test failed with Could not find a valid Docker environment. The runner container mounts the engine&#39;s socket, /var/run/docker.sock, and on the Mac that had always worked, because the Mac is not where the runner runs.</p><p>Fedora CoreOS, which is what podman machine boots, runs SELinux enforcing. A confined container cannot open the engine socket, whatever the file permissions say. The fix is one line in the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/docker-compose.yml">Compose file</a>, security_opt: label=disable, which Podman&#39;s <a href="https://docs.podman.io/en/latest/markdown/podman-run.1.html">podman-run</a> documents as &quot;Turn off label separation for the container.&quot;</p><p>That is a real reduction in isolation, and the PR says so in its notes for reviewers: it “lifts SELinux separation for the runner containers only. Those containers already hold the root-equivalent engine socket.” A container that can talk to the engine can start a privileged container on the host; SELinux labels were not the thing protecting the host from it. The safeguards section below is.</p><h3>Bug 2: 3,000 small files</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*CvjnRbWreQ_LVWfecTs2og.png" /></figure><p>RUNNER_WORKDIR pointed under /Users/…. That is a Mac folder, shared into the VM over virtiofs. A checkout, a Gradle build and an npm install are all thousands of small files, and the share is the wrong place for them by a factor the README measured: 3,000 small files written in 2.8 s on the share and 0.09 s on the VM&#39;s own disk. Thirty-one times. The README rounds it to &quot;30 to 50 times slower&quot; across the workloads it tried.</p><p>run-ephemeral.sh now puts each session&#39;s work directory on an engine volume, on native disk, and the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/README.md">README</a>&#39;s host requirements say in bold never to use a shared folder as the work directory. The same change fixed sizing: CPUs and memory now come from docker info, which answers for the VM (10 CPUs, 14 GiB on the development Mac), not from the machine the script happens to run on. --print-config prints what it resolved, which is the kind of flag you add after being wrong about it once.</p><h3>Bug 3: one timestamp, every build</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*u1YKtkmIZUDaljRWy2w3hQ.png" /></figure><p>This one is not about the Mac, and it had been costing every build since feature 004.</p><p>Spring Boot’s Gradle plugin generates META-INF/build-info.properties for the actuator&#39;s /info endpoint. By default it includes build.time. The Spring Boot reference says, in a paragraph that is easy to skip, exactly what that costs (<a href="https://docs.spring.io/spring-boot/gradle-plugin/integrating-with-actuator.html">Integrating with Actuator</a>):</p><blockquote><em>A side-effect of this is that the task will never be up-to-date. As a result, builds will take longer as more tasks, including the project’s tests, will have to be executed. Another side-effect is that the task’s output will always change and, therefore, the build will not be truly repeatable.</em></blockquote><p>Gradle’s build-cache guide calls the same thing an unstable task input, under the heading “Stable task inputs” (<a href="https://docs.gradle.org/current/userguide/build_cache_concepts.html">Build Cache Concepts</a>): a timestamp means “the inputs for the jar task will be different for each build execution&quot;, and unstable inputs &quot;will always result in a cache miss.&quot;</p><p>The timestamp sits in the main resources. The main resources are an input to every test task, to Pitest and to the boot jar. So for all eight boot applications, every test layer, every Pitest run and every jar missed the cache on every build, in CI and on the laptop. The fix is three lines in the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-boot-app.gradle.kts">kotlin-boot-app</a> convention:</p><pre>springBoot {<br>    buildInfo {<br>        excludes.set(setOf(&quot;time&quot;))<br>    }<br>}</pre><p>With that, the runner README can say what it now says: with the cache warm, “a service pipeline after verify (or the other way round) takes its compilation, unit, integration and acceptance tests and Pitest from the cache.&quot; The contract layers still run, on purpose, because their outputs go to the Pact broker rather than to Gradle.</p><p>I would not have found this by reading the build. It was found because a goal-driven run that is told “make CI fast” looks at why the cache misses, and the cache said why.</p><h3>Bug 4, and a half: caching to the network</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*I2pvCda5pvLW530Z1CHZ1A.png" /></figure><p>The workflows had been written the way GitHub’s documentation teaches, for GitHub-hosted runners: actions/setup-gradle and setup-node with the GitHub cache service on. On a hosted runner that service is the only storage that outlives a job, and it is worth its cost. On a self-hosted runner it means zipping the Gradle user home, uploading it, downloading it and unzipping it on every job, onto a disk that already has it. The service is also capped: &quot;by default, the limit is 10 GB per repository&quot;, evicted &quot;in order of last access date&quot; (<a href="https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows">Caching dependencies</a>). And actions/checkout wipes the workspace, taking the Gradle configuration cache and the frontend tool caches with it.</p><p>So PR #4 moved everything to the runner host. setup-gradle runs with cache-disabled: ${{ runner.environment == &#39;self-hosted&#39; }}, so the workflows still work on a hosted runner. setup-node runs without cache:. npm ci --prefer-offline reads a cache in npm_config_cache. A new <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/ci-cache.sh">ci-cache.sh</a> links the directories checkout would destroy, the Gradle configuration cache and node_modules/.cache and .vite, into a volume keyed by workspace path. Gradle validates its configuration cache against every input it was computed from, so a stale entry is never used, only re-checked (<a href="https://docs.gradle.org/current/userguide/configuration_cache.html">Configuration Cache</a>). The JDK and Node that setup-java and setup-node install land in a second volume mounted at /opt/hostedtoolcache, which is where those actions look.</p><p>The subtle entry in that table is the Stryker one. The storefront gate and the PR gate’s mutation job both run Stryker on the same merge commit, about 16 minutes each. ci-cache.sh stryker save keeps the first run&#39;s stryker-incremental.json; restore hands it to the second, which re-runs nothing it already ran. A pull request with no result for its commit gets the newest earlier one, and Stryker&#39;s incremental mode re-runs only the mutants of what changed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*cu2Ca_Is03FKwu20QymyrQ.png" /></figure><p>The PR’s “early numbers from the restarted runner”, measured with the old workflows on the new stack: npm ci 58 s to 10 s, lint 319 s to 54 s, Vitest 139 s to 11 s, setup-java 15 s to 1 s, and cart&#39;s full Gradle check passed with Testcontainers in 184 s, which it could not pass at all before bug 1 was fixed.</p><h3>Bug 5: a correct check at the wrong moment</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*6QVneOf2LGGw_yAf3xEwIg.png" /></figure><p>Every consumer pipeline ends with Pact’s can-i-deploy, which asks the broker whether this version has &quot;a successful verification result&quot; against the providers it talks to (<a href="https://docs.pact.io/pact_broker/can_i_deploy">docs.pact.io</a>). On GitHub-hosted runners, the provider pipelines run in parallel with the consumer&#39;s and usually finish first. On one self-hosted Gradle runner they queue behind it. The storefront asked the broker about the gateway while the gateway&#39;s job was still waiting for the runner, and the broker answered &quot;no such version exists&quot;.</p><p>The fix, in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/service-ci.yml">service-ci.yml</a> and <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/storefront.yml">storefront.yml</a>, changed the order rather than the check. Consumer gates now verify their own pacts at the providers, at the same commit, before asking. And a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/gateway/src/contractTest/kotlin/com/ecommerce/gateway/GatewayBrokerVerificationTest.kt">GatewayBrokerVerificationTest</a> was added because the gateway had never verified its storefront pact from the broker at all, so the storefront&#39;s can-i-deploy could never have found a result for it, whatever the ordering.</p><p>Then the second runner. runner-light, 2 CPUs and 1.5 GiB, carries the label ecommerce-light and takes the image builds, scans, publish, aggregate and hook-test jobs. Those jobs start containers in the engine, outside the light runner&#39;s cap, so the cap does not bound the work, only the runner process. An image build no longer waits for a Gradle verify to finish.</p><p>A smaller detail from the same PR that I like: on main the built image stays in the shared engine and the publish job checks its ID, instead of being packed into an artifact and unpacked again. IMAGE_HANDOVER=artifact restores the round trip for anyone with runners on more than one engine.</p><h3>What it costs, and who may run it</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*OXn-lnQ0-Xre7AC0p2mixA.png" /></figure><p>A self-hosted runner on a public repository is a decision GitHub tells you not to make: “self-hosted runners should almost never be used for public repositories, because any user can open pull requests against the repository and compromise the environment” (<a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions">Security hardening for GitHub Actions</a>). This repository is public, and it runs one. The README lists seven safeguards under the heading “Public-repository safeguards” and says they “only work together”:</p><ol><li>No fork pull requests on the runner. Every job carries if: github.event_name == &#39;push&#39; || github.event.pull_request.head.repo.full_name == github.repository; no pull_request_target, no workflow_run.</li><li>“Require approval for all outside collaborators” in the repository settings. The workflow guard is a second barrier, because a fork controls its own copy of the workflow file.</li><li>Actions pinned by full commit SHA; tool images by tag and digest.</li><li>Ephemeral runners: GitHub “only assigns one job to a runner” registered with --ephemeral (<a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners">Autoscaling with self-hosted runners</a>); a new container per session, the work directory emptied after.</li><li>Least privilege: permissions: contents: read, checkout without persisted credentials, secrets only in the steps that use them, never in Gradle check.</li><li>Images scanned by Trivy and osv-scanner before they are pushed; only main publishes.</li><li>The socket is root on the host, so the host is a dedicated VM that holds nothing else, treated as disposable.</li></ol><p>And then the sentence I would put at the top of any such README: “If this cannot be met, run pull-request checks on GitHub-hosted runners and keep this runner for post-merge image builds.”</p><h3>The case against, and what this run can’t tell you</h3><p><strong>label=disable is not nothing.</strong> It is justified by the socket mount, and the socket mount is justified by Testcontainers, and the whole chain ends in &quot;dedicated disposable VM&quot;. That is a correct argument and also a long one. The alternative, rootless Podman with a per-user socket, is mentioned in the README and not taken. A reviewer who stops at &quot;SELinux disabled for CI&quot; is not wrong to stop there.</p><p><strong>The early numbers are early.</strong> npm ci 58 → 10 s and lint 319 → 54 s are from one restart, with the old workflows, on a machine that is also a laptop. They are not a benchmark. They are consistent with 30× slower small-file writes going away, which is the claim, and nothing more.</p><p><strong>Two runners on one engine is still one engine.</strong> runner-light moves the image and publish jobs off the Gradle queue; it does not add CPUs. Pitest and Testcontainers still contend for the same 10, which is why the same PR added <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/com/ecommerce/build/HeavyTaskLimit.kt">HeavyTaskLimit</a>, a Gradle build service that sizes the number of concurrent test and Pitest JVMs from the memory the build sees. The comment in that file is the diary again: &quot;Six GiB still left the CI runner&#39;s Gradle daemon (3 to 4 GB heap) OOM-killed under its 10 GB cap with three heavy tasks.&quot;</p><p><strong>The Mac-shaped assumptions were the agent’s as much as mine.</strong> The original runner scripts were written by the same kind of session that fixed them. A goal that says “make it work on the VM” found the boundary bugs; nothing in the harness from Part 2 could have, because every gate ran on the laptop where the assumptions held. A different machine is a sensor, and this repository had only one.</p><p><strong>And no spec again.</strong> Two spec-less PRs in a row. The specs/ folder now describes a CI runner that does not exist. I have argued in Parts 1 and 4 that this is the right trade for tuning and fixing. I have not argued, because I cannot, that the specs will be updated.</p><h3>What I would keep</h3><ol><li><strong>Size from the engine, not the host.</strong> docker info tells you the truth about the machine that will run the job.</li><li><strong>Never put a work directory on a shared folder.</strong> 31× is not a tuning problem.</li><li><strong>Exclude </strong><strong>build.time.</strong> One line in the Boot convention plugin; every downstream task becomes cacheable.</li><li><strong>Keep caches where the data already is.</strong> On a self-hosted runner, the GitHub cache service is a network round trip to yourself.</li><li><strong>Gate on the condition the runner makes true, not the one hosted runners happened to make true.</strong> Verify your own pacts before you ask the broker.</li><li><strong>Write the safeguards down, as a list, with the exit condition.</strong> “If this cannot be met, don’t.”</li><li><strong>A </strong><strong>--print-config flag.</strong> For every script that infers its environment.</li></ol><h3>Go deeper</h3><p>The rest of the series, one piece at a time:</p><ul><li><strong>Part 1 — Seven specs, 620 tasks.</strong> Spec-driven development tested on a real microservices platform: the loop, clarify, converge, and the data nobody has. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Read Part 1 →</a></li><li><strong>Part 2 — The Stop hook said no 95 times.</strong> Harness engineering: three gates, a loop guard, and ten days of real timing data. <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Read Part 2 →</a></li><li><strong>Part 3–3,380 bugs on purpose.</strong> Mutation testing as the merge criterion: Pitest and Stryker at 80 %, survivors justified in the PR. <a href="https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8">Read Part 3 →</a></li><li><strong>Part 4 — A number instead of a spec.</strong> /goal for performance: JDK 25 AOT cache, compact object headers, native images, 4.3 GiB to 1.9. <a href="https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66">Read Part 4 →</a></li><li><strong>Part 6 — A constitution for an AI agent.</strong> Eight principles, 55 MUSTs, a check that runs twice per feature, and the PASS that turned out to be false. <a href="https://medium.com/@raphaelpanta/i-wrote-a-constitution-for-an-ai-agent-heres-what-it-could-and-couldn-t-enforce-da0e48c3b968">Read Part 6 →</a></li></ul><h3>Further reading</h3><ul><li>GitHub Docs, <em>Security hardening for GitHub Actions</em> — <a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions">hardening for self-hosted runners</a>; <em>Autoscaling with self-hosted runners</em>, ephemeral runners — <a href="https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/autoscaling-with-self-hosted-runners">docs.github.com</a>; <em>Caching dependencies to speed up workflows</em> — <a href="https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows">docs.github.com</a></li><li>Spring Boot Gradle Plugin, <em>Integrating with Actuator</em> (generating build information) — <a href="https://docs.spring.io/spring-boot/gradle-plugin/integrating-with-actuator.html">docs.spring.io</a></li><li>Gradle, <em>Build Cache Concepts</em> (stable task inputs) — <a href="https://docs.gradle.org/current/userguide/build_cache_concepts.html">docs.gradle.org</a>; <em>Configuration Cache</em> — <a href="https://docs.gradle.org/current/userguide/configuration_cache.html">docs.gradle.org</a></li><li>Podman, <em>podman-run</em>, --security-opt label=disable — <a href="https://docs.podman.io/en/latest/markdown/podman-run.1.html">docs.podman.io</a>; <em>podman-machine-init</em> — <a href="https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html">docs.podman.io</a>; Fedora CoreOS — <a href="https://docs.fedoraproject.org/en-US/fedora-coreos/">docs.fedoraproject.org</a></li><li>Pact, <em>can-i-deploy</em> — <a href="https://docs.pact.io/pact_broker/can_i_deploy">docs.pact.io</a></li><li>npm, npm ci and --prefer-offline — <a href="https://docs.npmjs.com/cli/v11/commands/npm-ci">docs.npmjs.com</a></li><li>Leslie Lamport, email to DEC SRC, 28 May 1987 — <a href="https://lamport.azurewebsites.net/pubs/distributed-system.txt">lamport.azurewebsites.net/pubs/distributed-system.txt</a></li><li>The repository: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">PR #4</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/README.md">platform/ci-runner/README.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/docker-compose.yml">docker-compose.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/ci-runner/scripts/run-ephemeral.sh">run-ephemeral.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/ci-cache.sh">ci-cache.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-boot-app.gradle.kts">kotlin-boot-app.gradle.kts</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/com/ecommerce/build/HeavyTaskLimit.kt">HeavyTaskLimit.kt</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/service-ci.yml">service-ci.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/gateway/src/contractTest/kotlin/com/ecommerce/gateway/GatewayBrokerVerificationTest.kt">GatewayBrokerVerificationTest.kt</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/ci-cd.md">docs/ci-cd.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">specs/004 research.md §4</a></li></ul><p><em>Numbers are from the description of PR #4 and from </em><em>platform/ci-runner/README.md at commit </em><em>9870792. The runner host at the time was the development Mac&#39;s Podman machine (Fedora CoreOS under libkrun, 10 CPUs, 14 GiB).</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5eece22825ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dei ao agente um número em vez de uma spec. Ele cortou a memória da plataforma de 4,3 GiB para 1,9.]]></title>
            <link>https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ce25059da5e</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[kotlin]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 12:09:45 GMT</pubDate>
            <atom:updated>2026-10-16T14:09:38.127Z</atom:updated>
            <content:encoded><![CDATA[<p><em>O `/goal` do Claude Code numa plataforma de microsserviços em Kotlin: AOT cache do JDK 25, compact object headers, imagens nativas GraalVM e uma flag de DNS que nunca tinha funcionado.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*xPA4vEsdaUK7atVPtyeM4w.png" /></figure><blockquote><em>“Devemos esquecer as pequenas eficiências, digamos em 97 % das vezes: otimização prematura é a raiz de todo mal.” — Donald Knuth, </em>Structured Programming with go to Statements<em>, Computing Surveys (1974)</em></blockquote><p>Todo mundo cita essa frase. Poucos citam a que Knuth escreveu logo ao lado: “No entanto, não devemos desperdiçar nossas oportunidades naqueles 3 % críticos.” Três partes de série depois, a plataforma funcionava. Seis serviços, um gateway, uma loja, Kafka, seis bancos, 21 containers, todos os cenários de aceitação verdes. Ela também precisava de 8,3 GiB de memória reservada para rodar e chegava a 4,3 GiB de pico num teste de carga de dois minutos. Os 3 % críticos tinham chegado.</p><p>A <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Parte 1</a> foi sobre specs, a <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Parte 2</a> sobre hooks, a <a href="https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781">Parte 3</a> sobre mutation testing. Esta parte é sobre a primeira mudança que não teve nada disso. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">pull request #3</a> abre com: “Não há spec: isto é tuning de plataforma.” Ele não mudou nenhum código de aplicação, tocou 27 arquivos e terminou com a stack chegando a 3,1 GiB na JVM e 1,9 GiB em imagens nativas, com throughput 18 % maior.</p><p>Foi executado como um goal, não como uma feature. Eis a diferença, e o que ele encontrou.</p><p><em>As citações de fontes em inglês estão em tradução livre; os links levam ao original.</em></p><h3>Uma spec diz o que construir; um goal diz quando parar</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*F4MdlVIkeVIukc5RKsc77w.png" /></figure><p>Spec-driven development combina com comportamento novo: muitas decisões, muitos arquivos, uma definição de pronto que é uma lista. “Faça este número diminuir sem fazer aquele aumentar” tem outro formato. Há um único estado final mensurável, uma checagem que o próprio agente pode rodar e restrições que precisam se manter no caminho. O Claude Code tem um comando para esse formato (<a href="https://code.claude.com/docs/en/goal">Keep Claude working toward a goal</a>):</p><blockquote><em>O comando </em><em>/goal define uma condição de conclusão e o Claude continua trabalhando até alcançá-la sem que você precise pedir cada passo. Depois de cada turno, um modelo verifica se a condição é satisfeita. Se o modelo julgar que ainda não foi, o Claude inicia outro turno em vez de devolver o controle a você.</em></blockquote><p>Por baixo, ele é “um wrapper em torno de um Stop hook baseado em prompt com escopo de sessão”, o que faz dele um primo do hook de Stop da Parte 2, com uma diferença que a documentação faz questão de destacar: “a conclusão é decidida por um modelo novo, não pelo que está fazendo o trabalho.” O avaliador não pode rodar comandos; ele lê o que o agente deixou visível na transcrição. Então o conselho da documentação sobre como escrever uma condição é, na verdade, um conselho sobre tornar o trabalho legível: um estado final mensurável, uma checagem declarada e as restrições que importam.</p><p>Para esta execução o formato foi: <em>o pico de memória da stack core sob o perfil do k6 igual ou abaixo de um alvo, com todos os thresholds do k6 passando e o subconjunto rápido da suíte de aceitação verde.</em> A restrição é a metade importante. Sem ela, o jeito mais rápido de cortar memória é deixar tudo mais lento, e, como você vai ver, o agente encontrou três formas de fazer exatamente isso e rejeitou cada uma porque a checagem de throughput falhou.</p><h3>O que foi medido</h3><p>A régua foi fixada antes de qualquer mudança: <a href="https://grafana.com/docs/k6/latest/">k6</a> dentro da rede da stack, 200 usuários virtuais navegando e 20 fazendo checkout, 30 s de rampa, 2 min de duração, o dataset de 10.000 produtos, docker stats amostrado a cada 5 s. Tudo em platform/docker/README.md, na seção <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/README.md">&quot;Memory measurements&quot;</a>, vem desse perfil, na mesma máquina (macOS, Podman com libkrun, 8 CPUs, 13 GiB, recém-reiniciada).</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*A6k56_9aTLzf5CaHFqtaRA.png" /></figure><ul><li><strong>Antes (2 execuções)</strong> — 4.321–4.374 MiB · 155–157 req/s · 955–958 ms · browse_list p95 falhando a 992–1.036 ms</li><li><strong>Depois, JVM padrão (4 execuções)</strong> — 3.139–3.173 MiB · 182–185 req/s · 73–233 ms · todos passam</li><li><strong>Depois, imagens nativas</strong> — 1.867 MiB · 184 req/s · 104 ms · todos passam</li></ul><p>Os limites dos containers foram de 8,3 GiB reservados para 5,4 GiB (3,4 GiB com imagens nativas). O próprio PR pede para “ler as colunas de throughput como ‘sem regressão’”, porque o throughput naquela máquina varia entre execuções mais do que a memória. Justo. A memória é estável em cerca de 10 %, e caiu um quarto na JVM e mais da metade em nativo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*gAdMtPwEUdYdyHO1WuU-Pw.png" /></figure><p>Por container, o maior ganho isolado não foi uma flag da JVM: trocar o broker Kafka em JVM pelo <a href="https://hub.docker.com/r/apache/kafka-native">apache/kafka-native</a> o levou de 803 MiB para 337. Entre os serviços, identity foi o que menos encolheu na JVM, porque o hash de senha Argon2id enche o heap por design, o que também explica por que identity e catalog acabaram com limites de 640m em vez de 512m depois de uma execução fora da curva em que catalog travou por dez segundos no seu limite.</p><h3>Para onde vai a memória de um container JVM</h3><p>O README tem um diagnóstico que eu queria que todo Dockerfile trouxesse. Um serviço sozinho, catalog, 1 CPU, 768 MiB, sob carga: o heap committed era só 70 a 100 MiB do seu teto de 307 MiB. O resto era metaspace (cerca de 85 MiB), código compilado pelo JIT (cerca de 65 MiB) e alocações nativas. O heap não era o problema. Tudo em volta dele era.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*5bdyJAFgtYoGO3DhtSgYmg.png" /></figure><p>Então as flags miram o tudo-em-volta. Cada uma está em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile">JAVA_TOOL_OPTIONS</a> com um motivo de uma linha no Dockerfile; aqui vão os motivos com suas fontes.</p><p><strong>-XX:+UseSerialGC.</strong> O <a href="https://docs.oracle.com/en/java/javase/25/gctuning/available-collectors.html">guia de tuning de GC</a> da Oracle: o coletor serial &quot;é o mais adequado para máquinas com um único processador&quot; e para &quot;aplicações com conjuntos de dados pequenos (até aproximadamente 100 MB)&quot;. Um heap vivo de 70 MiB num container de uma CPU é esse caso. A flag é explícita porque a JVM muda para G1 no instante em que um pod ganha duas CPUs, e o G1 traz remembered sets e threads de que o serviço não precisa.</p><p><strong>-XX:+UseCompactObjectHeaders.</strong> O <a href="https://openjdk.org/jeps/519">JEP 519</a>, JDK 25, torna pronto para produção o layout que o <a href="https://openjdk.org/jeps/450">JEP 450</a> introduziu como experimental: cabeçalhos de objeto vão &quot;de entre 96 e 128 bits para 64 bits em arquiteturas de 64 bits&quot;. A <a href="https://docs.oracle.com/en/java/javase/25/docs/specs/man/java.html">página de manual do </a><a href="https://docs.oracle.com/en/java/javase/25/docs/specs/man/java.html">java</a> diz que isso &quot;reduz o footprint de memória no heap Java em 4 bytes por objeto (em média) e frequentemente melhora a performance&quot;; o JEP reporta o SPECjbb2015 usando &quot;22 % menos espaço de heap e 8 % menos tempo de CPU&quot;. O número do próprio repositório: 10 a 20 % menos heap para os mesmos objetos.</p><p><strong>-XX:MaxRAMPercentage=50.</strong> Teto do heap como fração do limite do container: 256 MiB de 512m. Era 40 % de 768m antes; com a memória fora do heap caindo de 300–400 MB para 130–180 MiB, o heap podia ficar com uma fatia maior. 40 % a 512m foi medido e perdeu 10 % de throughput em coletas no caminho de navegação.</p><p><strong>-Xss1m.</strong> Pilhas de thread de 1 MiB em vez dos 2 MiB padrão em aarch64. Netty e os dispatchers de corrotinas não precisam do padrão.</p><p><strong>-XX:MaxDirectMemorySize=128m e </strong><strong>-Djdk.nio.maxCachedBufferSize=262144.</strong> Os buffers diretos em pool do Netty têm como padrão o tamanho do heap; o segundo limita os buffers temporários por thread que o NIO bloqueante (o JDBC do Flyway, o cliente Kafka) aloca.</p><p><strong>-XX:TrimNativeHeapInterval=30000 com </strong><strong>MALLOC_ARENA_MAX=2.</strong> O primeiro é um recurso do JDK 22 (<a href="https://bugs.openjdk.org/browse/JDK-8293114">JDK-8293114</a>): a página de manual diz que a JVM &quot;vai aparar o heap nativo&quot; nesse intervalo, &quot;suportado apenas em Linux com a GNU C Library (glibc)&quot;, devolvendo ao SO a memória liberada pelo malloc. O segundo é um ajuste da própria glibc (<a href="https://man7.org/linux/man-pages/man3/mallopt.3.html">mallopt(3)</a>): duas arenas de malloc em vez de oito por CPU do <em>host</em>, que num container é o número errado por uma margem larga.</p><h3>A flag que não fazia nada</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*iXWWkZumSvRaPakcENXeRw.png" /></figure><p>E tem a que não é sobre memória.</p><p>A descoberta de serviços nesta plataforma é DNS: nomes de serviço do Compose agora, Services do Kubernetes depois. A pesquisa da Feature 004 decidiu que uma réplica que desaparece precisa sair rápido do cache de endereços da JVM, e o FR-024 exigiu um TTL de 5 segundos. O Dockerfile tinha -Dnetworkaddress.cache.ttl=5 desde a primeira imagem. Nunca tinha feito nada.</p><p>A página de <a href="https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/net/doc-files/net-properties.html">propriedades de rede</a> da Oracle explica por quê, numa frase fácil de passar batido: as três propriedades networkaddress.cache.* &quot;não são definidas nem pela opção -D nem pela API System.setProperty(); em vez disso, são definidas como propriedades de segurança.&quot; Elas moram no java.security. Um -D é uma propriedade de sistema. O próprio <a href="https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/sun/net/InetAddressCachePolicy.java">InetAddressCachePolicy</a> do JDK lê primeiro a propriedade de segurança e depois cai para uma propriedade de sistema com outro nome, mais antigo:</p><pre>private static final String cachePolicyProp = &quot;networkaddress.cache.ttl&quot;;<br>private static final String cachePolicyPropFallback = &quot;sun.net.inetaddr.ttl&quot;;<br>public static final int DEFAULT_POSITIVE = 30;</pre><p>Trinta segundos, o tempo todo. A correção é -Dsun.net.inetaddr.ttl=5. Nenhum teste jamais tinha checado isso, porque a suíte de aceitação roda contra uma stack saudável onde nada desaparece.</p><p>Incluo isso não porque economizou memória (não economizou), mas por como foi encontrado. Uma execução guiada por goal lê tudo no caminho até o seu número. O agente auditou cada flag do Dockerfile contra o que a flag faz, e esta estava errada. Uma spec não teria perguntado.</p><h3>O AOT cache</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*4w9iHgoJ2Ohx_ZGJcuy8Sw.png" /></figure><p>A maior economia do lado da JVM foi em metaspace, e a ferramenta para isso é nova. O <a href="https://openjdk.org/jeps/483">JEP 483</a> (JDK 24) introduziu o AOT cache: a JVM roda uma vez em modo de treino, “armazenando as formas carregadas e linkadas de todas as classes num cache”, para que nas partidas seguintes essas classes estejam “instantaneamente disponíveis, em estado carregado e linkado”. O <a href="https://openjdk.org/jeps/514">JEP 514</a> (JDK 25) reduziu os dois passos de gravar e criar a uma única flag, -XX:AOTCacheOutput, e o <a href="https://openjdk.org/jeps/515">JEP 515</a> acrescentou perfis de execução de métodos ao cache, para que o aquecimento também seja mais curto.</p><p>O Spring torna a execução de treino prática. A referência <a href="https://docs.spring.io/spring-framework/reference/integration/cds.html">JVM AOT Cache</a> descreve -Dspring.context.exit=onRefresh: o processo encerra &quot;assim que o ApplicationContext for atualizado&quot;, depois de todos os singletons instanciados, mas antes de o ciclo de vida começar. Então a execução de treino não precisa de banco, broker nem collector; Flyway e os listeners Kafka são desligados por ambiente, e os segredos sem os quais um serviço se recusa a subir são valores descartáveis montados com bind só naquele RUN, nunca copiados para uma camada.</p><p>Dois detalhes no Dockerfile são do tipo que só se aprende fazendo. Primeiro, a imagem usa o layout simples de java -jar em vez do JarLauncher do Spring Boot, porque um AOT cache só pode conter classes carregadas de arquivos jar. Segundo, um cache só é válido para o JRE e as opções de JVM que o gravaram, então o passo de treino falha o build se app.aot estiver faltando, em vez de embarcar uma imagem cuja JVM logaria erros de AOT a cada partida e rodaria sem ele em silêncio.</p><p>Resultado: metaspace de cerca de 80 para 30 MiB por serviço, dados de classe compartilhados e mapeados de arquivo (o kernel pode descartá-los sob pressão), partida mais ou menos pela metade, imagens cerca de 100 MB maiores em disco. A documentação do Spring acrescenta uma ressalva que vale conhecer: com os perfis de método do JDK 25, uma execução de treino que exercite uma carga parecida com produção seria melhor do que uma que só chega ao refresh. Este repositório fica com a simples.</p><h3>Imagens nativas, e três bugs que o mundo fechado encontrou</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*S83G63lwEYmpnQgHSgCzSQ.png" /></figure><p>O caminho opcional é o GraalVM Native Image por meio de um segundo Dockerfile e um segundo arquivo Compose. Os números impressionam (catalog sozinho: 115 MiB de memória anônima contra 279 com o AOT cache; pronto em 1 s contra 25), e os custos também: 6 a 11 minutos de build por serviço e um pico de 7,5 GB no builder.</p><p>O Native Image trabalha num mundo fechado: a análise estática em tempo de build decide o que existe. Qualquer coisa alcançada só por reflexão ou por busca de recurso precisa ser declarada em <a href="https://www.graalvm.org/latest/reference-manual/native-image/metadata/">reachability metadata</a>, que o Native Image “lê … localizado nas entradas do classpath” como reachability-metadata.json. Três coisas nesta plataforma só eram alcançáveis desse jeito:</p><ul><li><strong>O Flyway</strong> não encontrou migrações em db/messaging e db/seed: buscas de recurso por padrão são invisíveis para a análise. -H:IncludeResources=db/.* as declara.</li><li><strong>/actuator/prometheus travava</strong>: o Micrometer alcançava uma classe por reflexão. Uma entrada em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/native/metadata/reachability-metadata.json">reachability-metadata.json</a>.</li><li><strong>O próprio builder precisava de um teto de heap.</strong> As três correções são opções de build, aplicadas por um <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/native/native-image.init.gradle.kts">init script do Gradle</a> que só o build Docker usa, então os arquivos de build normais do repositório não mudam.</li></ul><p>Os mesmos 95 cenários de aceitação passaram contra a stack nativa e contra a JVM. Essa frase está no checklist de constituição do PR, e é a frase que tornou o opcional aceitável.</p><h3>Medido e rejeitado</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*C_cZ2dDmcbMjLqWCUrjr1Q.png" /></figure><p>Esta é a parte que me convenceu de que o goal foi escrito corretamente. Cinco candidatos economizaram memória, foram medidos sob o mesmo perfil e perderam na restrição:</p><ul><li><strong>JRE Alpine (musl)</strong>: a mesma memória com diferença de poucos MiB, cerca de 20 % menos throughput (120–138 req/s a um p95 de 1,3–1,7 s). Alpine fica onde não custa nada: PostgreSQL, o broker nativo, nginx.</li><li><strong>JIT só com C1</strong> (-XX:TieredStopAtLevel=1): 60–90 MiB a menos por serviço e partida pela metade; cerca de 30 % menos throughput (112 req/s, p95 de 1,7 s). Mantido disponível por container via JDK_JAVA_OPTIONS para quando memória importar mais que velocidade.</li><li><strong>-XX:MaxHeapFreeRatio=30</strong>: cerca de 10 % de throughput por pouca memória.</li><li><strong>Heap de 40 % a 512m</strong>: 149 req/s a um p95 de 1,15 s.</li><li><strong>-H:+CompactingOldGen nativo</strong>: sem ganho.</li></ul><p>Cada um desses é uma mudança que um agente com a instrução “reduza a memória” teria mantido. A restrição de throughput no goal é o que fez deles derrotas. Esse é o argumento inteiro para escrever a restrição.</p><h3>O que uma execução por goal deixa para trás</h3><p>Uma execução por spec deixa specs/NNN/. Esta deixou uma seção de 186 linhas num README, uma descrição de PR com tabela de resultados, uma lista de &quot;medido e rejeitado&quot;, um checklist de constituição com uma caixa honestamente desmarcada (&quot;Não se aplica: esta mudança não tem spec de feature nem plan.md&quot;) e 797 linhas de diff entre Dockerfiles, arquivos Compose, scripts de CI e docs. É um tipo diferente de registro, mais perto de um caderno de laboratório do que de um documento de requisitos, e para esse tipo de trabalho eu prefiro assim.</p><h3>O caso contra, e o que os números não dizem</h3><p><strong>Uma máquina, um perfil, sem barras de erro.</strong> Todo número aqui vem de um MacBook rodando Podman numa VM. O throughput variou tanto entre execuções que o PR pede para lê-lo como “sem regressão” e não como “+18 %”. Duas execuções “antes”, quatro “depois”: o bastante para ver um quarto da memória ir embora, não o bastante para um intervalo de confiança. A execução fora da curva com catalog travando está no README porque aconteceu, e um outlier em seis é uma taxa que vale lembrar.</p><p><strong>O goal mede o que você mandou medir.</strong> Pico de memória do cgroup, que inclui page cache. O “pico” de um serviço nativo é em grande parte o seu executável mapeado, que o kernel recupera antes do limite; sua memória anônima é de 60–120 MiB. O README diz isso. Uma condição de goal escrita como “pico do docker stats” teria sido burlada por essa distinção sem que ninguém pretendesse burlar.</p><p><strong>O avaliador lê a transcrição, não a máquina.</strong> A documentação é explícita: ele &quot;não roda comandos nem lê arquivos por conta própria&quot;. Se o agente reporta uma execução de k6 que não aconteceu como descrita, o avaliador não tem como saber. A defesa é a mesma da Parte 3: um humano lê o PR. Eu li; as tabelas do README foram a minha checagem.</p><p><strong>Imagens nativas são opcionais por um motivo.</strong> 6–11 minutos por serviço, 7,5 GB por build, o Spring AOT fixando beans @Profile e @Conditional em tempo de build, reflexão fora de com.ecommerce.* precisando de metadata. As imagens JVM são o padrão porque as nativas trocam um quarto da memória por um build que você não consegue rodar no notebook enquanto faz qualquer outra coisa.</p><p><strong>E não há comparação.</strong> Eu não executei o mesmo tuning como spec. Minha convicção de que um goal cabe melhor nesse trabalho do que uma spec é uma opinião sobre formato, sustentada por uma única execução. Os PRs sem spec também são, como a Parte 1 admitiu, o primeiro desvio de specs/ descrevendo o sistema, e nada no harness percebe.</p><h3>O que eu manteria</h3><ol><li><strong>Fixar a régua antes de tocar em qualquer coisa.</strong> Mesmo perfil de k6, mesma máquina, mesma amostragem, antes e depois.</li><li><strong>Colocar a restrição no goal.</strong> &quot;Menos memória&quot; sozinho encontra Alpine e C1. &quot;Menos memória, throughput não pior, aceitação verde&quot; encontra o AOT cache.</li><li><strong>Auditar cada flag contra a sua documentação.</strong> Uma delas tinha sido um no-op por uma semana.</li><li><strong>Registrar o que foi rejeitado, com números.</strong> A lista de rejeitados vale mais para a próxima pessoa do que a lista de mantidos.</li><li><strong>Treinar o AOT cache no Dockerfile e falhar o build se ele vier vazio.</strong> Nunca embarcar uma JVM que loga erros de AOT na partida.</li><li><strong>Manter nativo como opcional até o build caber na máquina.</strong></li><li><strong>Dizer quando não há spec.</strong> O primeiro parágrafo do PR disse.</li></ol><h3>Para se aprofundar</h3><p>O resto da série, um por vez:</p><ul><li><strong>Parte 1 — Sete specs, 620 tarefas.</strong> Spec-driven development testado numa plataforma de microsserviços real: o loop, clarify, converge e os dados que ninguém tem. <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Leia a Parte 1 →</a></li><li><strong>Parte 2 — O hook de Stop disse não 95 vezes.</strong> Harness engineering: três gates, um guarda de loop e dez dias de dados reais de tempo. <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Leia a Parte 2 →</a></li><li><strong>Parte 3 — 3.380 bugs de propósito.</strong> Mutation testing como critério de merge: Pitest e Stryker a 80 %, sobreviventes justificados no PR. <a href="https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781">Leia a Parte 3 →</a></li><li><strong>Parte 5 — O Mac estava mentindo.</strong> /goal para correção: SELinux, virtiofs, um timestamp de build e um runner self-hosted que finalmente funcionou no Linux. <a href="https://medium.com/@raphaelpanta/meu-runner-de-ci-funcionava-no-meu-mac-o-mac-estava-mentindo-02e8e8ac6a9c">Leia a Parte 5 →</a></li><li><strong>Parte 6 — Uma constituição para um agente de IA.</strong> Oito princípios, 55 MUSTs, uma checagem que roda duas vezes por feature e o PASS que acabou sendo falso. <a href="https://medium.com/@raphaelpanta/escrevi-uma-constitui%C3%A7%C3%A3o-para-um-agente-de-ia-eis-o-que-ela-conseguiu-e-o-que-n%C3%A3o-conseguiu-impor-31332eadc192">Leia a Parte 6 →</a></li></ul><h3>Leitura adicional</h3><ul><li><em>Keep Claude working toward a goal</em> — <a href="https://code.claude.com/docs/en/goal">code.claude.com/docs/en/goal</a></li><li>JEP 483, <em>Ahead-of-Time Class Loading &amp; Linking</em> (JDK 24) — <a href="https://openjdk.org/jeps/483">openjdk.org/jeps/483</a>; JEP 514, <em>Ahead-of-Time Command-Line Ergonomics</em> (JDK 25) — <a href="https://openjdk.org/jeps/514">openjdk.org/jeps/514</a>; JEP 515, <em>Ahead-of-Time Method Profiling</em> (JDK 25) — <a href="https://openjdk.org/jeps/515">openjdk.org/jeps/515</a></li><li>JEP 450, <em>Compact Object Headers (Experimental)</em> (JDK 24) — <a href="https://openjdk.org/jeps/450">openjdk.org/jeps/450</a>; JEP 519, <em>Compact Object Headers</em> (JDK 25) — <a href="https://openjdk.org/jeps/519">openjdk.org/jeps/519</a></li><li>Oracle, <em>The java Command</em> (JDK 25) — <a href="https://docs.oracle.com/en/java/javase/25/docs/specs/man/java.html">docs.oracle.com</a>; <em>Available Collectors</em>, HotSpot GC Tuning Guide — <a href="https://docs.oracle.com/en/java/javase/25/gctuning/available-collectors.html">docs.oracle.com</a>; <em>Networking Properties</em> — <a href="https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/net/doc-files/net-properties.html">docs.oracle.com</a></li><li>OpenJDK, <a href="https://bugs.openjdk.org/browse/JDK-8293114">JDK-8293114: JVM should trim the native heap</a>; <a href="https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/sun/net/InetAddressCachePolicy.java">InetAddressCachePolicy.java</a></li><li>glibc, <a href="https://man7.org/linux/man-pages/man3/mallopt.3.html">mallopt(3)</a>, M_ARENA_MAX / MALLOC_ARENA_MAX</li><li>Spring Framework, <em>JVM AOT Cache</em> — <a href="https://docs.spring.io/spring-framework/reference/integration/cds.html">docs.spring.io</a></li><li>GraalVM, <em>Reachability Metadata</em> — <a href="https://www.graalvm.org/latest/reference-manual/native-image/metadata/">graalvm.org</a>; <em>Build Options</em> — <a href="https://www.graalvm.org/latest/reference-manual/native-image/overview/BuildOptions/">graalvm.org</a></li><li>Imagem nativa do Apache Kafka — <a href="https://hub.docker.com/r/apache/kafka-native">hub.docker.com/r/apache/kafka-native</a>; Grafana k6 — <a href="https://grafana.com/docs/k6/latest/">grafana.com/docs/k6</a></li><li>Donald E. Knuth, <em>Structured Programming with go to Statements</em>, ACM Computing Surveys 6(4), dezembro de 1974 — <a href="https://www.gwern.net/doc/cs/algorithm/1974-knuth.pdf">PDF</a></li><li>O repositório: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">PR #3</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile">platform/docker/Dockerfile</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile.native">Dockerfile.native</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/README.md">platform/docker/README.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/compose/compose.native.yml">compose.native.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/compose/docker-compose.yml">docker-compose.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/platform/perf">platform/perf</a></li></ul><p><em>Todas as medições vêm de </em><em>platform/docker/README.md e da descrição do PR #3, feitas em 2026-10-08 na máquina de desenvolvimento. Não as refiz para este artigo; os números são do repositório, e o repositório diz como foram obtidos.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ce25059da5e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Gave the Agent a Number Instead of a Spec. It Cut the Platform’s Memory From 4.3 GiB to 1.9.]]></title>
            <link>https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/ceb2f0b4da66</guid>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 12:08:41 GMT</pubDate>
            <atom:updated>2026-10-16T14:06:43.689Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Claude Code’s `/goal` on a Kotlin microservices platform: JDK 25 AOT caches, compact object headers, GraalVM native images, and a DNS flag that had never worked.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*eWbS9fzpfGTwteXJv5PEiw.png" /></figure><blockquote><em>“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.” — Donald Knuth, </em>Structured Programming with go to Statements<em>, Computing Surveys (1974)</em></blockquote><p>Everyone quotes that line. Fewer quote the sentence Knuth wrote next to it: “Yet we should not pass up our opportunities in that critical 3 %.” Three parts into this series, the platform worked. Six services, a gateway, a storefront, Kafka, six databases, 21 containers, every acceptance scenario green. It also needed 8.3 GiB of reserved memory to run and peaked at 4.3 GiB under a two-minute load test. The critical 3 % had arrived.</p><p><a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Part 1</a> was specs, <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Part 2</a> hooks, <a href="https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8">Part 3</a> mutation testing. This part is about the first change that had none of those. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">Pull request #3</a> opens with: “There is no spec: this is platform tuning.” It changed no application code, touched 27 files, and ended with the stack peaking at 3.1 GiB on the JVM and 1.9 GiB on native images, with throughput up 18 %.</p><p>It was run as a goal, not a feature. Here is the difference, and what it found.</p><h3>A spec says what to build; a goal says when to stop</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*vnUFzeygMRfgGg8Bcpb4yw.png" /></figure><p>Spec-driven development fits new behaviour: many decisions, many files, a definition of done that is a list. “Make this number smaller without making that number bigger” is a different shape. There is one measurable end state, a check the agent can run itself, and constraints that must hold on the way. Claude Code has a command for that shape (<a href="https://code.claude.com/docs/en/goal">Keep Claude working toward a goal</a>):</p><blockquote><em>The </em><em>/goal command sets a completion condition and Claude keeps working toward it without you prompting each step. After each turn, a model checks whether the condition holds. If the model judges it not yet met, Claude starts another turn instead of returning control to you.</em></blockquote><p>Under the hood it is “a wrapper around a session-scoped prompt-based Stop hook”, which makes it a cousin of the Stop hook from Part 2, with one difference the docs are careful about: “completion is decided by a fresh model rather than the one doing the work.” The evaluator cannot run commands; it reads what the agent surfaced in the transcript. So the docs’ advice on writing a condition is really advice on making the work legible: one measurable end state, a stated check, and the constraints that matter.</p><p>For this run the shape was: <em>the core stack’s peak memory under the k6 profile at or below a target, with every k6 threshold passing and the acceptance suite’s fast subset green.</em> The constraint is the important half. Without it, the fastest way to cut memory is to make everything slower, and as you will see, the agent found three ways to do exactly that and rejected each one because the throughput check failed.</p><h3>What was measured</h3><p>The measuring stick was fixed before anything changed: <a href="https://grafana.com/docs/k6/latest/">k6</a> inside the stack’s network, 200 browsing and 20 checkout virtual users, 30 s ramp, 2 min duration, the 10,000-product dataset, docker stats sampled every 5 s. Everything in platform/docker/README.md under <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/README.md">&quot;Memory measurements&quot;</a> comes from that profile, on the same machine (macOS, Podman with libkrun, 8 CPUs, 13 GiB, freshly restarted).</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*AHYMHCi2aMVnC3CcxULp_Q.png" /></figure><ul><li><strong>Before (2 runs)</strong> — 4,321–4,374 MiB · 155–157 req/s · 955–958 ms · browse_list p95 failing at 992–1,036 ms</li><li><strong>After, JVM default (4 runs)</strong> — 3,139–3,173 MiB · 182–185 req/s · 73–233 ms · all pass</li><li><strong>After, native images</strong> — 1,867 MiB · 184 req/s · 104 ms · all pass</li></ul><p>Container limits went from 8.3 GiB reserved to 5.4 GiB (3.4 GiB with native images). The PR itself says to “read the throughput columns as ‘no regression’”, because throughput on that machine varies between runs more than memory does. Fair. Memory is stable to about 10 %, and it went down by a quarter on the JVM and by more than half native.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*BlPQ7z827PJ-NTo41m7mAg.png" /></figure><p>Per container, the biggest single win was not a JVM flag at all: replacing the JVM Kafka broker with <a href="https://hub.docker.com/r/apache/kafka-native">apache/kafka-native</a> took it from 803 MiB to 337. Among the services, identity shrank least on the JVM, because Argon2id password hashing fills its heap by design, which is also why identity and catalog ended up with 640m limits instead of 512m after one outlier run in which catalog stalled for ten seconds at its limit.</p><h3>Where a JVM container’s memory goes</h3><p>The README has a diagnostic I wish every Dockerfile came with. One service alone, catalog, 1 CPU, 768 MiB, under load: heap committed was only 70 to 100 MiB of its 307 MiB ceiling. The rest was metaspace (about 85 MiB), JIT-compiled code (about 65 MiB) and native allocations. The heap was not the problem. Everything around it was.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*tk_YmonKyQ5Euhb9awi7vw.png" /></figure><p>So the flags target the everything-around-it. Each is in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile">JAVA_TOOL_OPTIONS</a> with a one-line reason in the Dockerfile; here are the reasons with their sources.</p><p><strong>-XX:+UseSerialGC.</strong> Oracle&#39;s <a href="https://docs.oracle.com/en/java/javase/25/gctuning/available-collectors.html">GC tuning guide</a>: the serial collector &quot;is best-suited to single processor machines&quot; and for &quot;applications with small data sets (up to approximately 100 MB)&quot;. A 70 MiB live heap in a one-CPU container is that case. The flag is explicit because the JVM switches to G1 the moment a pod gets two CPUs, and G1 brings remembered sets and threads the service does not need.</p><p><strong>-XX:+UseCompactObjectHeaders.</strong> <a href="https://openjdk.org/jeps/519">JEP 519</a>, JDK 25, makes production-ready the layout <a href="https://openjdk.org/jeps/450">JEP 450</a> introduced experimentally: object headers go &quot;from between 96 and 128 bits down to 64 bits on 64-bit architectures&quot;. The <a href="https://docs.oracle.com/en/java/javase/25/docs/specs/man/java.html">java man page</a> says it &quot;reduces memory footprint in the Java heap by 4 bytes per object (on average) and often improves performance&quot;; the JEP reports SPECjbb2015 using &quot;22% less heap space and 8% less CPU time&quot;. The repository&#39;s own number: 10 to 20 % less heap for the same objects.</p><p><strong>-XX:MaxRAMPercentage=50.</strong> Heap ceiling as a share of the container limit: 256 MiB of 512m. It was 40 % of 768m before; with off-heap memory down from 300–400 MB to 130–180 MiB, the heap could take a larger share. 40 % at 512m was measured and lost 10 % of throughput to collections on the browse path.</p><p><strong>-Xss1m.</strong> Thread stacks of 1 MiB instead of the 2 MiB default on aarch64. Netty and the coroutine dispatchers do not need the default.</p><p><strong>-XX:MaxDirectMemorySize=128m and </strong><strong>-Djdk.nio.maxCachedBufferSize=262144.</strong> Netty&#39;s pooled direct buffers default to the heap size; the second bounds the per-thread temporary buffers that blocking NIO (Flyway&#39;s JDBC, the Kafka client) allocates.</p><p><strong>-XX:TrimNativeHeapInterval=30000 with </strong><strong>MALLOC_ARENA_MAX=2.</strong> The first is a JDK 22 feature (<a href="https://bugs.openjdk.org/browse/JDK-8293114">JDK-8293114</a>): the man page says the JVM &quot;will trim the native heap&quot; at that interval, &quot;only supported on Linux with GNU C Library (glibc)&quot;, returning freed malloc memory to the OS. The second is glibc&#39;s own tunable (<a href="https://man7.org/linux/man-pages/man3/mallopt.3.html">mallopt(3)</a>): two malloc arenas instead of eight per CPU of the <em>host</em>, which in a container is the wrong number by a wide margin.</p><h3>The flag that did nothing</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*V1iqQyvblQwFq2rkjd0-jg.png" /></figure><p>Then there is the one that is not about memory.</p><p>Service discovery in this platform is DNS: Compose service names now, Kubernetes Services later. Feature 004’s research decided that a replica that disappears must leave the JVM’s address cache quickly, and FR-024 required a 5-second TTL. The Dockerfile had -Dnetworkaddress.cache.ttl=5 since the first image. It had never done anything.</p><p>Oracle’s <a href="https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/net/doc-files/net-properties.html">networking properties</a> page explains why, in a sentence that is easy to skim past: the three networkaddress.cache.* properties &quot;are not set by either the -D option or the System.setProperty() API, instead they are set as security properties.&quot; They live in java.security. A -D is a system property. The JDK&#39;s own <a href="https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/sun/net/InetAddressCachePolicy.java">InetAddressCachePolicy</a> reads the security property first and then falls back to a system property with a different, older name:</p><pre>private static final String cachePolicyProp = &quot;networkaddress.cache.ttl&quot;;<br>private static final String cachePolicyPropFallback = &quot;sun.net.inetaddr.ttl&quot;;<br>public static final int DEFAULT_POSITIVE = 30;</pre><p>Thirty seconds, the whole time. The fix is -Dsun.net.inetaddr.ttl=5. No test had ever checked it, because the acceptance suite runs against a healthy stack where nothing disappears.</p><p>I include this not because it saved memory (it did not) but because of how it was found. A goal-driven run reads everything on the way to its number. The agent audited every flag in the Dockerfile against what the flag does, and this one was wrong. A spec would not have asked.</p><h3>The AOT cache</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*WFoEQhV3vuJIvc-aZ06hhQ.png" /></figure><p>The largest JVM-side saving was metaspace, and the tool for it is new. <a href="https://openjdk.org/jeps/483">JEP 483</a> (JDK 24) introduced the AOT cache: the JVM is run once in training mode, “storing the loaded and linked forms of all classes in a cache”, so that on later starts those classes are “instantly available, in a loaded and linked state”. <a href="https://openjdk.org/jeps/514">JEP 514</a> (JDK 25) collapsed the two-step record-and-create into one flag, -XX:AOTCacheOutput, and <a href="https://openjdk.org/jeps/515">JEP 515</a> added method-execution profiles to the cache so warm-up is shorter too.</p><p>Spring makes the training run practical. Its <a href="https://docs.spring.io/spring-framework/reference/integration/cds.html">JVM AOT Cache</a> reference describes -Dspring.context.exit=onRefresh: the process exits &quot;once the ApplicationContext has refreshed&quot;, after all singletons are instantiated but before the lifecycle starts. So the training run needs no database, broker or collector; Flyway and the Kafka listeners are turned off by environment, and the secrets a service refuses to start without are bind-mounted throw-away values for that one RUN step, never copied into a layer.</p><p>Two details in the Dockerfile are the kind you only learn by doing it. First, the image uses the plain java -jar layout rather than Spring Boot&#39;s JarLauncher, because an AOT cache can only hold classes loaded from jar files. Second, a cache is only valid for the JRE and the JVM options that recorded it, so the training step fails the build if app.aot is missing, rather than shipping an image whose JVM would log AOT errors at every start and silently run without it.</p><p>Result: metaspace from about 80 to 30 MiB per service, class data shared and file-backed (the kernel can drop it under pressure), start-up roughly halved, images about 100 MB larger on disk. Spring’s docs add a caveat worth knowing: with JDK 25’s method profiles, a training run that exercises a production-like workload would do better than one that only reaches refresh. This repository takes the simple one.</p><h3>Native images, and three bugs the closed world found</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*JuZSTWLR5KhZKt79cLuLYg.png" /></figure><p>The opt-in path is GraalVM Native Image through a second Dockerfile and a second Compose file. The numbers are striking (catalog alone: 115 MiB anonymous memory against 279 with the AOT cache; ready in 1 s against 25), and so are the costs: 6 to 11 minutes of build per service and a 7.5 GB builder peak.</p><p>Native Image works on a closed world: static analysis at build time decides what exists. Anything reached only by reflection or a resource lookup must be declared in <a href="https://www.graalvm.org/latest/reference-manual/native-image/metadata/">reachability metadata</a>, which Native Image “reads … located in classpath entries” as reachability-metadata.json. Three things in this platform were only reachable that way:</p><ul><li><strong>Flyway</strong> found no migrations in db/messaging and db/seed: resource lookups by pattern are invisible to the analysis. -H:IncludeResources=db/.* declares them.</li><li><strong>/actuator/prometheus hung</strong>: Micrometer reached a class reflectively. One entry in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/native/metadata/reachability-metadata.json">reachability-metadata.json</a>.</li><li><strong>The builder itself needed a heap cap.</strong> All three fixes are build options, applied through a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/native/native-image.init.gradle.kts">Gradle init script</a> that only the Docker build uses, so the repository’s normal build files do not change.</li></ul><p>The same 95 acceptance scenarios passed against the native stack as against the JVM one. That sentence is in the PR’s constitution checklist, and it is the sentence that made the opt-in acceptable.</p><h3>Measured and rejected</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*DevMWvy7NFzxfQGdtTxq8w.png" /></figure><p>This is the part that convinced me the goal was written correctly. Five candidates saved memory, were measured under the same profile, and lost on the constraint:</p><ul><li><strong>Alpine (musl) JRE</strong>: the same memory within a few MiB, about 20 % less throughput (120–138 req/s at a 1.3–1.7 s p95). Alpine stays where it costs nothing: PostgreSQL, the native broker, nginx.</li><li><strong>C1-only JIT</strong> (-XX:TieredStopAtLevel=1): 60–90 MiB less per service and start-up halved; about 30 % less throughput (112 req/s, 1.7 s p95). Kept available per container through JDK_JAVA_OPTIONS for when memory matters more than speed.</li><li><strong>-XX:MaxHeapFreeRatio=30</strong>: about 10 % of throughput for little memory.</li><li><strong>A 40 % heap at 512m</strong>: 149 req/s at a 1.15 s p95.</li><li><strong>Native </strong><strong>-H:+CompactingOldGen</strong>: no gain.</li></ul><p>Each of these is a change an agent with the instruction “reduce memory” would have kept. The throughput constraint in the goal is what made them losses. That is the whole argument for writing the constraint down.</p><h3>What a goal run leaves behind</h3><p>A spec run leaves specs/NNN/. This run left a 186-line section of a README, a PR description with a results table, a &quot;measured and rejected&quot; list, a constitution checklist with one box honestly unticked (&quot;Not applicable: this change has no feature spec or plan.md&quot;), and 797 lines of diff across Dockerfiles, Compose files, CI scripts and docs. It is a different kind of record, closer to a lab notebook than a requirements document, and for this kind of work I prefer it.</p><h3>The case against, and what the numbers can’t tell you</h3><p><strong>One machine, one profile, no error bars.</strong> Every number here comes from a MacBook running Podman in a VM. Throughput varied enough between runs that the PR tells you to read it as “no regression” rather than “+18 %”. Two “before” runs, four “after”: enough to see a quarter of the memory go, not enough for a confidence interval. The outlier run with catalog stalling is in the README because it happened, and one outlier in six is a rate worth remembering.</p><p><strong>The goal measures what you told it to.</strong> Peak cgroup memory, which includes page cache. A native service’s “peak” is mostly its mapped executable, which the kernel reclaims before the limit; its anonymous memory is 60–120 MiB. The README says this. A goal condition written as “docker stats peak” would have been gamed by that distinction without anyone intending to game it.</p><p><strong>The evaluator reads the transcript, not the machine.</strong> The docs are explicit: it “doesn’t run commands or read files independently”. If the agent reports a k6 run that did not happen as it described, the evaluator has no way to know. The defence is the same as Part 3’s: a human reads the PR. I did; the README’s tables were my check.</p><p><strong>Native images are opt-in for a reason.</strong> 6–11 minutes per service, 7.5 GB per build, Spring AOT fixing @Profile and @Conditional beans at build time, reflection outside com.ecommerce.* needing metadata. The JVM images are the default because the native ones trade a quarter of the memory for a build you cannot run on the laptop while doing anything else.</p><p><strong>And there is no comparison.</strong> I did not run the same tuning as a spec. My belief that a goal fits this work better than a spec is an opinion about shape, supported by one run. The spec-less PRs are also, as Part 1 admitted, the first drift away from specs/ describing the system, and nothing in the harness notices.</p><h3>What I would keep</h3><ol><li><strong>Fix the measuring stick before touching anything.</strong> Same k6 profile, same machine, same sampling, before and after.</li><li><strong>Put the constraint in the goal.</strong> “Lower memory” alone finds Alpine and C1. “Lower memory, throughput not worse, acceptance green” finds the AOT cache.</li><li><strong>Audit every flag against its documentation.</strong> One of them had been a no-op for a week.</li><li><strong>Record what you rejected, with numbers.</strong> The rejected list is worth more than the kept list to the next person.</li><li><strong>Train the AOT cache in the Dockerfile and fail the build if it is empty.</strong> Never ship a JVM that logs AOT errors at start.</li><li><strong>Keep native opt-in until the build fits the machine.</strong></li><li><strong>Say when there is no spec.</strong> The PR’s first paragraph did.</li></ol><h3>Go deeper</h3><p>The rest of the series, one piece at a time:</p><ul><li><strong>Part 1 — Seven specs, 620 tasks.</strong> Spec-driven development tested on a real microservices platform: the loop, clarify, converge, and the data nobody has. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Read Part 1 →</a></li><li><strong>Part 2 — The Stop hook said no 95 times.</strong> Harness engineering: three gates, a loop guard, and ten days of real timing data. <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Read Part 2 →</a></li><li><strong>Part 3–3,380 bugs on purpose.</strong> Mutation testing as the merge criterion: Pitest and Stryker at 80 %, survivors justified in the PR. <a href="https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8">Read Part 3 →</a></li><li><strong>Part 5 — The Mac was lying.</strong> /goal for fixing: SELinux, virtiofs, a build-time timestamp, and a self-hosted runner that finally worked on Linux. <a href="https://medium.com/@raphaelpanta/my-ci-runner-worked-on-my-mac-the-mac-was-lying-5eece22825ce">Read Part 5 →</a></li><li><strong>Part 6 — A constitution for an AI agent.</strong> Eight principles, 55 MUSTs, a check that runs twice per feature, and the PASS that turned out to be false. <a href="https://medium.com/@raphaelpanta/i-wrote-a-constitution-for-an-ai-agent-heres-what-it-could-and-couldn-t-enforce-da0e48c3b968">Read Part 6 →</a></li></ul><h3>Further reading</h3><ul><li><em>Keep Claude working toward a goal</em> — <a href="https://code.claude.com/docs/en/goal">code.claude.com/docs/en/goal</a></li><li>JEP 483, <em>Ahead-of-Time Class Loading &amp; Linking</em> (JDK 24) — <a href="https://openjdk.org/jeps/483">openjdk.org/jeps/483</a>; JEP 514, <em>Ahead-of-Time Command-Line Ergonomics</em> (JDK 25) — <a href="https://openjdk.org/jeps/514">openjdk.org/jeps/514</a>; JEP 515, <em>Ahead-of-Time Method Profiling</em> (JDK 25) — <a href="https://openjdk.org/jeps/515">openjdk.org/jeps/515</a></li><li>JEP 450, <em>Compact Object Headers (Experimental)</em> (JDK 24) — <a href="https://openjdk.org/jeps/450">openjdk.org/jeps/450</a>; JEP 519, <em>Compact Object Headers</em> (JDK 25) — <a href="https://openjdk.org/jeps/519">openjdk.org/jeps/519</a></li><li>Oracle, <em>The java Command</em> (JDK 25) — <a href="https://docs.oracle.com/en/java/javase/25/docs/specs/man/java.html">docs.oracle.com</a>; <em>Available Collectors</em>, HotSpot GC Tuning Guide — <a href="https://docs.oracle.com/en/java/javase/25/gctuning/available-collectors.html">docs.oracle.com</a>; <em>Networking Properties</em> — <a href="https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/net/doc-files/net-properties.html">docs.oracle.com</a></li><li>OpenJDK, <a href="https://bugs.openjdk.org/browse/JDK-8293114">JDK-8293114: JVM should trim the native heap</a>; <a href="https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/sun/net/InetAddressCachePolicy.java">InetAddressCachePolicy.java</a></li><li>glibc, <a href="https://man7.org/linux/man-pages/man3/mallopt.3.html">mallopt(3)</a>, M_ARENA_MAX / MALLOC_ARENA_MAX</li><li>Spring Framework, <em>JVM AOT Cache</em> — <a href="https://docs.spring.io/spring-framework/reference/integration/cds.html">docs.spring.io</a></li><li>GraalVM, <em>Reachability Metadata</em> — <a href="https://www.graalvm.org/latest/reference-manual/native-image/metadata/">graalvm.org</a>; <em>Build Options</em> — <a href="https://www.graalvm.org/latest/reference-manual/native-image/overview/BuildOptions/">graalvm.org</a></li><li>Apache Kafka native image — <a href="https://hub.docker.com/r/apache/kafka-native">hub.docker.com/r/apache/kafka-native</a>; Grafana k6 — <a href="https://grafana.com/docs/k6/latest/">grafana.com/docs/k6</a></li><li>Donald E. Knuth, <em>Structured Programming with go to Statements</em>, ACM Computing Surveys 6(4), December 1974 — <a href="https://www.gwern.net/doc/cs/algorithm/1974-knuth.pdf">PDF</a></li><li>The repository: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">PR #3</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile">platform/docker/Dockerfile</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/Dockerfile.native">Dockerfile.native</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/docker/README.md">platform/docker/README.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/compose/compose.native.yml">compose.native.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/platform/compose/docker-compose.yml">docker-compose.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/platform/perf">platform/perf</a></li></ul><p><em>All measurements are from </em><em>platform/docker/README.md and the description of PR #3, taken on 2026-10-08 on the development machine. I did not re-run them for this article; the numbers are the repository&#39;s, and the repository says how they were taken.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ceb2f0b4da66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Plantei 3.380 bugs no meu próprio código. Os testes deixaram passar 273.]]></title>
            <link>https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/146993812781</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Thu, 15 Oct 2026 12:07:13 GMT</pubDate>
            <atom:updated>2026-10-16T14:09:02.672Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Mutation testing como critério de merge: por que este repositório proíbe cobertura, como o gate lê cada linha alterada e o que os sobreviventes dizem sobre testes escritos por uma IA.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*DpKHfPKdPKQdt4BJNUl20w.png" /></figure><blockquote>Quis custodiet ipsos custodes?<em> — Quem vigia os vigilantes? — Juvenal, </em>Sátiras<em> VI</em></blockquote><p>Os testes vigiam o código. Ninguém vigia os testes. Uma suíte pode executar todas as linhas, não afirmar nada que importe e reportar 100 % de cobertura sem corar. Quando um humano escreve os testes, pelo menos dá para perguntar. Quando um agente os escreve, e os escreveu às três da manhã enquanto você dormia, você precisa de um segundo vigia.</p><p>Esta é a Parte 3 de uma série sobre <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin">uma plataforma de microsserviços em Kotlin</a> construída com Spec Kit e Claude Code. A <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Parte 1</a> foi sobre as specs; a <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Parte 2</a>, sobre os hooks, incluindo o que bloqueou o agente 95 vezes. Esta parte é sobre o que esses hooks estavam checando quando disseram não: um score de mutação, por módulo, que nunca pode cair abaixo de 80 %.</p><p>Tenho os relatórios da última execução completa, de 11 de outubro, para 14 módulos Kotlin e a loja em TypeScript. 3.380 mutantes. 273 passaram. Deixa eu mostrar quais foram.</p><p><em>As citações de fontes em inglês estão em tradução livre; os links levam ao original.</em></p><h3>Quebrar o código de propósito</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*2tmMkytidfsC5ypGzs3v0g.png" /></figure><p>Mutation testing é mais velho do que a maioria das pessoas que o usam. DeMillo, Lipton e Sayward o descreveram em 1978 (<a href="https://users.cs.northwestern.edu/~chrdimo/teaching/eecs396-w19/16.pdf">Hints on Test Data Selection</a>) a partir de duas premissas que continuam valendo: programadores escrevem código que está perto de correto, então os bugs que valem a pena encontrar são pequenos; e uma suíte de testes que pega todos os bugs pequenos tende a pegar os grandes também. Logo: faça uma mudança pequena, rode os testes, veja se algo falha.</p><p>O <a href="https://pitest.org/">Pitest</a> faz isso sobre bytecode JVM, ligado a cada módulo domain e application pelo <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">plugin de convenção </a><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">pitest</a>. Seus <a href="https://pitest.org/quickstart/mutators/">mutadores padrão</a> invertem uma condicional, deslocam uma fronteira (&lt; vira &lt;=), trocam um valor de retorno por vazio, nulo ou zero, apagam a chamada a um método void, trocam + por -. Cada mudança é um <em>mutante</em>. Se um teste falha, o mutante foi <em>morto</em>. Se todos os testes continuam passando, ele <em>sobreviveu</em>. Se nenhum teste sequer executou a linha, é <em>sem cobertura</em>, que também conta como falha, só que uma falha mais constrangedora.</p><p>O score é detectados ÷ total. A constituição deste repositório fixa o piso no <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">Princípio VIII</a>: todo módulo domain e application declara um limiar de pelo menos 80 % que não pode diminuir, e mutantes sobreviventes em linhas alteradas precisam ser mortos ou justificados no pull request. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">Princípio V</a> acrescenta a frase que eu acho a mais importante do documento inteiro:</p><blockquote><em>A eficácia dos testes é medida pelo score de mutação (Princípio VIII), não por cobertura de linhas. Percentuais de cobertura NÃO DEVEM ser usados como critério de merge por si só.</em></blockquote><blockquote><em>“Quando uma medida vira meta, ela deixa de ser uma boa medida.” — Formulação de Marilyn Strathern para a lei de Goodhart (1997)</em></blockquote><p>Cobertura virou meta faz muito tempo, e agentes são muito bons em bater metas. Peça 90 % de cobertura a um e você vai receber 90 % de cobertura. Peça um score de mutação e ele é obrigado a escrever testes que saibam distinguir certo de errado, porque é a única forma de o número se mexer.</p><h3>Os scores</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*V8jKe3ObOL6EqH14SwMKaw.png" /></figure><p>Catorze módulos, relatórios de 11 de outubro: as camadas domain e application dos seis serviços, o gateway e a biblioteca compartilhada platform-core. Todos passam de 80 %. O melhor é order/domain, com 98 % (212 mutantes, dois sobreviveram, dois sem cobertura). O pior é platform-core, com 85 %, e eu volto ao motivo mais adiante.</p><p>No total: 3.380 mutantes, 3.107 detectados, 92 %. Dos 273 não detectados, 99 sobreviveram a uma rodada de testes e 174 nunca foram cobertos por nenhuma.</p><p>O gate calcula o score do mesmo jeito que o Pitest, inclusive o arredondamento (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">mutation-gate.sh</a>: (detected * 200 + total) / (2 * total)), e compara com dois números: o limiar do módulo (80 por padrão; um módulo pode subir o seu, e o <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">plugin de convenção</a> lança erro se alguém tentar baixá-lo) e a entrada do módulo em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/quality/mutation-baseline.json">quality/mutation-baseline.json</a>, a catraca.</p><p>A catraca ainda é {}. A <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/tasks.md">Feature 001</a> a criou vazia de propósito (tarefa T045), para que as primeiras entradas fossem medições reais e não chutes, e nenhum pull request acrescentou uma desde então. Com os números acima, eu poderia commitar &quot;:services:order:domain&quot;: 98 hoje e o gate recusaria qualquer PR que o derrubasse para 97. Não fiz. Uma catraca que ninguém gira é só um limiar com passos a mais, e eu disse isso na Parte 1. Considere este o segundo lembrete.</p><h3>A cara dos sobreviventes</h3><p>Um score é um resumo. A parte interessante é ler os 273 um a um, porque eles não são todos o mesmo tipo de falha.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Wc_QNqwZMitVa0Urhe_7vA.png" /></figure><p><strong>A lacuna real.</strong> <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/catalog/application/src/main/kotlin/com/ecommerce/catalog/application/Reservations.kt">catalog/application/Reservations.kt</a>, linha 272: o Pitest deslocou uma fronteira condicional e nada falhou. Duas linhas acima, ele negou a condição e nada falhou também. É uma regra de reserva de estoque sem nenhum teste sentado na sua borda. A correção é um teste de propriedade com <a href="https://kotest.io/docs/proptest/property-based-testing.html">Kotest</a> tendo a fronteira como caso de borda nomeado, que é exatamente para o que a constituição diz que testes baseados em exemplo servem. Esse mutante é a razão de o gate existir.</p><p><strong>A lacuna de ferramenta.</strong> <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/libs/platform-core/src/main/kotlin/com/ecommerce/platform/core/values/Email.kt">platform-core/Email.kt</a>, linha 20: &quot;replaced return value with &quot;&quot; for Email::getValue&quot;. Não existe getValue no fonte Kotlin. Email é uma @JvmInline value class, o compilador gera o acessor, o Pitest o muta, e nenhum teste o chama porque nenhum código Kotlin jamais chama. Nove dos 51 mutantes não detectados de platform-core têm exatamente esse formato (SecretToken, Quantity, CorrelationId, PhoneNumber, IdempotencyKey, Currency…), e, somando os 14 módulos, 103 dos 273 são um valor de retorno trocado em algum acessor get…, muitos deles gerados pelo compilador. É boa parte do motivo de a biblioteca de value objects ter o score mais baixo: é o módulo com mais value classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*AT4tOIyKoZ8qpHaR662THw.png" /></figure><p>Agrupar os 273 por mutador deixa o padrão visível. <em>Negate conditionals</em> é 60 % de todos os mutantes gerados, mas só 4 % deles passam: os testes são bons em lógica. <em>Empty returns</em> é 12 % dos mutantes gerados e 23 % deles passam; <em>primitive returns</em>, 20 %. A suíte é muito pior em perceber uma função que silenciosamente devolve uma lista vazia, uma string vazia ou zero do que uma que toma o branch errado. Parte disso é o artefato das value classes. Parte é um hábito real, e é um hábito que vale a pena saber que o agente tem.</p><h3>Por que não há plugin de Kotlin</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*wiWpxOTz_VkQNOhP2_Glwg.png" /></figure><p>O problema das value classes tem nome: Kotlin compila para bytecode que ninguém escreveu. Tabelas $WhenMappings para when sobre enums, $DefaultImpls para métodos default de interface, máquinas de estado de corrotinas, guardas throwCountOverflow dentro de count {}. O Pitest muta tudo isso, e os mutantes ou são impossíveis de matar ou não significam nada.</p><p>A pesquisa da Feature 001, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/research.md">decisão D5</a>, olhou as três opções. O plugin open source <a href="https://github.com/pitest/pitest-kotlin">pitest-kotlin</a> “não é mantido” e está arquivado desde março de 2023. O mantido é o da <a href="https://docs.arcmutate.com/docs/kotlin.html">Arcmutate</a>, que “filtra essas mutações-lixo e remove o ruído confuso das descrições dos mutantes” e acrescenta mutadores específicos de Kotlin. É comercial; as licenças são pagas, por pacote, com prazo, e não há uma para um repositório público MIT. Está plugado atrás de -Pharness.pitest.arcmutate=true e nunca é ativado.</p><p>Então o padrão é uma aproximação escrita à mão: excludedClasses para *$WhenMappings, *$DefaultImpls, *Kt$*$1; excludedMethods para toString, hashCode, equals, copy, component*; avoidCallsTo para kotlin.jvm.internal, kotlin.Intrinsics, kotlinx.coroutines e kotlin.ResultKt. Isso leva todos os módulos acima de 80 %. Não pega acessores de value class, e tem mais um custo fácil de deixar passar nos comentários do plugin de convenção: o Pitest 1.30 delega a análise incremental a um plugin, então sem o plugin de histórico do Arcmutate <strong>toda execução é uma execução completa</strong>. O Pitest &quot;incremental&quot; do hook de Stop reduz o escopo por nome de classe, não por histórico.</p><h3>A regra que lê o seu diff</h3><p>Um score por módulo tem um buraco. Uma mudança ruim dentro de um módulo bom pode derrubar o score de 98 para 96 e passar tranquila. Por isso o gate do PR tem um segundo script que ignora o score por completo e lê o diff.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*uECQYowq-wP71FHqwGrwGw.png" /></figure><p><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">surviving-mutants.sh</a> pega git diff -U0 base...HEAD, guarda os números das linhas adicionadas por arquivo e cruza com toda linha SURVIVED ou NO_COVERAGE de todo mutations.xml. Cada cruzamento precisa de uma linha na descrição do pull request, sob ## Mutant justifications, no formato caminho:linha motivo. A descrição vem do payload do evento, então a checagem não precisa de rede; o workflow escuta edited além de synchronize, então acrescentar uma justificativa roda o gate de novo com o texto novo. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">template de PR</a> já traz o título da seção, com um exemplo: Price.kt:42 equivalent mutant: the bound is unreachable.</p><p>Essa expressão, <em>mutante equivalente</em>, é a saída de emergência honesta. Alguns mutantes não podem ser mortos porque o programa mutado se comporta de forma idêntica ao original para qualquer entrada. Decidir isso em geral é indecidível; decidir para uma linha é uma frase num PR. O gate não tenta ser esperto. Ele pede que um humano escreva a frase.</p><p>Os dois PRs já mesclados, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">#3</a> e <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">#4</a>, têm a seção de justificativas vazia. Nenhum dos dois tocou um módulo domain ou application, então não havia sobreviventes em linhas alteradas para justificar. A regra ainda não foi exercitada por um sobrevivente real no CI. Registro isso em vez de esconder.</p><h3>A loja: Stryker</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*Bq1awT9WbxLvhGkHFP0iDw.png" /></figure><p>O lado TypeScript roda o <a href="https://stryker-mutator.io/">Stryker</a> sobre <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/domain">src/domain</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/app">src/app</a> e <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/telemetry">src/telemetry</a>, com thresholds.break em 80 no <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/stryker.config.json">stryker.config.json</a>. O hook de Stop o roda em <a href="https://stryker-mutator.io/docs/stryker-js/incremental/">modo incremental</a>, que &quot;acompanha as mudanças que você faz no código e nos testes e só roda mutation testing no código alterado&quot;; o gate do PR o roda completo. O arquivo incremental de 11 de outubro tem 49 arquivos, 1.684 mortos, 196 sobreviventes, 26 sem cobertura, 272 ignorados: 88 %.</p><p>Os sobreviventes se agrupam. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/src/telemetry/setup.ts">src/telemetry/setup.ts</a> tem 36 de 130 não detectados; <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/src/app/checkout/useCheckout.ts">useCheckout.ts</a>, 29 de 100. Setup de telemetria no navegador é código de fiação, difícil de testar unitariamente e tentador de deixar para a suíte de aceitação, que o Stryker não roda. É uma decisão razoável e é também exatamente o tipo de decisão que um score deixa você tomar sem perceber. A catraca do Pitest não cobre o Stryker; a única catraca da loja é o limiar na sua própria configuração.</p><h3>Quanto custa</h3><p>Nada disso é de graça, e os comentários do <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">plugin de convenção</a> se leem como um diário de quem descobriu isso na prática.</p><p>O Pitest sobe uma JVM “minion” por thread, cerca de 300 MB cada, fora do limite de workers do Gradle. A primeira configuração dava a cada módulo todas as CPUs e, com vários módulos rodando Pitest ao mesmo tempo, o runner de CI chegou a “load average 37 em 10 CPUs”, com Testcontainers dando timeout ao lado. Cada task agora recebe o dobro da sua fatia de CPUs e um -Xmx512m explícito, porque uma JVM sem isso toma um quarto do container como teto e o kernel matou o daemon do Gradle.</p><p>No <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/stop-full-check.sh">hook de Stop</a>, verify roda o Pitest completo de todo módulo domain e application. É a maior parte da mediana de cinco minutos do veredito verde da Parte 2. No CI, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/verify.yml">verify</a> roda uma vez e entrega o XML ao <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">job </a><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">mutation</a> como artefato, então nunca se paga duas vezes por pull request.</p><h3>O caso contra, e o que os relatórios não dizem</h3><p><strong>Score de mutação também é uma meta ruim.</strong> A <a href="https://pt.wikipedia.org/wiki/Lei_de_Goodhart">lei de Goodhart</a> não deixa de valer porque a medida ficou mais esperta. Um agente instruído a chegar a 80 % pode escrever um teste que mata um mutante sem testar nada que importe: afirme que o retorno não é vazio, e o mutante de <em>empty returns</em> morre. O score sobe; o teste não vale nada. Mutation testing sobe o piso do que um teste precisa fazer. Não sobe o teto de quão bobo um teste pode ser enquanto faz isso.</p><p><strong>Mutantes equivalentes são indecidíveis, e a saída de emergência é um campo de texto.</strong> A linha de justificativa é escrita por quem abre o PR. Neste repositório, normalmente é o agente. O mesmo modelo que escreveu o código e os testes escreve a frase explicando por que um mutante não pode ser morto. O alerta de Böckeler da Parte 2, de que um harness centrado em testes “deposita muita fé nos testes gerados pela IA”, vale com uma camada a mais: fé na desculpa gerada pela IA. A única defesa é um revisor que leia a seção. O gate não consegue checar isso.</p><p><strong>O Google não usa o score, e eles pensaram nisso mais do que qualquer um.</strong> <a href="https://research.google/pubs/state-of-mutation-testing-at-google/">State of Mutation Testing at Google</a>, de Petrović e Ivanković (ICSE SEIP 2018), abandonou a abordagem tradicional em favor de “reduzir o número de mutantes e selecionar com cuidado apenas os mais interessantes”, mostrados durante o code review em vez de usados como gate. A continuação, <a href="https://arxiv.org/abs/2102.11378">Practical Mutation Testing at Scale</a> (com Fraser e Just, 2021), avaliada com “mais de 24.000 desenvolvedores em mais de 1.000 projetos”, é direta: “A abordagem tradicional de mutation testing não escala para um ambiente desses.” Ela produz “ordens de grandeza menos mutantes” suprimindo os improdutivos. Este repositório roda a abordagem tradicional, todos os mutantes, toda execução, e usa o score como gate de merge. Com 3.380 mutantes isso funciona. Com 338.000, não funcionaria.</p><p><strong>Execuções completas, sempre.</strong> Sem o plugin de histórico não existe Pitest incremental. Todo hook de Stop paga por todo mutante de todo módulo alterado. É a troca por não pagar o Arcmutate, e é uma troca que piora linearmente com o tamanho da base de código.</p><p><strong>A catraca continua vazia.</strong> Dois artigos seguidos.</p><p><strong>E ninguém mediu isso com agentes.</strong> A literatura de mutation testing é sobre humanos. Não encontro nenhum estudo sobre o que um agente faz quando um score de mutação é a sua meta em vez de cobertura, se ele escreve testes melhores ou apenas diferentes, ou com que frequência uma justificativa de mutante equivalente escrita por um agente está errada. Os 273 sobreviventes acima são um ponto de dado de um repositório. A distribuição por mutador é sugestiva, não é evidência.</p><h3>O que eu manteria</h3><ol><li><strong>Proibir cobertura como critério, por escrito.</strong> Uma frase na constituição. Ela muda o que o agente otimiza.</li><li><strong>Limiares por módulo que só podem subir.</strong> Imponha o piso no plugin de build para que ninguém o baixe num script de build.</li><li><strong>Ler o diff, não só o score.</strong> Sobreviventes em linhas adicionadas, justificados linha a linha, ou o PR falha.</li><li><strong>Deixar a saída de emergência visível.</strong> equivalent mutant: &lt;motivo&gt; na descrição, onde um revisor vê.</li><li><strong>Pagar o Pitest uma vez por PR.</strong> Rode em verify, entregue o XML ao job do gate.</li><li><strong>Ler os sobreviventes por mutador.</strong> A distribuição me disse mais sobre os hábitos de teste do agente do que o score.</li><li><strong>Girar a catraca.</strong> De verdade. No próximo PR.</li></ol><h3>Para se aprofundar</h3><p>O resto da série, um por vez:</p><ul><li><strong>Parte 1 — Sete specs, 620 tarefas.</strong> Spec-driven development testado numa plataforma de microsserviços real: o loop, clarify, converge e os dados que ninguém tem. <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Leia a Parte 1 →</a></li><li><strong>Parte 2 — O hook de Stop disse não 95 vezes.</strong> Harness engineering: três gates, um guarda de loop e dez dias de dados reais de tempo. <a href="https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc">Leia a Parte 2 →</a></li><li><strong>Parte 4 — Um número em vez de uma spec.</strong> /goal para performance: AOT cache do JDK 25, compact object headers, imagens nativas, de 4,3 GiB para 1,9. <a href="https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e">Leia a Parte 4 →</a></li><li><strong>Parte 5 — O Mac estava mentindo.</strong> /goal para correção: SELinux, virtiofs, um timestamp de build e um runner self-hosted que finalmente funcionou no Linux. <a href="https://medium.com/@raphaelpanta/meu-runner-de-ci-funcionava-no-meu-mac-o-mac-estava-mentindo-02e8e8ac6a9c">Leia a Parte 5 →</a></li><li><strong>Parte 6 — Uma constituição para um agente de IA.</strong> Oito princípios, 55 MUSTs, uma checagem que roda duas vezes por feature e o PASS que acabou sendo falso. <a href="https://medium.com/@raphaelpanta/escrevi-uma-constitui%C3%A7%C3%A3o-para-um-agente-de-ia-eis-o-que-ela-conseguiu-e-o-que-n%C3%A3o-conseguiu-impor-31332eadc192">Leia a Parte 6 →</a></li></ul><h3>Leitura adicional</h3><ul><li>R. A. DeMillo, R. J. Lipton, F. G. Sayward, <em>Hints on Test Data Selection: Help for the Practicing Programmer</em> — IEEE Computer, abril de 1978 (<a href="https://users.cs.northwestern.edu/~chrdimo/teaching/eecs396-w19/16.pdf">PDF</a>)</li><li>Pitest — <a href="https://pitest.org/">pitest.org</a>, <a href="https://pitest.org/quickstart/mutators/">mutadores disponíveis</a></li><li>Arcmutate, <em>Kotlin Support</em> — <a href="https://docs.arcmutate.com/docs/kotlin.html">docs.arcmutate.com/docs/kotlin.html</a>; o plugin arquivado <a href="https://github.com/pitest/pitest-kotlin">pitest-kotlin</a></li><li>Stryker, <em>Incremental</em> — <a href="https://stryker-mutator.io/docs/stryker-js/incremental/">stryker-mutator.io/docs/stryker-js/incremental</a></li><li>Goran Petrović, Marko Ivanković, <em>State of Mutation Testing at Google</em> — ICSE SEIP 2018 (<a href="https://research.google/pubs/state-of-mutation-testing-at-google/">research.google</a>)</li><li>Goran Petrović, Marko Ivanković, Gordon Fraser, René Just, <em>Practical Mutation Testing at Scale</em> — <a href="https://arxiv.org/abs/2102.11378">arXiv:2102.11378</a> (2021)</li><li>Lei de Goodhart — <a href="https://pt.wikipedia.org/wiki/Lei_de_Goodhart">Wikipédia</a></li><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com</a></li><li>O repositório: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">pitest.gradle.kts</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">mutation-gate.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">surviving-mutants.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/quality/mutation-baseline.json">mutation-baseline.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/stryker.config.json">stryker.config.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/research.md">research D5</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constituição</a></li></ul><p><em>Os números vêm do </em><em>build/reports/pitest/mutations.xml de cada módulo e do </em><em>frontend/build/stryker-incremental.json na máquina de desenvolvimento, todos de 11 de outubro de 2026, no commit </em><em>9870792. Os scores são detectados ÷ total com o arredondamento do Pitest; o score do Stryker exclui mutantes ignorados e erros de runtime, como o próprio Stryker faz.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=146993812781" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[I Planted 3,380 Bugs in My Own Codebase. The Tests Missed 273.]]></title>
            <link>https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/fc0ae0ec8fe8</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 11:24:36 GMT</pubDate>
            <atom:updated>2026-10-16T14:06:08.724Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Mutation testing as the merge criterion: why this repository bans coverage, how the gate reads every changed line, and what the survivors say about tests an AI wrote.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*qy_f7OLelfT546uGadxzaQ.png" /></figure><blockquote>Quis custodiet ipsos custodes?<em> — Who will guard the guards themselves? — Juvenal, </em>Satires<em> VI</em></blockquote><p>Tests guard the code. Nothing guards the tests. A suite can run every line, assert nothing of consequence, and report 100 % coverage with a straight face. When a human writes the tests you can at least ask them. When an agent writes them, and it wrote them at three in the morning while you were asleep, you need a second guard.</p><p>This is Part 3 of a series about <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin">a Kotlin microservices platform</a> built with Spec Kit and Claude Code. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Part 1</a> was the specs; <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Part 2</a> was the hooks, including the one that blocked the agent 95 times. This part is about what those hooks were checking when they said no: a mutation score, per module, that must never drop below 80 %.</p><p>I have the reports from the last full run on 11 October, for 14 Kotlin modules and the TypeScript storefront. 3,380 mutants. 273 got through. Let me show you what they were.</p><h3>Break the code on purpose</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*qPA7a8TjTKPLf3-tVnS9xQ.png" /></figure><p>Mutation testing is older than most of the people using it. DeMillo, Lipton and Sayward described it in 1978 (<a href="https://users.cs.northwestern.edu/~chrdimo/teaching/eecs396-w19/16.pdf">Hints on Test Data Selection</a>) on two assumptions that still hold: programmers write code that is close to correct, so the bugs worth finding are small; and a test suite that catches all the small bugs tends to catch the large ones too. So: make a small change, run the tests, see if anything fails.</p><p><a href="https://pitest.org/">Pitest</a> does this on JVM bytecode, wired into every domain and application module by the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">pitest convention plugin</a>. Its <a href="https://pitest.org/quickstart/mutators/">default mutators</a> flip a conditional, move a boundary (&lt; to &lt;=), replace a return value with empty, null or zero, delete a call to a void method, swap + for -. Each change is a <em>mutant</em>. If a test fails, the mutant is <em>killed</em>. If every test still passes, it <em>survived</em>. If no test even executed the line, it is <em>no coverage</em>, which is also a failure, just a more embarrassing one.</p><p>The score is detected ÷ total. This repository’s constitution sets the floor in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">Principle VIII</a>: every domain and application module declares a threshold of at least 80 % that must not decrease, and surviving mutants on changed lines must be killed or justified in the pull request. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">Principle V</a> adds the sentence I think is the most important one in the document:</p><blockquote><em>Test effectiveness is measured by mutation score (Principle VIII), not by line coverage. Coverage percentages MUST NOT be used as a merge criterion on their own.</em></blockquote><blockquote><em>“When a measure becomes a target, it ceases to be a good measure.” — Marilyn Strathern’s phrasing of Goodhart’s law (1997)</em></blockquote><p>Coverage became a target a long time ago, and agents are very good at hitting targets. Ask one for 90 % coverage and you will get 90 % coverage. Ask it for a mutation score and it has to write tests that can tell right from wrong, because that is the only way the number moves.</p><h3>The scores</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*h0018FgWpLlP6q_ibfyBvA.png" /></figure><p>Fourteen modules, reports dated 11 October: the six services’ domain and application layers, the gateway, and the shared platform-core library. Every one clears 80 %. The best is order/domain at 98 % (212 mutants, two survived, two uncovered). The worst is platform-core at 85 %, and I will come back to why.</p><p>In total: 3,380 mutants, 3,107 detected, 92 %. Of the 273 not detected, 99 survived a test run and 174 were never covered by one.</p><p>The gate computes the score the same way Pitest does, including the rounding (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">mutation-gate.sh</a>: (detected * 200 + total) / (2 * total)), and compares it against two numbers: the module&#39;s threshold (80 by default; a module may raise it, and the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">convention plugin</a> throws if anyone tries to lower it) and the module&#39;s entry in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/quality/mutation-baseline.json">quality/mutation-baseline.json</a>, the ratchet.</p><p>The ratchet is still {}. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/tasks.md">Feature 001</a> created it empty on purpose (task T045), so that the first entries would be real measurements rather than guesses, and no pull request has added one since. With the numbers above, I could commit &quot;:services:order:domain&quot;: 98 today and the gate would refuse any PR that pushed it to 97. I have not done it. A ratchet nobody turns is just a threshold with extra steps, and I said so in Part 1. Consider this the second reminder.</p><h3>What the survivors look like</h3><p>A score is a summary. The interesting part is reading the 273 one at a time, because they are not all the same kind of failure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b_QVwLTWzg_BcQvOikjIRg.png" /></figure><p><strong>The real gap.</strong> <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/services/catalog/application/src/main/kotlin/com/ecommerce/catalog/application/Reservations.kt">catalog/application/Reservations.kt</a>, line 272: Pitest moved a conditional boundary and nothing failed. Two lines up, it negated the condition and nothing failed either. This is a stock-reservation rule with no test sitting on its edge. The fix is one <a href="https://kotest.io/docs/proptest/property-based-testing.html">Kotest</a> property test with the boundary as a named edge case, which is exactly what the constitution says example-based tests are for. This mutant is why the gate exists.</p><p><strong>The tooling gap.</strong> <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/libs/platform-core/src/main/kotlin/com/ecommerce/platform/core/values/Email.kt">platform-core/Email.kt</a>, line 20: &quot;replaced return value with &quot;&quot; for Email::getValue&quot;. There is no getValue in the Kotlin source. Email is a @JvmInline value class, the compiler generates the accessor, Pitest mutates it, and no test calls it because no Kotlin code ever does. Nine of platform-core&#39;s 51 undetected mutants are this exact shape (SecretToken, Quantity, CorrelationId, PhoneNumber, IdempotencyKey, Currency…), and across all 14 modules 103 of the 273 are a replaced return value of some get… accessor, many of them compiler-generated. That is a large part of why the value-object library has the lowest score: it is the module with the most value classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*7KlBZ6cxeODPysNbxHeXxA.png" /></figure><p>Grouping all 273 by mutator makes the pattern visible. <em>Negate conditionals</em> is 60 % of all mutants generated but only 4 % of them get through: the tests are good at logic. <em>Empty returns</em> is 12 % of mutants generated and 23 % of them get through; <em>primitive returns</em>, 20 %. The suite is much worse at noticing a function that quietly returns an empty list, an empty string or zero than one that takes the wrong branch. Some of that is the value-class artefact. Some of it is a real habit, and it is a habit worth knowing an agent has.</p><h3>Why there is no Kotlin plugin</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*KznXeQ3gt7_zfWSJzINFYg.png" /></figure><p>The value-class problem has a name: Kotlin compiles to bytecode nobody wrote. $WhenMappings tables for when over enums, $DefaultImpls for interface defaults, coroutine state machines, throwCountOverflow guards inside count {}. Pitest mutates all of it, and the mutants are either unkillable or meaningless.</p><p>Feature 001’s research, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/research.md">decision D5</a>, looked at the three options. The open-source <a href="https://github.com/pitest/pitest-kotlin">pitest-kotlin</a> plugin “is not maintained” and has been archived since March 2023. The maintained one is <a href="https://docs.arcmutate.com/docs/kotlin.html">Arcmutate’s</a>, which “filters out these junk mutations and removes confusing noise from mutant descriptions” and adds Kotlin-specific mutators. It is commercial; licences are paid, per package, time-limited, and there is none for a public MIT repository. It is wired in behind -Pharness.pitest.arcmutate=true and never set.</p><p>So the default is a hand-written approximation: excludedClasses for *$WhenMappings, *$DefaultImpls, *Kt$*$1; excludedMethods for toString, hashCode, equals, copy, component*; avoidCallsTo for kotlin.jvm.internal, kotlin.Intrinsics, kotlinx.coroutines and kotlin.ResultKt. It gets every module over 80 %. It does not catch value-class accessors, and it has one more cost that is easy to miss in the convention plugin&#39;s comments: Pitest 1.30 delegates incremental analysis to a plugin, so without Arcmutate&#39;s history plugin <strong>every run is a full run</strong>. The Stop hook&#39;s &quot;incremental&quot; Pitest narrows by class name, not by history.</p><h3>The rule that reads your diff</h3><p>A module-wide score has a hole in it. A bad change inside a good module can drop the score from 98 to 96 and sail through. So the PR gate has a second script that ignores the score entirely and reads the diff.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*SoTVwWxfHYIZakhlBwteBw.png" /></figure><p><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">surviving-mutants.sh</a> takes git diff -U0 base...HEAD, keeps the added line numbers per file, and joins them with every SURVIVED or NO_COVERAGE row in every mutations.xml. Each match must have a line in the pull request description, under ## Mutant justifications, of the form path:line reason. The description comes from the event payload, so the check needs no network; the workflow listens to edited as well as synchronize, so adding a justification re-runs the gate with the new text. The <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">PR template</a> already has the heading, with an example: Price.kt:42 equivalent mutant: the bound is unreachable.</p><p>That phrase, <em>equivalent mutant</em>, is the honest escape hatch. Some mutants cannot be killed because the mutated program behaves identically to the original for every input. Deciding that in general is undecidable; deciding it for one line is a sentence in a PR. The gate does not try to be clever. It asks a human to write the sentence.</p><p>Both merged PRs so far, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/3">#3</a> and <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/pull/4">#4</a>, have an empty justifications section. Neither touched a domain or application module, so there were no survivors on changed lines to justify. The rule has not yet been exercised by a real survivor in CI. I note that rather than hide it.</p><h3>The storefront: Stryker</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*NAQhhSNXbNuC-RhN_ODcIQ.png" /></figure><p>The TypeScript side runs <a href="https://stryker-mutator.io/">Stryker</a> over <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/domain">src/domain</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/app">src/app</a> and <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/frontend/src/telemetry">src/telemetry</a>, with thresholds.break at 80 in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/stryker.config.json">stryker.config.json</a>. The Stop hook runs it in <a href="https://stryker-mutator.io/docs/stryker-js/incremental/">incremental mode</a>, which &quot;will track the changes you make to your code and tests and only runs mutation testing on the changed code&quot;; the PR gate runs it in full. The incremental file from 11 October has 49 files, 1,684 killed, 196 survived, 26 uncovered, 272 ignored: 88 %.</p><p>The survivors cluster. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/src/telemetry/setup.ts">src/telemetry/setup.ts</a> has 36 of 130 undetected; <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/src/app/checkout/useCheckout.ts">useCheckout.ts</a>, 29 of 100. Browser telemetry setup is wiring code, hard to unit-test and tempting to leave for the acceptance suite, which Stryker does not run. That is a reasonable call and also exactly the kind of call a score lets you make without noticing. The Pitest ratchet does not cover Stryker; the storefront&#39;s only ratchet is the threshold in its own config.</p><h3>What it costs</h3><p>Nothing about this is free, and the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">convention plugin</a>’s comments read like a diary of finding that out.</p><p>Pitest forks one minion JVM per thread, about 300 MB each, outside Gradle’s worker limit. The first configuration gave every module all the CPUs, and with several modules running Pitest at once the CI runner hit “load average 37 on 10 CPUs”, with Testcontainers timing out next to it. Each task now gets twice its share of the CPUs and an explicit -Xmx512m, because a JVM without one takes a quarter of the container as its ceiling and the kernel killed the Gradle daemon.</p><p>In the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/stop-full-check.sh">Stop hook</a>, verify runs the full Pitest of every domain and application module. That is most of the five-minute median green verdict from Part 2. In CI, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/verify.yml">verify</a> runs it once and hands the XML to the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">mutation job</a> as an artifact, so it is never paid twice per pull request.</p><h3>The case against, and what the reports can’t tell you</h3><p><strong>Mutation score is a bad target too.</strong> <a href="https://en.wikipedia.org/wiki/Goodhart%27s_law">Goodhart’s law</a> does not stop applying because the measure got smarter. An agent asked to reach 80 % can write a test that kills a mutant without testing anything meaningful: assert that the return value is not empty, and the <em>empty returns</em> mutant dies. The score goes up; the test is worthless. Mutation testing raises the floor on what a test must do. It does not raise the ceiling on how silly a test can be while doing it.</p><p><strong>Equivalent mutants are undecidable, and the escape hatch is a text field.</strong> The justification line is written by whoever opens the PR. In this repository, that is usually the agent. The same model that wrote the code and the tests writes the sentence explaining why a mutant cannot be killed. Böckeler’s warning from Part 2, that a test-centred harness “puts a lot of faith into the AI-generated tests”, applies with one more layer: faith in the AI-generated excuse. The only defence is a reviewer who reads the section. The gate cannot check that.</p><p><strong>Google does not use the score, and they have thought about this more than anyone.</strong> Petrović and Ivanković’s <a href="https://research.google/pubs/state-of-mutation-testing-at-google/">State of Mutation Testing at Google</a> (ICSE SEIP 2018) abandoned the traditional approach in favour of “reducing the number of mutants and carefully selecting only the most interesting ones”, surfaced during code review rather than as a gate. The follow-up, <a href="https://arxiv.org/abs/2102.11378">Practical Mutation Testing at Scale</a> (with Fraser and Just, 2021), evaluated on “more than 24,000 developers on more than 1,000 projects”, is blunt: “The traditional approach to mutation testing does not scale to such an environment.” It produces “orders of magnitude fewer mutants” by suppressing the unproductive ones. This repository runs the traditional approach, all mutants, every run, and uses the score as a merge gate. At 3,380 mutants that works. At 338,000 it would not.</p><p><strong>Full runs, every time.</strong> Without the history plugin there is no incremental Pitest. Every Stop hook pays for every mutant in every changed module. That is the trade for not paying Arcmutate, and it is a trade that gets worse linearly with the codebase.</p><p><strong>The ratchet is still empty.</strong> Two articles in a row now.</p><p><strong>And no one has measured this with agents.</strong> The mutation-testing literature is about humans. I can find no study of what an agent does when a mutation score is its target rather than coverage, whether it writes better tests or just different ones, or how often an agent-written justification for an equivalent mutant is wrong. The 273 survivors above are one data point from one repository. The breakdown by mutator is suggestive, not evidence.</p><h3>What I would keep</h3><ol><li><strong>Ban coverage as a criterion, in writing.</strong> One sentence in the constitution. It changes what the agent optimises for.</li><li><strong>Per-module thresholds that can only go up.</strong> Enforce the floor in the build plugin so nobody can lower it in a build script.</li><li><strong>Read the diff, not just the score.</strong> Survivors on added lines, justified by line, or the PR fails.</li><li><strong>Make the escape hatch visible.</strong> equivalent mutant: &lt;reason&gt; in the description, where a reviewer sees it.</li><li><strong>Pay Pitest once per PR.</strong> Run it in verify, hand the XML to the gate job.</li><li><strong>Read the survivors by mutator.</strong> The breakdown told me more about the agent’s testing habits than the score did.</li><li><strong>Turn the ratchet.</strong> Really. Next PR.</li></ol><h3>Go deeper</h3><p>The rest of the series, one piece at a time:</p><ul><li><strong>Part 1 — Seven specs, 620 tasks.</strong> Spec-driven development tested on a real microservices platform: the loop, clarify, converge, and the data nobody has. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Read Part 1 →</a></li><li><strong>Part 2 — The Stop hook said no 95 times.</strong> Harness engineering: three gates, a loop guard, and ten days of real timing data. <a href="https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579">Read Part 2 →</a></li><li><strong>Part 4 — A number instead of a spec.</strong> /goal for performance: JDK 25 AOT cache, compact object headers, native images, 4.3 GiB to 1.9. <a href="https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66">Read Part 4 →</a></li><li><strong>Part 5 — The Mac was lying.</strong> /goal for fixing: SELinux, virtiofs, a build-time timestamp, and a self-hosted runner that finally worked on Linux. <a href="https://medium.com/@raphaelpanta/my-ci-runner-worked-on-my-mac-the-mac-was-lying-5eece22825ce">Read Part 5 →</a></li><li><strong>Part 6 — A constitution for an AI agent.</strong> Eight principles, 55 MUSTs, a check that runs twice per feature, and the PASS that turned out to be false. <a href="https://medium.com/@raphaelpanta/i-wrote-a-constitution-for-an-ai-agent-heres-what-it-could-and-couldn-t-enforce-da0e48c3b968">Read Part 6 →</a></li></ul><h3>Further reading</h3><ul><li>R. A. DeMillo, R. J. Lipton, F. G. Sayward, <em>Hints on Test Data Selection: Help for the Practicing Programmer</em> — IEEE Computer, April 1978 (<a href="https://users.cs.northwestern.edu/~chrdimo/teaching/eecs396-w19/16.pdf">PDF</a>)</li><li>Pitest — <a href="https://pitest.org/">pitest.org</a>, <a href="https://pitest.org/quickstart/mutators/">available mutators</a></li><li>Arcmutate, <em>Kotlin Support</em> — <a href="https://docs.arcmutate.com/docs/kotlin.html">docs.arcmutate.com/docs/kotlin.html</a>; the archived <a href="https://github.com/pitest/pitest-kotlin">pitest-kotlin</a> plugin</li><li>Stryker, <em>Incremental</em> — <a href="https://stryker-mutator.io/docs/stryker-js/incremental/">stryker-mutator.io/docs/stryker-js/incremental</a></li><li>Goran Petrović, Marko Ivanković, <em>State of Mutation Testing at Google</em> — ICSE SEIP 2018 (<a href="https://research.google/pubs/state-of-mutation-testing-at-google/">research.google</a>)</li><li>Goran Petrović, Marko Ivanković, Gordon Fraser, René Just, <em>Practical Mutation Testing at Scale</em> — <a href="https://arxiv.org/abs/2102.11378">arXiv:2102.11378</a> (2021)</li><li>Goodhart’s law — <a href="https://en.wikipedia.org/wiki/Goodhart%27s_law">Wikipedia</a></li><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com</a></li><li>The repository: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/pitest.gradle.kts">pitest.gradle.kts</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">mutation-gate.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">surviving-mutants.sh</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/quality/mutation-baseline.json">mutation-baseline.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/frontend/stryker.config.json">stryker.config.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/research.md">research D5</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constitution</a></li></ul><p><em>Numbers come from the </em><em>build/reports/pitest/mutations.xml of each module and </em><em>frontend/build/stryker-incremental.json on the development machine, all dated 11 October 2026, at commit </em><em>9870792. Scores are detected ÷ total with Pitest&#39;s rounding; Stryker&#39;s score excludes ignored mutants and runtime errors, as Stryker does.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fc0ae0ec8fe8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Meu agente de IA disse “pronto” 220 vezes. Um script shell disse não a 95 delas.]]></title>
            <link>https://medium.com/@raphaelpanta/meu-agente-de-ia-disse-pronto-220-vezes-um-script-shell-disse-n%C3%A3o-a-95-delas-eab038b4eafc?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/eab038b4eafc</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[kotlin]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 11:20:55 GMT</pubDate>
            <atom:updated>2026-10-16T14:08:28.529Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Harness engineering num monorepo Kotlin de microsserviços: três gates, um hook que se recusa a deixar o agente terminar, e dez dias de dados reais de tempo.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*uRGCImxAisfQVRB_6UFWmg.png" /></figure><blockquote><em>“Testar um programa pode servir para mostrar a presença de bugs, mas nunca para mostrar sua ausência!” — Edsger W. Dijkstra, </em>Notes on Structured Programming<em> (1970)</em></blockquote><p>A <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Parte 1</a> desta série foi sobre as specs: sete delas, 620 tarefas, uma plataforma de e-commerce construída a partir do brief do roadmap.sh. Terminei com uma confissão. Lendo o relato da OpenAI sobre um milhão de linhas escritas por agentes, tive que admitir que as partes do meu repositório que mais trabalhavam provavelmente não eram as specs. Eram os hooks.</p><p>Esta parte é sobre esses hooks. Não a ideia deles; o log. O repositório mantém um arquivo chamado gate-timing.log (ignorado pelo git; formato em <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>) com uma linha por execução de gate, e entre 2 e 11 de outubro ele acumulou 244 delas. O gate de fim de tarefa rodou 220 vezes. Deixou o agente parar 92 vezes e recusou 95.</p><p>Esse número é o artigo inteiro. Um gate de qualidade que passa sempre é decoração. Um que falha mais do que passa está fazendo alguma coisa.</p><p><em>As citações de fontes em inglês estão em tradução livre; os links levam ao original.</em></p><h3>O que “harness” significa aqui</h3><p>A palavra se espalhou rápido o bastante para significar quase qualquer coisa, então eis a definição que estou usando, do texto de Birgitta Böckeler de abril de 2026 no martinfowler.com: “O termo harness surgiu como abreviação para tudo num agente de IA exceto o próprio modelo” (<a href="https://www.martinfowler.com/articles/harness-engineering.html">Harness engineering for coding agent users</a>). Ela divide esse “tudo” em dois tipos de controle, emprestados da cibernética:</p><ul><li><strong>Guias</strong> (feedforward) “antecipam o comportamento do agente e tentam direcioná-lo <em>antes</em> de agir.” Instruções, templates, scaffolding, constituições.</li><li><strong>Sensores</strong> (feedback) “observam <em>depois</em> que o agente age e o ajudam a se corrigir.” Compiladores, linters, testes, hooks que bloqueiam.</li></ul><p>E ela empresta uma lei para acompanhar. A lei da variedade requisito, de Ashby: “um regulador precisa ter pelo menos tanta variedade quanto o sistema que governa.” Uma página de regras em Markdown não consegue regular 29.000 linhas de Kotlin. Algo com o mesmo alcance do código precisa olhar para o código.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*LheLK3Q_8ml_T2-7mpvhBw.png" /></figure><p>Essa é a matriz com as partes deste repositório posicionadas nela. O peso está embaixo à esquerda: sensores determinísticos. A constituição e as specs da Parte 1 moram na célula de cima à esquerda, e importam, mas são conselho. A célula de baixo à esquerda é onde conselho vira muro.</p><p>A própria documentação da Anthropic traça a mesma linha. O CLAUDE.md é “consultivo”; hooks dão “controle determinístico: certas ações sempre acontecem, em vez de depender de o LLM escolher executá-las” (<a href="https://code.claude.com/docs/en/hooks-guide">Automate actions with hooks</a>). A página de boas práticas vai além e diz o que fazer com uma regra que o agente insiste em ignorar: “Se o Claude já faz algo corretamente sem a instrução, apague-a ou converta-a num hook” (<a href="https://code.claude.com/docs/en/best-practices">Best practices</a>).</p><h3>Três gates</h3><p>O oitavo princípio da constituição exigiu três deles antes de qualquer serviço ser escrito, e a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/specs/001-harness-quality-gates">feature 001</a> os construiu. Cada um tem um gatilho, um orçamento de tempo e um limite para quanto pode falar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*TdcMSNFO4Qwk7N1mbuGNgg.png" /></figure><p><strong>O </strong><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/post-edit-check.sh"><strong>gate por arquivo</strong></a> dispara após cada Write ou Edit. Para um arquivo Kotlin, roda ktlint e detekt no módulo Gradle do arquivo, depois toda classe de teste cujo nome começa com o nome do arquivo (Price* → PriceSpec, PricePropertyTest), no máximo vinte. Para TypeScript: eslint, tsc, prettier se houver configuração, e vitest related, que roda só os testes que importam o arquivo. Orçamento de 300 segundos, relatório limitado a 60 linhas, exit 2 em falha, o que impede o agente de seguir em frente.</p><p><strong>O gate de fim de tarefa</strong> dispara no Stop, quando o agente acredita que terminou. Roda ./gradlew -q verify, que é o check do repositório inteiro mais Pitest em cada módulo de domínio e aplicação, depois lint e testes do frontend, depois Stryker incremental. Orçamento de 1200 segundos. Exit 2 significa que o agente não tem permissão para parar.</p><p><strong>O </strong><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml"><strong>gate de pull request</strong></a> roda no GitHub. Executa o verify de novo, depois aplica dois scripts aos relatórios do Pitest: a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">catraca de mutação</a>, e uma <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">verificação</a> de que todo mutante sobrevivente numa linha adicionada pelo PR está justificado na descrição. Também roda as suítes de teste dos próprios hooks. É o único status que a proteção de branch exige. (A parte de mutação é a Parte 3 desta série.)</p><p>A regra que os três compartilham está na <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constituição</a>: silêncio no sucesso, limite na falha. O Gradle roda com --quiet, o log de testes mostra só eventos FAILED (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-base.gradle.kts">kotlin-base.gradle.kts</a>), o npm roda com loglevel=error, NO_COLOR=1 (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/settings.json">settings.json</a>). No sucesso, o agente não vê nada. Na falha, vê isto:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*roPNXwypOYRtG62XhiIcOQ.png" /></figure><p>Um cabeçalho com o gate e a verificação, o comando exato num formato que dá para colar, e no máximo quarenta linhas de saída por verificação, com a contagem de quantas foram cortadas. Verificações que nem rodaram porque o orçamento acabou são listadas pelo nome. Esse formato é um requisito funcional da <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/spec.md">feature 001</a>, com <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/test-failure-format.sh">testes</a> que o verificam, o que parece exagero até você lembrar que o leitor é um modelo com contexto finito e sem capacidade de rolar para cima.</p><h3>Por dentro do hook de Stop</h3><p>O hook de Stop é o que paga o próprio salário, então eis o que acontece quando o agente diz “pronto”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*hs-VIs8H35Y6HNyZMg67KA.png" /></figure><p><strong>Pular se nada relevante mudou.</strong> Um arquivo marcador, .claude/.cache/last-full-check, só é escrito depois que toda verificação passou. No próximo stop, se nenhum arquivo .kt, .kts, .ts, .tsx, .toml, .json, .feature, .yml ou .sql for mais novo que o marcador, o hook sai em menos de dois segundos. Isso aconteceu 33 vezes. Sem isso, cada &quot;deixa eu só arrumar aquele comentário também&quot; custaria cinco minutos.</p><p><strong>Bypass, mas deixando rastro.</strong> HOOK_BYPASS=1 pula o gate uma vez e acrescenta uma linha em .claude/.cache/bypass.log: timestamp, gate, usuário, arquivo, motivo. O log é ignorado pelo git, mas o <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">template de PR</a> tem uma seção obrigatória &quot;Gate bypasses&quot; que precisa conter none ou as linhas coladas, e o gate de PR falha se a seção estiver faltando. Em dez dias o bypass foi usado uma vez, com o motivo &quot;quickstart rehearsal (T080), not a real bypass&quot;. O mecanismo existe; também é visível.</p><p><strong>Rodar tudo, dividindo um relógio só.</strong> Cada verificação recebe o que sobrou dos 1200 segundos. Uma que estoura é morta e reportada como TIMEOUT after N s; nada fica rodando em segundo plano além do daemon do Gradle, que foi feito para morrer de inatividade.</p><p><strong>Nunca bloquear duas vezes seguidas.</strong> Essa é a parte que eu copiaria para qualquer projeto. O Claude Code manda ao hook um JSON de entrada com um campo chamado stop_hook_active, verdadeiro quando o hook já bloqueou neste turno. A plataforma tem sua própria rede de segurança: ela &quot;sobrepõe um hook de Stop depois que ele bloqueia oito vezes seguidas sem nenhuma chamada de ferramenta do Claude no meio&quot; (<a href="https://code.claude.com/docs/en/hooks-guide">Hooks guide</a>). O hook deste repositório é mais rigoroso. Na segunda tentativa, imprime o mesmo relatório como systemMessage e sai com 0, de modo que o agente vê a falha mas tem permissão para parar. Um bloqueio, uma chance de consertar, depois o humano é avisado. O teste disso, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/test-stop-loop-guard.sh">test-stop-loop-guard.sh</a>, tem vinte linhas e cobre os quatro estados.</p><p><strong>Caso contrário, bloquear.</strong> Exit 2, relatório no stderr. Noventa e cinco vezes.</p><h3>O que o log diz</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*kwTGcju6Lx1X2HWdyBuPww.png" /></figure><p>Cada ponto é uma execução do hook de Stop: vermelho foi bloqueado, verde passou, cinza pulou. Algumas coisas saltam aos olhos.</p><p>Os dois primeiros dias são vermelhos. Em 2 e 3 de outubro, enquanto as features 001 a 004 eram construídas, o gate bloqueou 62 vezes e passou 13. A noite de 3 de outubro tem uma sequência de 19 bloqueios consecutivos começando às 19:08 UTC e só terminando depois da meia-noite: horas em que o agente disse “pronto”, ouviu não, consertou algo, disse “pronto”, e ouviu não de novo. Eu não estava olhando na maior parte do tempo. É para isso que a trava de loop existe: cada um desses bloqueios foi uma recusa única seguida de um systemMessage, não uma armadilha.</p><p>A partir de 8 de outubro a imagem vira. A loja estava no lugar, a suíte de testes tinha assentado, e em 8 de outubro o gate passou 19 vezes seguidas, cada execução levando cerca de cinco minutos. O gate não tinha ficado mais brando. O código tinha ficado melhor em passar por ele.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*GZINAGmwDSfXwpzSI5ioCQ.png" /></figure><p>Um veredicto vermelho é mais rápido que um verde: mediana de 142 segundos contra 293. É o Gradle parando no primeiro módulo que falha, enquanto um passe precisa rodar todos, e isso significa que o caminho caro é o que você quer. A única execução acima do orçamento, 1369 segundos em 7 de outubro no meio da feature 005, foi um timeout; a execução seguinte, 886 segundos, passou.</p><p>A <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/spec.md">feature 001</a> escreveu critérios de sucesso para tudo isso. SC-001: o percentil 90 do gate por arquivo abaixo de 60 segundos. Medido: 50 segundos, em 22 execuções que passaram. SC-002: o gate completo tipicamente abaixo de 15 minutos. Medido: mediana de 293 segundos, percentil 90 de 515. SC-007: saída de build e testes abaixo de 10 % do contexto de uma sessão. Não mensurável a partir deste log, porque no sucesso não há saída nenhuma, que é o design.</p><p>Tempo total gasto dentro dos gates em dez dias: 13,2 horas. Guarde esse número; ele volta nas críticas.</p><h3>Os hooks têm testes</h3><p>Este é o detalhe que me fez confiar no resto. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks/tests">.claude/hooks/tests/</a> tem 23 scripts de teste e um monorepo de fixture com binários falsos de gradlew, npm e npx que devolvem o exit code e a saída que o teste pedir. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/run-all.sh">run-all.sh</a> é silencioso no sucesso e imprime os nomes dos testes que falharam. O gate de PR o roda em todo pull request, num segundo runner pequeno para não entrar na fila atrás do Gradle.</p><pre># test-stop-loop-guard.sh — never two blocks in a row<br>run_hook stop-full-check.sh &quot;$(stop_json false)&quot;<br>assert_exit 2 &quot;first attempt blocks&quot;<br>assert_contains stderr &quot;stop check FAILED: gradle:verify&quot;<br>run_hook stop-full-check.sh &quot;$(stop_json true)&quot;<br>assert_exit 0 &quot;stop_hook_active=true must not block&quot;<br>jq -e &#39;.systemMessage | contains(&quot;stop check FAILED: gradle:verify&quot;)&#39; &quot;$T_OUT/stdout&quot; &gt;/dev/null</pre><p>Há testes para o orçamento, para o dry run, para a linha do log de bypass, para o limite de 40 linhas, para o marcador (N earlier lines omitted), para um Task &#39;verify&#39; not found do Gradle ser tratado como skip e não como falha (para que os hooks funcionem num repositório anterior ao build), e para execuções concorrentes do gate por arquivo não pisarem uma na outra. Todas as 1.349 linhas de shell dos <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks">hooks</a> e dos <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.github/scripts">scripts dos gates</a> passam no shellcheck -S warning.</p><p>Um harness é software. Se não tem testes, é o único componente do sistema que ninguém verifica.</p><h3>O gate de PR</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*nsFHNZ0i4jZ0BZgSGxBHEA.png" /></figure><p>Três jobs alimentam um status (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">pr-gate.yml</a>). O verify (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/verify.yml">verify.yml</a>) refaz o build completo e entrega os relatórios do Pitest ao mutation, que aplica a catraca e a regra dos mutantes sobreviventes. O hook-tests roda as suítes acima. O pr-gate tem if: always() e nenhum checkout: falha se qualquer job falhou <em>ou foi pulado</em>, porque o GitHub conta uma verificação obrigatória pulada como aprovada, e pull requests de fork pulam todo job que faz checkout de código. Também falha quando a descrição não tem a seção &quot;Gate bypasses&quot; preenchida.</p><p>As salvaguardas vêm das <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">notas de pesquisa do MVP, seção 4</a>, escritas porque o runner executa código de pull request: só pull_request (nunca pull_request_target), token somente leitura, nenhum segredo no runner, actions fixadas por SHA de commit, um runner efêmero por job. Nada disso é específico de agentes. Tudo isso importa mais quando é um agente que abre os pull requests.</p><h3>O caso contra, e o que o log não consegue dizer</h3><p>Disse na Parte 1 que incluiria os críticos toda vez. Eis os do harness.</p><p><strong>É caro, e dá para pôr um número.</strong> 13,2 horas de gate em dez dias, num laptop. Cada “pronto” custa cinco minutos de Gradle quando passa. O gate por arquivo acrescenta 47 segundos a cada edição de Kotlin, o que o próprio <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a> nota estar “dentro da meta de 60 s do SC-001, com pouca folga”. A <a href="https://code.claude.com/docs/en/hooks">referência de hooks</a> da Anthropic avisa que um hook “que roda antes de cada prompt e bloqueia o processamento do modelo até terminar” trava a sessão; o mesmo vale para um hook de Stop que leva cinco minutos. Aceitei essa troca. Um time de seis sentiria de outro jeito.</p><p><strong>Ficar dando voltas é real.</strong> Dezenove bloqueios seguidos em 3 de outubro é o agente em círculos. A trava de loop evita a armadilha, mas não impede o agente de dizer “pronto” de novo um minuto depois. O <a href="https://www.martinfowler.com/articles/harness-engineering.html">artigo</a> de Böckeler levanta uma preocupação parecida: um harness construído em torno de testes “deposita muita fé nos testes gerados por IA, e isso ainda não é bom o suficiente”. Os sensores deste repositório são testes que o agente escreveu. Teste de mutação é minha resposta a isso, e a Parte 3 é sobre quão boa é essa resposta. Não é completa.</p><p><strong>Sensores que disparam muito podem estar mal calibrados, e sensores que nunca disparam podem estar cegos.</strong> Essa segunda metade também é de <a href="https://www.martinfowler.com/articles/harness-engineering.html">Böckeler</a>: um sensor que nunca dispara “pode sinalizar detecção fraca em vez de alta qualidade”, e a área “ainda não tem como avaliar a cobertura de um harness”. Posso dizer que meu hook de Stop disparou 95 vezes. Não posso dizer quantas mudanças ruins ele não pegou. Não existe log para isso.</p><p><strong>Documentação não basta, e um hook também não.</strong> O relato da OpenAI sobre construir um milhão de linhas com o Codex é a evidência de campo mais forte a favor do harness: “Documentação sozinha não mantém coerente uma base de código inteiramente gerada por agentes”, e as restrições de arquitetura “são impostas mecanicamente por linters customizados (gerados pelo Codex, claro!) e testes estruturais” (<a href="https://openai.com/index/harness-engineering/">Harness engineering</a>). Lido com atenção, é também um aviso. A base de código deles ainda assim desviou (“Com o tempo, isso inevitavelmente leva a desvio”), então eles rodam agentes em segundo plano que “varrem desvios, atualizam notas de qualidade e abrem pull requests de refatoração direcionados”, além de um agente de “jardinagem de docs” para documentação obsoleta. “Isso funciona como coleta de lixo”, escrevem. Este repositório não teve nenhuma passagem dessas. Os dois PRs sem spec da Parte 1 são o primeiro desvio, e nada no harness perceberia.</p><p><strong>O harness agora é um produto.</strong> 1.349 linhas de shell, 23 testes, um monorepo de fixture, dois arquivos de workflow, um template de PR com quatro seções obrigatórias. A frase da Thoughtworks sobre spec-driven development se aplica aqui trocando uma palavra: “escrever regras detalhadas à mão para a IA, no fim, não escala” (<a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">Technology Radar</a>). Böckeler nota que templates de harness compartilhados “enfrentariam os mesmos problemas de versionamento e contribuição” de qualquer outro código compartilhado. O meu está preso a um repositório, uma ferramenta de build, o protocolo de hooks de um agente. No dia em que o Claude Code mudar a forma do <a href="https://code.claude.com/docs/en/hooks-guide">stop_hook_active</a>, os testes vão me avisar, o que é bom; ainda assim vou ter que consertar.</p><p><strong>O gate por arquivo está submedido.</strong> O log tem 24 linhas de gate por arquivo contra centenas de edições. Não sei por quê. O <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/post-edit-check.sh">hook</a> ignora arquivos em .claude/, .specify/ e build/, e não faz nada com Markdown, mas isso não explica a diferença. Pode ser que a maioria das edições tenha acontecido dentro de subagentes cuja saída de hook não foi registrada no mesmo arquivo. Até eu descobrir, os números por arquivo acima descrevem 24 execuções, não o projeto.</p><p><strong>E, de novo, nenhuma comparação.</strong> Não construí esta plataforma sem hooks. Não posso dizer que o gate economizou tempo, só que recusou 95 vezes e que as recusas ficaram mais raras conforme o código amadureceu. Os números da METR da Parte 1 continuam sendo os únicos dados controlados sobre desenvolvimento assistido por agentes, e não cobrem hooks.</p><h3>O que eu manteria</h3><ol><li><strong>Silêncio no sucesso.</strong> Zero linhas quando está verde. O contexto do agente é o recurso mais escasso da sala.</li><li><strong>Limite na falha.</strong> Um cabeçalho, o comando exato, quarenta linhas. Transforme o relatório numa especificação e teste-o.</li><li><strong>Um arquivo marcador.</strong> Pule o gate completo quando nada relevante mudou. 33 de 220 execuções levaram menos de dois segundos por causa de um touch.</li><li><strong>Bloqueie uma vez, depois avise o humano.</strong> Leia o <a href="https://code.claude.com/docs/en/hooks-guide">stop_hook_active</a>. Uma recusa, um systemMessage, nunca um loop.</li><li><strong>Registre toda execução.</strong> Uma linha por gate com segundos e status. Este artigo existe por causa dessa linha.</li><li><strong>Bypass com recibo.</strong> Permita, registre, faça o PR carregar.</li><li><strong>Teste o harness.</strong> Falsifique as ferramentas, rode os hooks, verifique o relatório. Rode esses testes no CI.</li></ol><h3>Para ir mais fundo</h3><p>O resto da série, uma peça de cada vez:</p><ul><li><strong>Parte 1 — Sete specs, 620 tarefas.</strong> Spec-driven development testado numa plataforma real de microsserviços: o ciclo, clarify, converge, e os dados que ninguém tem. <a href="https://medium.com/@raphaelpanta/testei-spec-driven-development-numa-plataforma-real-de-microsservi%C3%A7os-3f72c69b308a">Leia a Parte 1 →</a></li><li><strong>Parte 3–3.380 bugs de propósito.</strong> Teste de mutação como critério de merge: Pitest e Stryker a 80 %, sobreviventes justificados no PR. <a href="https://medium.com/@raphaelpanta/plantei-3-380-bugs-no-meu-pr%C3%B3prio-c%C3%B3digo-os-testes-deixaram-passar-273-146993812781">Leia a Parte 3 →</a></li><li><strong>Parte 4 — Um número em vez de uma spec.</strong> /goal para performance: AOT cache do JDK 25, compact object headers, imagens nativas, de 4,3 GiB a 1,9. <a href="https://medium.com/@raphaelpanta/dei-ao-agente-um-n%C3%BAmero-em-vez-de-uma-spec-ele-cortou-a-mem%C3%B3ria-da-plataforma-de-4-3-gib-para-1-9-9ce25059da5e">Leia a Parte 4 →</a></li><li><strong>Parte 5 — O Mac estava mentindo.</strong> /goal para consertar: SELinux, virtiofs, um timestamp de build e um runner self-hosted que enfim funcionou no Linux. <a href="https://medium.com/@raphaelpanta/meu-runner-de-ci-funcionava-no-meu-mac-o-mac-estava-mentindo-02e8e8ac6a9c">Leia a Parte 5 →</a></li><li><strong>Parte 6 — Uma constituição para um agente de IA.</strong> Oito princípios, 55 MUSTs, uma verificação que roda duas vezes por feature, e o PASS que se revelou falso. <a href="https://medium.com/@raphaelpanta/escrevi-uma-constitui%C3%A7%C3%A3o-para-um-agente-de-ia-eis-o-que-ela-conseguiu-e-o-que-n%C3%A3o-conseguiu-impor-31332eadc192">Leia a Parte 6 →</a></li></ul><h3>Leituras</h3><ul><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com, 2 abr 2026</a></li><li>Ryan Lopopolo, <em>Harness engineering: leveraging Codex in an agent-first world</em> — <a href="https://openai.com/index/harness-engineering/">OpenAI, 11 fev 2026</a></li><li><em>Automate actions with hooks</em> e <em>Hooks reference</em> — <a href="https://code.claude.com/docs/en/hooks-guide">code.claude.com/docs/en/hooks-guide</a>, <a href="https://code.claude.com/docs/en/hooks">code.claude.com/docs/en/hooks</a></li><li><em>Best practices for Claude Code</em> — <a href="https://code.claude.com/docs/en/best-practices">code.claude.com/docs/en/best-practices</a></li><li>Anthropic, <em>Effective context engineering for AI agents</em> — <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">anthropic.com/engineering, 29 set 2025</a></li><li>Thoughtworks Technology Radar, <em>Spec-driven development</em> (Assess, nov 2025) — <a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">thoughtworks.com/radar</a></li><li>W. Ross Ashby, <em>An Introduction to Cybernetics</em> (1956), a lei da variedade requisito — <a href="https://en.wikipedia.org/wiki/Variety_(cybernetics)">Wikipedia</a></li><li>Edsger W. Dijkstra, <em>Notes on Structured Programming</em> (1970), EWD249 — <a href="https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF">cs.utexas.edu</a></li><li>O repositório: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks">.claude/hooks/</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/settings.json">.claude/settings.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">.github/workflows/pr-gate.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.github/scripts">.github/scripts/</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/specs/001-harness-quality-gates">specs/001-harness-quality-gates</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constituição, Princípio VIII</a></li></ul><p><em>Os números deste artigo vêm do </em><em>.claude/.cache/gate-timing.log na máquina de desenvolvimento (macOS, Apple Silicon, daemon do Gradle aquecido), 244 linhas entre 2026-10-02 e 2026-10-11, e do repositório no commit </em><em>9870792. O log é ignorado pelo git; o formato está documentado em </em><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md"><em>docs/harness.md</em></a><em>, em &quot;Measuring the success criteria&quot;.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eab038b4eafc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My AI Agent Said “Done” 220 Times. A Shell Script Said No to 95 of Them.]]></title>
            <link>https://medium.com/@raphaelpanta/my-ai-agent-said-done-220-times-a-shell-script-said-no-to-95-of-them-d60338ec8579?source=rss-4fe4510de473------2</link>
            <guid isPermaLink="false">https://medium.com/p/d60338ec8579</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[kotlin]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Raphael Pantaleão]]></dc:creator>
            <pubDate>Wed, 14 Oct 2026 11:18:29 GMT</pubDate>
            <atom:updated>2026-10-16T14:05:34.573Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Harness engineering on a Kotlin microservices monorepo: three gates, a hook that refuses to let the agent finish, and ten days of real timing data.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*IvvoWUp6FxuukT2tXdJY2w.png" /></figure><blockquote><em>“Program testing can be used to show the presence of bugs, but never to show their absence!” — Edsger W. Dijkstra, </em>Notes on Structured Programming<em> (1970)</em></blockquote><p><a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Part 1</a> of this series was about the specs: seven of them, 620 tasks, one e-commerce platform built from roadmap.sh’s brief. I ended it with a confession. Reading OpenAI’s account of a million agent-written lines, I had to admit that the parts of my repository doing the most work were probably not the specs. They were the hooks.</p><p>This part is about those hooks. Not the idea of them; the log. The repository keeps a file called gate-timing.log (git-ignored; format in <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>) with one line per gate run, and between 2 and 11 October it collected 244 of them. The end-of-task gate ran 220 times. It let the agent stop 92 times and refused 95 times.</p><p>That number is the whole article. A quality gate that passes every time is decoration. One that fails more often than it passes is doing something.</p><h3>What “harness” means here</h3><p>The word has spread fast enough to mean almost anything, so here is the definition I am using, from Birgitta Böckeler’s April 2026 piece on martinfowler.com: “The term harness has emerged as a shorthand to mean everything in an AI agent except the model itself” (<a href="https://www.martinfowler.com/articles/harness-engineering.html">Harness engineering for coding agent users</a>). She splits that everything into two kinds of control, borrowed from cybernetics:</p><ul><li><strong>Guides</strong> (feedforward) “anticipate the agent’s behaviour and aim to steer it <em>before</em> it acts.” Instructions, templates, scaffolding, constitutions.</li><li><strong>Sensors</strong> (feedback) “observe <em>after</em> the agent acts and help it self-correct.” Compilers, linters, tests, hooks that block.</li></ul><p>And she borrows a law to go with it. Ashby’s law of requisite variety: “a regulator must have at least as much variety as the system it governs.” A page of Markdown rules cannot regulate 29,000 lines of Kotlin. Something with the same reach as the code has to look at the code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*d8EDiE31DTIcolb8W2LOVw.png" /></figure><p>That is the matrix with this repository’s parts placed in it. The weight is bottom-left: deterministic sensors. The constitution and the specs from Part 1 live in the top-left cell, and they matter, but they are advice. The bottom-left cell is where advice becomes a wall.</p><p>Anthropic’s own documentation draws the same line. CLAUDE.md is “advisory”; hooks give “deterministic control: certain actions always happen rather than relying on the LLM to choose to run them” (<a href="https://code.claude.com/docs/en/hooks-guide">Automate actions with hooks</a>). The best-practices page goes further and tells you what to do with a rule the agent keeps ignoring: “If Claude already does something correctly without the instruction, delete it or convert it to a hook” (<a href="https://code.claude.com/docs/en/best-practices">Best practices</a>).</p><h3>Three gates</h3><p>The constitution’s eighth principle required three of them before any service was written, and <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/specs/001-harness-quality-gates">feature 001</a> built them. Each has a trigger, a time budget, and a cap on how much it may say.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*IOWQr0CQGUJucVHPEKuERw.png" /></figure><p><strong>The </strong><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/post-edit-check.sh"><strong>per-file gate</strong></a> fires after every Write or Edit. For a Kotlin file it runs ktlint and detekt on the file&#39;s Gradle module, then every test class whose name starts with the file&#39;s name (Price* → PriceSpec, PricePropertyTest), at most twenty. For TypeScript: eslint, tsc, prettier if there is a config, and vitest related, which runs only the tests that import the file. Budget 300 seconds, report cap 60 lines, exit 2 on failure, which blocks the agent from moving on.</p><p><strong>The end-of-task gate</strong> fires on Stop, when the agent believes it is done. It runs ./gradlew -q verify, which is the whole repository&#39;s check plus Pitest on every domain and application module, then the frontend lint and tests, then Stryker incrementally. Budget 1200 seconds. Exit 2 means the agent is not allowed to stop.</p><p><strong>The </strong><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml"><strong>pull-request gate</strong></a> runs on GitHub. It reruns verify, then applies two scripts to the Pitest reports: the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/mutation-gate.sh">mutation ratchet</a>, and a <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/scripts/surviving-mutants.sh">check</a> that every surviving mutant on a line added by the PR is justified in the description. It also runs the hooks&#39; own test suites. It is the single status branch protection requires. (The mutation part is Part 3 of this series.)</p><p>The rule all three share is in the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constitution</a>: quiet on success, bounded on failure. Gradle runs with --quiet, test logging shows only FAILED events (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/build-logic/src/main/kotlin/kotlin-base.gradle.kts">kotlin-base.gradle.kts</a>), npm runs with loglevel=error, NO_COLOR=1 (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/settings.json">settings.json</a>). On success the agent sees nothing at all. On failure it sees this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*X0i7O_uHMyQMVwfEBoaSbA.png" /></figure><p>A header naming the gate and the check, the exact command in a form you can paste, and at most forty lines of output per check, with a count of how many were cut. Checks that never ran because the budget was spent are listed by name. This format is a functional requirement of <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/spec.md">feature 001</a> with <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/test-failure-format.sh">tests</a> asserting on it, which sounds like overkill until you remember that the reader is a model with a finite context and no ability to scroll up.</p><h3>Inside the Stop hook</h3><p>The Stop hook is the one that earns its keep, so here is what happens when the agent says “done”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*L5aQpZ8xrmdu7nM-rx6nng.png" /></figure><p><strong>Skip if nothing relevant changed.</strong> A marker file, .claude/.cache/last-full-check, is written only after every check passed. On the next stop, if no .kt, .kts, .ts, .tsx, .toml, .json, .feature, .yml or .sql file is newer than the marker, the hook exits in under two seconds. This happened 33 times. Without it, every &quot;let me also fix that comment&quot; would cost five minutes.</p><p><strong>Bypass, but leave a trace.</strong> HOOK_BYPASS=1 skips the gate once and appends a line to .claude/.cache/bypass.log: timestamp, gate, user, file, reason. The log is git-ignored, but the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/PULL_REQUEST_TEMPLATE.md">PR template</a> has a mandatory &quot;Gate bypasses&quot; section that must contain either none or the pasted lines, and the PR gate fails if the section is missing. In ten days the bypass was used once, with the reason &quot;quickstart rehearsal (T080), not a real bypass&quot;. The mechanism is there; it is also visible.</p><p><strong>Run everything, sharing one clock.</strong> Each check gets whatever is left of the 1200 seconds. One that overruns is killed and reported as TIMEOUT after N s; nothing is left running in the background except the Gradle daemon, which is meant to idle out.</p><p><strong>Never block twice in a row.</strong> This is the part I would copy into any project. Claude Code sends the hook a JSON input with a field called stop_hook_active, true when the hook already blocked this turn. The platform has its own safety net: it &quot;overrides a Stop hook after it blocks eight times in a row with no tool call from Claude in between&quot; (<a href="https://code.claude.com/docs/en/hooks-guide">Hooks guide</a>). This repository&#39;s hook is stricter. On the second attempt it prints the same report as a systemMessage and exits 0, so the agent sees the failure but is allowed to stop. One block, one chance to fix it, then the human is told. The test for this, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/test-stop-loop-guard.sh">test-stop-loop-guard.sh</a>, is twenty lines and checks all four states.</p><p><strong>Otherwise, block.</strong> Exit 2, report on stderr. Ninety-five times.</p><h3>What the log says</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*QFkRbM0-J0F-o5YF0R7W_A.png" /></figure><p>Every dot is one run of the Stop hook: red was blocked, green passed, grey skipped. A few things stand out.</p><p>The first two days are red. On 2 and 3 October, while features 001 to 004 were being built, the gate blocked 62 times and passed 13. The evening of 3 October has a run of 19 consecutive blocks starting at 19:08 UTC and not ending until after midnight: hours in which the agent said “done”, was told no, fixed something, said “done”, and was told no again. I was not watching for most of it. That is what the loop guard is for: each of those blocks was a single refusal followed by a systemMessage, not a trap.</p><p>From 8 October the picture flips. The storefront was in place, the test suite had settled, and on 8 October the gate passed 19 times in a row, each run taking about five minutes. The gate had not got softer. The code had got better at passing it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*FoDPxVYg1DKdFO9ZxutWPw.png" /></figure><p>A red verdict is faster than a green one: median 142 seconds against 293. That is Gradle stopping at the first failing module while a pass has to run all of them, and it means the expensive path is the one you want. The one run above the budget, 1369 seconds on 7 October in the middle of feature 005, was a timeout; the next run, 886 seconds, passed.</p><p><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/001-harness-quality-gates/spec.md">Feature 001</a> wrote success criteria for all this. SC-001: the per-file gate’s 90th percentile under 60 seconds. Measured: 50 seconds, from 22 passing runs. SC-002: the full gate typically under 15 minutes. Measured: median 293 seconds, 90th percentile 515. SC-007: build and test output under 10 % of a session’s context. Not measurable from this log, because on success there is no output at all, which is the design.</p><p>Total time spent inside gates across ten days: 13.2 hours. Hold that number; it comes back in the criticism.</p><h3>The hooks have tests</h3><p>This is the detail that made me trust the rest. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks/tests">.claude/hooks/tests/</a> has 23 test scripts and a fixture monorepo with fake gradlew, npm and npx binaries that return whatever exit code and output the test asks for. <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/tests/run-all.sh">run-all.sh</a> is silent on success and prints the names of failing tests. The PR gate runs it on every pull request, on a small second runner so it does not queue behind Gradle.</p><pre># test-stop-loop-guard.sh — never two blocks in a row<br>run_hook stop-full-check.sh &quot;$(stop_json false)&quot;<br>assert_exit 2 &quot;first attempt blocks&quot;<br>assert_contains stderr &quot;stop check FAILED: gradle:verify&quot;<br>run_hook stop-full-check.sh &quot;$(stop_json true)&quot;<br>assert_exit 0 &quot;stop_hook_active=true must not block&quot;<br>jq -e &#39;.systemMessage | contains(&quot;stop check FAILED: gradle:verify&quot;)&#39; &quot;$T_OUT/stdout&quot; &gt;/dev/null</pre><p>There are tests for the budget, for the dry run, for the bypass log line, for the 40-line cap, for the (N earlier lines omitted) marker, for a Gradle Task &#39;verify&#39; not found being treated as a skip rather than a failure (so the hooks work in a repository that predates the build), and for concurrent per-file runs not stepping on each other. All 1,349 lines of shell across the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks">hooks</a> and the <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.github/scripts">gate scripts</a> pass shellcheck -S warning.</p><p>A harness is software. If it has no tests, it is the one component of the system that nobody checks.</p><h3>The PR gate</h3><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*a1rUxVq-zOhq0qT6QKYZqg.png" /></figure><p>Three jobs feed one status (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">pr-gate.yml</a>). verify (<a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/verify.yml">verify.yml</a>) reruns the full build and hands the Pitest reports to mutation, which applies the ratchet and the surviving-mutant rule. hook-tests runs the suites above. pr-gate has if: always() and no checkout: it fails if any job failed <em>or was skipped</em>, because GitHub counts a skipped required check as passed, and fork pull requests skip every job that checks out code. It also fails when the description lacks a filled &quot;Gate bypasses&quot; section.</p><p>The safeguards come from the MVP’s <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/specs/004-ecommerce-platform-mvp/research.md">research notes, section 4</a>, written because the runner executes pull-request code: pull_request only (never pull_request_target), a read-only token, no secrets on the runner, actions pinned by commit SHA, an ephemeral runner per job. None of that is specific to agents. All of it matters more when an agent is opening the pull requests.</p><h3>The case against, and what the log can’t tell you</h3><p>I said in Part 1 that I would include the critics each time. Here they are for the harness.</p><p><strong>It is expensive, and I can put a number on it.</strong> 13.2 hours of gate time in ten days, on a laptop. Every “done” costs five minutes of Gradle when it passes. The per-file gate adds 47 seconds to every Kotlin edit, which <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a> itself notes is “inside the 60 s target of SC-001 with little headroom”. Anthropic’s <a href="https://code.claude.com/docs/en/hooks">hooks reference</a> warns that a hook “that runs before every prompt and blocks model processing until it completes” will stall the session; the same is true of a Stop hook that takes five minutes. I accepted that trade. A team of six would feel it differently.</p><p><strong>Thrashing is real.</strong> Nineteen blocks in a row on 3 October is the agent circling. The loop guard prevents a trap, but it does not prevent the agent from saying “done” again a minute later. Böckeler’s <a href="https://www.martinfowler.com/articles/harness-engineering.html">article</a> raises a related worry: a harness built around tests “puts a lot of faith into the AI-generated tests, that’s not good enough yet.” The sensors in this repository are tests the agent wrote. Mutation testing is my answer to that, and Part 3 is about how good an answer it is. It is not a complete one.</p><p><strong>Sensors that fire a lot may be badly tuned, and sensors that never fire may be blind.</strong> That second half is <a href="https://www.martinfowler.com/articles/harness-engineering.html">Böckeler’s</a> too: a sensor that never trips “may signal weak detection rather than high quality”, and the field “still lacks a way to evaluate harness coverage”. I can tell you my Stop hook fired 95 times. I cannot tell you how many bad changes it did not catch. There is no log for that.</p><p><strong>Documentation is not enough, and neither is a hook.</strong> OpenAI’s account of building a million lines with Codex is the strongest field evidence for the harness-first view: “Documentation alone doesn’t keep a fully agent-generated codebase coherent,” and architectural constraints “are enforced mechanically via custom linters (Codex-generated, of course!) and structural tests” (<a href="https://openai.com/index/harness-engineering/">Harness engineering</a>). Read carefully, it is also a warning. Their codebase still drifted (“Over time, this inevitably leads to drift”), so they run background agents that “scan for deviations, update quality grades, and open targeted refactoring pull requests”, plus a “doc-gardening” agent for stale documentation. “This functions like garbage collection,” they write. This repository has had no such pass. The two spec-less PRs from Part 1 are the first drift, and nothing in the harness would notice it.</p><p><strong>The harness is now a product.</strong> 1,349 lines of shell, 23 tests, a fixture monorepo, two workflow files, a PR template with four mandatory sections. Thoughtworks’ line about spec-driven development applies here with the word changed: “handcrafting detailed rules for AI ultimately doesn’t scale” (<a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">Technology Radar</a>). Böckeler notes that shared harness templates “would face the same versioning and contribution problems” as any other shared code. Mine is tied to one repository, one build tool, one agent’s hook protocol. The day Claude Code changes the shape of <a href="https://code.claude.com/docs/en/hooks-guide">stop_hook_active</a>, the tests will tell me, which is good; I will still have to fix it.</p><p><strong>The per-file gate is under-measured.</strong> The log has 24 per-file rows against hundreds of edits. I do not know why. The <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/hooks/post-edit-check.sh">hook</a> ignores files under .claude/, .specify/ and build/, and does nothing for Markdown, but that does not account for the gap. It may be that most edits happened inside subagents whose hook output was not logged to the same file. Until I find out, the per-file numbers above describe 24 runs, not the project.</p><p><strong>And, again, no comparison.</strong> I did not build this platform without hooks. I cannot tell you the gate saved time, only that it refused 95 times and that the refusals got rarer as the code matured. The METR numbers from Part 1 are still the only controlled data on agent-assisted development, and they do not cover hooks at all.</p><h3>What I would keep</h3><ol><li><strong>Quiet on success.</strong> Zero lines when green. The agent’s context is the scarcest resource in the room.</li><li><strong>Bounded on failure.</strong> A header, the exact command, forty lines. Make the report a specification and test it.</li><li><strong>A marker file.</strong> Skip the full gate when nothing relevant changed. 33 of 220 runs took under two seconds because of one touch.</li><li><strong>Block once, then tell the human.</strong> Read <a href="https://code.claude.com/docs/en/hooks-guide">stop_hook_active</a>. One refusal, one systemMessage, never a loop.</li><li><strong>Log every run.</strong> One line per gate with seconds and status. This article exists because of that line.</li><li><strong>Bypass with a receipt.</strong> Allow it, log it, make the PR carry it.</li><li><strong>Test the harness.</strong> Fake the tools, run the hooks, assert on the report. Run those tests in CI.</li></ol><h3>Go deeper</h3><p>The rest of the series, one piece at a time:</p><ul><li><strong>Part 1 — Seven specs, 620 tasks.</strong> Spec-driven development tested on a real microservices platform: the loop, clarify, converge, and the data nobody has. <a href="https://medium.com/@raphaelpanta/i-tested-spec-driven-development-on-a-real-microservices-platform-cda61ccd9fa6">Read Part 1 →</a></li><li><strong>Part 3–3,380 bugs on purpose.</strong> Mutation testing as the merge criterion: Pitest and Stryker at 80 %, survivors justified in the PR. <a href="https://medium.com/@raphaelpanta/i-planted-3-380-bugs-in-my-own-codebase-the-tests-missed-273-fc0ae0ec8fe8">Read Part 3 →</a></li><li><strong>Part 4 — A number instead of a spec.</strong> /goal for performance: JDK 25 AOT cache, compact object headers, native images, 4.3 GiB to 1.9. <a href="https://medium.com/@raphaelpanta/i-gave-the-agent-a-number-instead-of-a-spec-it-cut-the-platforms-memory-from-4-3-gib-to-1-9-ceb2f0b4da66">Read Part 4 →</a></li><li><strong>Part 5 — The Mac was lying.</strong> /goal for fixing: SELinux, virtiofs, a build-time timestamp, and a self-hosted runner that finally worked on Linux. <a href="https://medium.com/@raphaelpanta/my-ci-runner-worked-on-my-mac-the-mac-was-lying-5eece22825ce">Read Part 5 →</a></li><li><strong>Part 6 — A constitution for an AI agent.</strong> Eight principles, 55 MUSTs, a check that runs twice per feature, and the PASS that turned out to be false. <a href="https://medium.com/@raphaelpanta/i-wrote-a-constitution-for-an-ai-agent-heres-what-it-could-and-couldn-t-enforce-da0e48c3b968">Read Part 6 →</a></li></ul><h3>Further reading</h3><ul><li>Birgitta Böckeler, <em>Harness engineering for coding agent users</em> — <a href="https://www.martinfowler.com/articles/harness-engineering.html">martinfowler.com, 2 Apr 2026</a></li><li>Ryan Lopopolo, <em>Harness engineering: leveraging Codex in an agent-first world</em> — <a href="https://openai.com/index/harness-engineering/">OpenAI, 11 Feb 2026</a></li><li><em>Automate actions with hooks</em> and <em>Hooks reference</em> — <a href="https://code.claude.com/docs/en/hooks-guide">code.claude.com/docs/en/hooks-guide</a>, <a href="https://code.claude.com/docs/en/hooks">code.claude.com/docs/en/hooks</a></li><li><em>Best practices for Claude Code</em> — <a href="https://code.claude.com/docs/en/best-practices">code.claude.com/docs/en/best-practices</a></li><li>Anthropic, <em>Effective context engineering for AI agents</em> — <a href="https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents">anthropic.com/engineering, 29 Sep 2025</a></li><li>Thoughtworks Technology Radar, <em>Spec-driven development</em> (Assess, Nov 2025) — <a href="https://www.thoughtworks.com/radar/techniques/spec-driven-development">thoughtworks.com/radar</a></li><li>W. Ross Ashby, <em>An Introduction to Cybernetics</em> (1956), the law of requisite variety — <a href="https://en.wikipedia.org/wiki/Variety_(cybernetics)">Wikipedia</a></li><li>Edsger W. Dijkstra, <em>Notes on Structured Programming</em> (1970), EWD249 — <a href="https://www.cs.utexas.edu/~EWD/ewd02xx/EWD249.PDF">cs.utexas.edu</a></li><li>The repository: <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md">docs/harness.md</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.claude/hooks">.claude/hooks/</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.claude/settings.json">.claude/settings.json</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.github/workflows/pr-gate.yml">.github/workflows/pr-gate.yml</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/.github/scripts">.github/scripts/</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/tree/main/specs/001-harness-quality-gates">specs/001-harness-quality-gates</a>, <a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/.specify/memory/constitution.md">constitution, Principle VIII</a></li></ul><p><em>Numbers in this article come from </em><em>.claude/.cache/gate-timing.log on the development machine (macOS, Apple silicon, warm Gradle daemon), 244 rows between 2026-10-02 and 2026-10-11, and from the repository at commit </em><em>9870792. The log is git-ignored; the format is documented in </em><a href="https://github.com/raphaelpanta/scalable-e-commerce-platform-kotlin/blob/main/docs/harness.md"><em>docs/harness.md</em></a><em> under &quot;Measuring the success criteria&quot;.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d60338ec8579" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>