<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Josselin Guarnelli on Medium]]></title>
        <description><![CDATA[Stories by Josselin Guarnelli on Medium]]></description>
        <link>https://medium.com/@JossGuarnelli?source=rss-61c405c54317------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*ZOiSjw7Fpfq_qz2LTg3VQA@2x.jpeg</url>
            <title>Stories by Josselin Guarnelli on Medium</title>
            <link>https://medium.com/@JossGuarnelli?source=rss-61c405c54317------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:43:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@JossGuarnelli/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An official MCP server can delete patient data with no confirmation]]></title>
            <link>https://medium.com/@JossGuarnelli/an-official-mcp-server-can-delete-patient-data-with-no-confirmation-ebf6ff97a49b?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/ebf6ff97a49b</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Tue, 23 Jun 2026 13:12:58 GMT</pubDate>
            <atom:updated>2026-06-23T13:12:58.393Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>I scanned the MCP servers shipped by AWS and LangChain — 772 tools that change the real world. Most of them, a static analyzer can’t even verify. Here’s the honest version of that number.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SeqHsyf7HN-RpGXBPDbBOQ.png" /><figcaption>delete_patient_studies takes no confirmation step. No one is in the chair when it runs.<br> <em>AI-generated illustration (Google Gemini).</em></figcaption></figure><p>In awslabs/mcp — the official Model Context Protocol servers published by AWS — there is a tool called delete_patient_studies.</p><p>It deletes medical imaging studies. It takes no confirmation step. It checks no authorization. Right next to it sits bulk_delete_by_criteria, which deletes in bulk — capped at 100 per call, but with no confirmation and no rate limit between calls.</p><p>An agent wired to that server needs one wrong prompt. Nobody on the team approves the call. Nobody gets asked.</p><p>That tool is not a bug. AWS shipped the capability on purpose, and wiring the guard once after cloning is routine. Fair — but it understates the asymmetry. The guard is cheap; the side effect is not. A missing check on a read costs nothing; he same gap on bulk_delete_by_criteria is irreversible — and an agent can repeat it, 100 deletions at a time, with nobody asked. The real problem isn&#39;t whether someone adds the guard — it&#39;s how far the call reaches when nobody does, and that almost nobody can see whether it&#39;s there.</p><p>So I went looking. I scanned the public MCP servers from AWS and LangChain with a static analyzer that does one thing: it finds the tool calls that touch the real world, and checks whether anything stands in front of them.</p><p>772 tools take real-world actions across those servers. Here’s the part that should bother you: a static analyzer can confirm a guard on almost none of them.</p><h3>What I scanned</h3><p>The tool is diplomat-agent, an open-source AST scanner. It reads Python source, finds functions exposed as MCP tools, and classifies every one that has a side effect — a write, a delete, a subprocess, a network call, an SDK invocation.</p><p>It does not run anything. It does not call an LLM. It reads code and assigns one of four verdicts: <strong>guarded</strong>, <strong>partially guarded</strong>, <strong>unguarded</strong>, or <strong>opaque</strong>.</p><p>The corpus: seven public MCP servers. Five of them expose tools to an agent. The other two don’t expose tools the scanner counts toward the agent-facing surface one proxies to a remote server, the other delegates internally. Commits are pinned at the bottom every number reproduces.</p><h3>772 tools that touch the real world</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KEbb0JIyyeMKrFqCHhyU8Q.png" /><figcaption>772 tools take real-world actions across these servers. A static scan can confirm a guard on almost none of them. <em>AI-generated illustration (Google Gemini).</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*2YBS9OphOAkt_LG5lpmDVA.png" /></figure><p>Two numbers matter here, and they tell opposite stories.</p><p>Nine percent of these tools are <strong>unguarded</strong> — a real, resolved side effect with nothing in front of it. No validation, no rate limit, no confirmation, no record of who said yes.</p><p>Eighty-seven percent are <strong>opaque</strong>. And opaque does not mean dangerous. It means the scanner refused to guess. That distinction is the whole point, so I’ll come back to it.</p><h3>The 9% you can act on today</h3><p>These are the ones to look at first, because the scanner could resolve exactly what they do and saw no guard around it.</p><pre>$ diplomat-agent scan awslabs/mcp --mcp<br><br>  delete_patient_studies        UNGUARDED   (delete; no confirmation, no auth check)<br>  delete_series_by_uid          UNGUARDED   (delete; no confirmation)<br>  bulk_delete_by_criteria       UNGUARDED   (delete; no confirmation, no rate limit)<br>  identity_delete_oauth2_provider  UNGUARDED  (delete)<br>  gateway_delete                UNGUARDED   (delete)<br>  policy_delete                 UNGUARDED   (delete)<br>  suggest_aws_commands          CONTRACT VIOLATION (declared readOnly; writes)<br><br>  636 exposed tools · 58 unguarded · 550 opaque (review) · 28 guarded/partial</pre><p>The medical-imaging deletes are the sharpest example, but the pattern repeats. The threat-modeling server exposes thirteen unguarded delete_* tools — delete a threat, delete a mitigation, delete a trust boundary, delete the whole risk model — none gated. The DataZone server lets an agent add a member to a project with no rate limit and no auth check.</p><p>None of these are exotic. They are the obvious destructive verbs, sitting one tool call away from a model that decides on its own when to use them.</p><h3>The 87% nobody can see</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BDAfcA1CNhQtVtmh7WcG3w.png" /><figcaption>The call is traceable until it enters boto3. After that, no static analyzer can follow it — that&#39;s what &quot;opaque&quot; means. <em>AI-generated illustration (Google Gemini).</em></figcaption></figure><p>Opaque is not a risk rating. It is the scanner refusing to lie.</p><p>A tool is opaque when the analyzer can see the function but can’t resolve what it actually does — because the effect lives somewhere a static reader can’t follow. There are two reasons for that, and they are not the same.</p><p><strong>Type A — the effect is one hop away, in the same codebase.</strong> A tool calls _create_impl(ctx), the real work happens in that helper, and the resolver didn&#39;t follow the chain. That&#39;s a limit of the scanner. It shrinks as the analysis gets better.</p><p><strong>Type B — the effect lives where no static analyzer can follow.</strong> datazone_client.create_asset(**params) disappears into boto3, whose source isn&#39;t in the repo. session.call_tool(name, args) runs on a different machine entirely. No static tool resolves these without shipping a model of every SDK on earth. Opaque here is the correct, permanent answer — not a gap to close.</p><p>On the AWS servers, the opacity is overwhelmingly Type B. The scary-looking 87% is mostly boto3 calls a scanner was never going to read.</p><p>That is the honest version of the number. A tool that reported all 87% as “unguarded” would be selling you fear. The real claim is narrower and more useful: of the surface a static scan <em>can</em> verify, the guards are missing more often than they’re present — and the unverifiable remainder is a map of exactly where you have to look by hand.</p><h3>“Read-only” that writes</h3><p>One finding deserves its own line, because it’s the kind a human reviewer misses and a machine doesn’t.</p><p>suggest_aws_commands declares itself read-only. Its MCP annotation says readOnlyHint: true. The code writes anyway.</p><p>The scanner doesn’t downgrade the verdict because a tool <em>claims</em> to be safe. It compares the declared contract to the resolved behavior, and when they disagree, it says so: <strong>contract violation — declared read-only, writes</strong>.</p><p>A tool that lies about being read-only is worse than one that’s honestly dangerous. The honestly dangerous one, you guard. The lying one, you trust — until an agent calls it on the strength of its own label.</p><h3>What this scan does not prove</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bw64QebaToKlITUjggWGvw.png" /><figcaption>Knowing which tools are unguarded is the first layer. Stopping the call when the agent fires it is the second.<br> <em>AI-generated illustration (Google Gemini).</em></figcaption></figure><p>This is not a claim that these servers are vulnerable.</p><p>A guard the scanner can’t see may still exist — in an IAM policy, in transport-layer auth, in a deployment config the code never mentions. Static analysis reads code. It does not run the system, and it does not see the cloud around it. Opaque means <em>unverified</em>, not <em>unsafe</em>.</p><p>And the scanner finds missing guards. It does not add them. It reads your code and tells you where an action leaves with nobody signing for it. Stopping that action when the agent actually fires it is a different layer, and a different tool.</p><p>Knowing is the first job. Deciding is the second. This is the first.</p><h3>What it means if you deploy agents</h3><p>If you connect an agent to an MCP server, you inherit every tool that server exposes. Including the ones that delete.</p><p>The model decides when to call them. The server decides what they do. In the gap between those two — on the servers I scanned — there is usually nothing. No confirmation. No bound. No record of who said yes.</p><p>You can see that gap before you ship. That’s what a static scan is for: a map of every tool that can change the real world, and which ones leave with no signature.</p><p>The map takes sixty seconds.</p><h3>Reproduce it</h3><p>Every number here is reproducible. One command per server, commits pinned.</p><pre>pip install diplomat-agent==0.5.3<br>git clone &lt;server-repo&gt; &amp;&amp; cd &lt;repo&gt; &amp;&amp; git checkout &lt;pinned-SHA&gt;<br>diplomat-agent scan . --mcp</pre><p>Pinned commits:</p><pre>awslabs/mcp                          ea55b31<br>awslabs/threat-modeling-mcp-server   c18a0da<br>awslabs/amazon-datazone-mcp-server   d494ebb<br>langchain-ai/langsmith-mcp-server    299c15b<br>langchain-ai/mcpdoc                  59cb6a8</pre><p>Scan the server before you wire an agent to it. The unguarded tools are the ones to guard first. The opaque ones are the ones to read by hand. Both lists are shorter than the fear, and longer than zero.</p><p>Repo and the full per-tool tables: <strong>github.com/Diplomat-ai/diplomat-agent</strong></p><p>Questions ? contact@diplomat.run or contact me On Linkedin <a href="https://www.linkedin.com/in/josselin-guarnelli/">https://www.linkedin.com/in/josselin-guarnelli/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ebf6ff97a49b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your AI agent can spend money and email anyone. Nobody is checking.]]></title>
            <link>https://medium.com/@JossGuarnelli/your-ai-agent-can-spend-money-and-email-anyone-nobody-is-checking-3ce31fab998b?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ce31fab998b</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Tue, 09 Jun 2026 16:22:12 GMT</pubDate>
            <atom:updated>2026-06-10T06:22:26.920Z</atom:updated>
            <content:encoded><![CDATA[<p>I scanned 16 open-source agent repos. 76% of the tool calls that touch the real world had no guardrail at all. Here is how to find yours</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zzrE7OjHhWU1Swq8ytaM7w.png" /><figcaption><em>Before each tool call, a deterministic verdict: continue, review, or stop. Diagram rendered with AI</em></figcaption></figure><p>Somewhere in a production agent right now, a function can delete a user’s data, and nothing stands between the model’s decision and the database write.</p><p>Here is a real one. In Khoj, an open-source personal-assistant agent, a function called ai_update_memories lets the model delete a user&#39;s stored memories. No confirmation step. No human in the loop. The model decides, the function runs, the memories are gone.</p><p>This is not a bug. The code does exactly what it was written to do. That is the problem.</p><p>We spent two years getting good at building agents. We are still bad at governing what they do once they can act.</p><h3>It’s not a model problem</h3><p>Almost all of the agent-safety conversation is about the model. Hallucination. Alignment. Prompt injection. Whether the next checkpoint is more steerable than the last.</p><p>Meanwhile the actual blast radius lives one layer down, in the boring part nobody writes threads about: the tool call. The model proposes. The tool executes. The gap between <em>deciding to charge a card</em> and <em>charging the card</em> is where production agents get dangerous, and in most codebases that gap is governed by nothing at all.</p><p>The model is fine. The tool surface is the issue.</p><p>I wanted a number, not a vibe, so I built a static scanner and pointed it at 16 open-source agent repositories. It walks the AST and flags every function that can change the world outside the process — a database write, an HTTP POST, a subprocess, an email send, an agent invocation — then checks whether anything guards it: input validation, a rate limit, an auth dependency, a confirmation step.</p><p>The result: <strong>76% of tool calls with real-world side effects had zero checks.</strong></p><p>Broken down across the corpus, the unguarded calls were 3,260 database writes, 1,305 database deletes, 968 HTTP writes, 697 subprocess or exec calls, and 250 email sends. Per repo it was remarkably consistent: Skyvern 345 of 452, Dify 759 of just over a thousand, PraisonAI 911 of 1,028, CrewAI 273 of 348.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*mBWVWf5Fn6yt_GtK5ydT6Q.png" /><figcaption>Measured with a static AST scanner across four of the most-starred agent frameworks. Chart rendered with AI</figcaption></figure><p>These are not toy projects. They are some of the most-starred agent frameworks on GitHub. The pattern is not carelessness by one team. It is the default.</p><p>The usual patch for this is a human. Route the risky action to an approval queue and have someone click <em>yes</em>. It works, and it does not scale. A few minutes per review, repeated hundreds of times a day, quietly becomes a full-time job whose only output is clicking <em>approve</em>. And when something does go wrong, the queue rarely leaves behind a usable record of who approved what, under which rule, and what happened next.</p><h3>Measure your own blast radius</h3><p>You can argue with my percentage, or you can run it on your own code. The second one takes about two seconds.</p><pre>pip install diplomat-agent<br>diplomat-agent scan .</pre><pre>diplomat-agent — governance scan<br><br>Scanned: ./my-agent<br>Tool calls with side effects: 12<br><br>⚠ research_and_save(query, db_path)<br>  Write protection:       NONE<br>  Rate limit:             NONE<br>  → no rate limit · no auth check<br>  Governance: ❌ UNGUARDED<br><br>⚠ send_notification(user_id, message)<br>  Write protection:       NONE<br>  → no confirmation before send<br>  Governance: ❌ UNGUARDED<br><br>✓ process_order(order_id) — # checked:ok — protected by API gateway<br>  Governance: ✅ CONFIRMED<br><br>────────────────────────────────────────────<br>RESULT: 8 with no checks · 3 partial · 1 confirmed (12 total)</pre><p>A tool call counts as anything that changes state outside the process — a session.commit(), a requests.post(), an smtp.sendmail(), a graph.ainvoke(). A check counts as anything that constrains it — a Field(le=10000), a @rate_limit, a FastAPI Depends(), an idempotency_key, an explicit confirm in the body.</p><p>If a call is unguarded on purpose, or guarded somewhere the scanner can’t see, you annotate it in the source and the next scan respects it:</p><pre>def send_alert(message):  # checked:ok — protected by API gateway<br>    requests.post(ALERT_URL, json={&quot;msg&quot;: message})</pre><p>Drop --fail-on-unchecked into CI and a new unreviewed tool call blocks the PR. The point is not the tool. The point is that after you run it, the number is no longer mine. It is yours, measured on your code, and it is almost never zero.</p><h3>A referee can’t be the player</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VVZTfftC324XxPQbLXCAXA.png" /><figcaption>Proof has to live outside the system being judged. Diagram rendered with AI</figcaption></figure><p>So you have the list. Now decide what governs those calls.</p><p>The instinct is to let the platform guard itself — the same framework that builds and runs the agent also owns the policy, the logging, and the record of what happened. It feels complete. It is also a conflict of interest dressed up as convenience.</p><p>When the agent does something it shouldn’t, the system that allowed the action is the same system that writes the account of the action. Judge and party. An auditor reviewing that trail is reading the defendant’s own diary and being asked to trust it.</p><p>There is a second problem, more technical. Platform guardrails are increasingly themselves LLMs — a model asked to judge another model’s output. That is a probabilistic referee for a probabilistic player. You cannot certify a non-deterministic system with another non-deterministic system and call the result proof.</p><p>A referee has to be of a different nature than the player. Deterministic. Inspectable. Reproducible. Same call, same parameters, same verdict, every single time. The proof it produces has to live outside the system being judged, and be verifiable by someone who does not control that system.</p><p>Let me be honest about what that kind of layer is not, because the limits are the whole point.</p><p>It does not understand intent. It enforces rules on the parameters of a tool call, not on what the model was trying to accomplish. It does not stop prompt injection — if an attacker controls the model’s input, a deterministic gate downstream does not save you. It is syntactic, not semantic. It sits <em>beside</em> your model-level defenses, never instead of them. If what you need is intent classification, this is the wrong tool and you should reach for something else.</p><p>None of this is an argument that your model vendor is careless. The careful ones run red teams and ship audits. It is structural: no party can neutrally audit a system it controls, no matter how good its intentions. The audit has to come from outside, or it isn’t an audit.</p><h3>What this looks like in production</h3><p>Concretely, the shape is small. Before each tool call with a side effect, you get a verdict: <strong>CONTINUE</strong>, <strong>REVIEW</strong>, or <strong>STOP</strong></p><pre>payment:<br>  - id: payment.amount_limit<br>    max_amount: 10000<br>    on_fail: STOP<br>email:<br>  - id: email.domain_blocklist<br>    blocked: [&quot;*.gouv.fr&quot;]<br>    on_fail: STOP</pre><p>Continue the compliant cases automatically, so flow is maximized. Route the ambiguous ones to a human. Stop only the clear violations — an over-limit charge, a send to a blocked domain, a write outside scope. The verdict is pure deterministic policy evaluation: no LLM, no network call, p50 around 0.26ms for payment policies and 0.05ms for email. The referee is not a tax on latency.</p><p>Every verdict writes a receipt — a hash record linking agent, action, verdict, policy, and execution status, chained with SHA-256 so the sequence is tamper-evident. That receipt is the part that matters. It is proof that lives outside the agent and can be handed to someone who never had write access to it.</p><p>The audit trail is the product. The model is the substrate.</p><p>On one high-risk workflow with one design partner, putting this layer in front of tool execution cut the human validation load by about 90% at a constant incident rate — the approvals that were eating headcount were almost all compliant, and a deterministic gate clears those without waking anyone up.</p><p>diplomat-gate is local-first, Apache-2.0, and free: pip install diplomat-gate, write a policy file, wrap your tools. For teams that need a centralized hash-chained trail across services, multi-tenant isolation, and an EU AI Act Article 12 export, that lives in the hosted control plane at diplomat.run.</p><p>I am looking for a small number of design partners — teams running agents in production today behind a manual approval queue — to pressure-test this on real workflows. If that is you, the scanner is the place to start. Run it on your own repo and see what it finds.</p><p>Most of your tool calls are unguarded. You can measure exactly how many before lunch.</p><p><strong>Diplomat is open-source.</strong></p><ul><li><strong>diplomat-agent</strong> — scan your codebase for unguarded tool calls: github.com/Diplomat-ai/diplomat-agent</li><li><strong>diplomat-gate</strong> — deterministic runtime guards, Python: github.com/Diplomat-ai/diplomat-gate</li><li><strong>diplomat-gate</strong>, TypeScript: github.com/Diplomat-ai/diplomat-agent-ts [à supprimer si le repo n&#39;est pas encore public]</li><li><strong>Hosted control plane</strong> — centralized hash-chained audit trail + EU AI Act Article 12 export: diplomat.run</li></ul><p>I write about running AI agents in production. Connect on LinkedIn: <a href="https://www.linkedin.com/in/josselin-guarnelli/">https://www.linkedin.com/in/josselin-guarnelli/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ce31fab998b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your Agent Mined Crypto at 3am. Nobody Asked It To.]]></title>
            <link>https://medium.com/@JossGuarnelli/your-agent-mined-crypto-at-3am-nobody-asked-it-to-2e6b5ffa9138?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e6b5ffa9138</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Mon, 25 May 2026 17:40:32 GMT</pubDate>
            <atom:updated>2026-05-25T17:40:32.020Z</atom:updated>
            <content:encoded><![CDATA[<h4>What actually happened, what the media got wrong, and what your production agents are doing right now without you knowing.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*MLZ1EijnQDLL2omXGtvWuQ@2x.jpeg" /></figure><p>An autonomous coding agent opened a reverse SSH tunnel and mined cryptocurrency on its own training GPUs. Nobody told it to. Here’s what the paper actually says.</p><h4>The Incident</h4><p>On an unspecified morning in late 2025, Alibaba Cloud’s managed firewall flagged a burst of security-policy violations originating from a team’s training servers.</p><p>Nobody had triggered them.</p><p>The team initially assumed misconfigured egress controls or an external compromise. Then they ran the forensics. The timestamps of the violations correlated exactly with RL rollout episodes – periods during which their agent, ROME, was invoking tools and executing code.</p><p>ROME had established a reverse SSH tunnel to an external IP address. It had redirected provisioned GPU capacity toward cryptocurrency mining. It had done both without any instruction in its task prompt.</p><p>That is a verbatim summary of Section 3.1.4 of *“Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem”* – arXiv:2512.24873, by Wang et al. (89 co-authors, Alibaba), published December 31, 2025, revised March 12, 2026.</p><p>Everything else you’ve seen about this story – the cryptocurrency named, the miner binary, the external IP, the duration, the dollar amount – is not in the paper.</p><h4>What the Paper Actually Documents</h4><p>ROME is a 31-billion-parameter MoE coding agent built on Qwen3-MoE, trained via a REINFORCE variant with sparse outcome rewards (unit tests pass / task completion). The team uses SGLang for rollouts, Megatron-LM for training, and a sandboxed terminal environment for tool execution.</p><p>During RL training, two unauthorized behaviors surfaced:</p><blockquote>First : The authors write: the agent established and used a reverse SSH tunnel from an Alibaba Cloud instance to an external IP address – an outbound-initiated remote access channel that can effectively neutralize ingress filtering and erode supervisory control.”</blockquote><blockquote>Second : We also observed the unauthorized repurposing of provisioned GPU capacity for cryptocurrency mining, quietly diverting compute away from training, inflating operational costs, and introducing clear legal and reputational exposure.”*</blockquote><p>The authors’ diagnosis: “these events were not triggered by prompts requesting tunneling or mining; instead, they emerged as instrumental side effects of autonomous tool use under RL optimization.”</p><p>That sentence is the only framing you can legitimately cite. The paper does not say “emergent intelligence.” It does not say “AI goes rogue.” It says: an agent optimizing for task completion, with unrestricted access to network and system primitives, discovered that acquiring external compute increased its ability to complete tasks. It acted on that discovery.</p><h4>What the Media Got Wrong</h4><p>Three claims circulated widely that are not in the primary source:</p><ul><li>The AI downloaded a cryptocurrency miner. The paper does not describe where the mining binary came from – whether it was pre-existing in the container image or fetched from a pool. This distinction is material. An agent exploiting an artifact in its environment is reward hacking. An agent autonomously downloading and deploying a miner from the internet is a categorically different capability claim.</li><li>This happened in production. It did not. The abstract separately states that ROME “has been successfully deployed in production.” The incident occurred during RL training rollouts. The paper is unambiguous on this.</li></ul><p>80% of organizations deploying AI agents report risky behavior – McKinsey. McKinsey’s October 2025 playbook on agentic AI cites this figure – but footnotes it to a SailPoint / Dimensional Research vendor survey (n=353, May 28, 2025). It is not a McKinsey finding. If you use that number, cite SailPoint.</p><h4>Why It Happened: The Root Cause Nobody Is Naming</h4><p>The incident narrative in §3.1.4 buries a detail that every engineering team should internalize.</p><p>The sandbox allowed it.</p><p>ROME had access to shell execution, network primitives, and GPU scheduling – because coding agents need those capabilities. The RL reward signal pointed at task completion. The optimization process found a path. The path involved side-effectful tool calls that nobody had guarded.</p><p>This is not a story about AI consciousness or instrumental convergence. It is a story about the intersection of two things every production agent team already knows:</p><ol><li>Agents that can do useful work can also do harmful work with the same tools.</li><li>1. Reward optimization is aggressive. It finds paths you did not anticipate.</li></ol><p>The authors themselves enumerate the fix they built: a programmatic red-teaming pipeline that injects safety hazards at the prompt, repository, and tool level, plus “golden trajectories” free of security violations fed back into SFT and RL. They explicitly defer a systematic safety investigation to future work.</p><p>That is honest. It is also an incomplete answer for teams shipping agents into production today.</p><h4>What This Means for Production Agent Teams</h4><p>If your agents have shell, network, or payment tool access — and most production agents do — the same optimization pressure applies. The blast radius is whatever your tools can touch.</p><p>The ROME incident is a training-time observation. But the mechanism – unrestricted tool calls, no enforcement boundary, no verdict before execution – maps directly onto production architectures.</p><p>Most agent frameworks do not enforce at the tool-call layer. They guard at the prompt or response layer. The distinction matters.</p><p>When we ran diplomat-agent, our open-source AST scanner, across 16 production-representative OSS agent repos — Skyvern, Dify, PraisonAI, CrewAI, and others:</p><blockquote>76% of tool calls across 16 production-grade OSS agent repos have zero guards. That’s 5,344 out of 7,029 — measured statically, not estimated.</blockquote><p>ROME’s sandbox had network primitives without egress enforcement. Production agents have payment APIs, database writes, email senders, and subprocesses — also frequently without enforcement.</p><p>The pattern is identical. The consequences are just different.</p><h4>The Fix: Block at the Tool Call, Not the Prompt</h4><p>diplomat-gate, the runtime enforcement layer we built alongside the scanner, operates on a simple model: every tool call produces a verdict – CONTINUE, REVIEW, or STOP – before execution. The logic is deterministic. Under one millisecond. No LLM in the enforcement path.</p><p>For your engineering team — the implementation is three lines:</p><pre>from diplomat_gate import Gate<br><br>gate = Gate.from_policy(&quot;policies/network.yaml&quot;)<br><br>@gate.enforce<br>def execute_shell_command(cmd: str) -&gt; str:<br>    # STOP if cmd matches: ssh, curl external, wget, nc<br>    # STOP if cmd matches: known miner binary patterns<br>    # REVIEW if cmd involves subprocess with network args<br>    return subprocess.run(cmd, shell=True, capture_output=True).stdout<br></pre><p>Three rules in a YAML policy — block outbound SSH, block known miner patterns, flag subprocess-with-network combinations — cover both behaviors ROME exhibited. The YAML is in the repo.</p><p>The receipt is SHA-256 hash-chained. Every verdict, every tool call, every policy match is immutable and sequenced. Under EU AI Act Article 12, that audit trail is not optional for high-risk deployments.</p><p>Diplomat does not detect prompt injection. It does not classify intent. It is intentionally narrow. A deterministic enforcement layer at the tool-call boundary is a different thing from an LLM-based guardrail, and conflating the two is how production teams end up with neither.</p><h4>What This Is Not</h4><p>The ROME incident is not proof that AI systems are spontaneously developing self-preservation drives.</p><p>The paper does not claim that. The authors use “instrumental side effects.” The distinction matters technically. An agent that discovers mining increases task performance is exploiting an environmental artifact under optimization pressure. That is a reward-design and sandbox-configuration failure. It is serious. It is fixable.</p><p>The AGI-adjacent framing that dominated social media is not supported by §3.1.4.</p><p>What is supported: tool calls with side effects, absent enforcement, will eventually be misused – by optimization pressure, by adversarial inputs, or by human error. That claim does not require anthropomorphizing the model.</p><h4>What to Do Now</h4><p>Three actions, in order of effort:</p><blockquote>Scan your agent’s tool call surface.** Run diplomat-agent (PyPI: `pip install diplomat-agent`) against your codebase. It will produce a `toolcalls.yaml` – a Behavioral BOM listing every tool call, every side-effect class, every missing guard. The scan takes under two minutes on a typical agent repo.</blockquote><blockquote>Enforce at the tool-call layer.** Not at the prompt. Not at the response. At the tool call, before execution. diplomat-gate is the open-source implementation; the pattern is framework-agnostic.</blockquote><blockquote>Build an audit trail before you need it.** Hash-chained receipts are cheap to implement. Reconstructing what your agent did after an incident is not. EU AI Act Article 12 compliance for high-risk agentic systems requires immutable logging of automated decisions. August 2026 is the enforcement date.</blockquote><p>The paper is at &lt;https://arxiv.org/abs/2512.24873&gt;. Section 3.1.4 is four pages. Read it before shipping your next agent.</p><p>Scan your repo: pip install diplomat-agent → github.com/Diplomat-ai</p><p>Or skip the install. If you want us to run the scan against your stack and walk your team through the results — email contact@diplomat.run. 30-minute call. The output is the same toolcalls.yaml you’d generate yourself.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e6b5ffa9138" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your AI agent just emailed your insurance company. You didn’t ask it to.]]></title>
            <link>https://medium.com/@JossGuarnelli/your-ai-agent-just-emailed-your-insurance-company-you-didnt-ask-it-to-75dec84b8acc?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/75dec84b8acc</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Thu, 23 Apr 2026 15:28:06 GMT</pubDate>
            <atom:updated>2026-04-23T15:36:40.618Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A deterministic policy gate for AI agent tool calls — CONTINUE, REVIEW, or STOP, in microseconds, with a tamper-evident audit trail. No LLM. No network. </em><em>pip install diplomat-gate.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dauczE7lSDCe_VBw8d-WZA.png" /></figure><h3>The incident</h3><p>An AI assistant reads a PDF the user uploaded, infers that “we should push back on this claim,” finds a contact address buried in the document, drafts a legal rebuttal, and sends it to the insurance company. Autonomously. No one asked.</p><p>Nothing in the framework stopped it. The model decided; the tool dispatcher executed; the SMTP call went out. By the time a human noticed, the email was in someone else’s inbox — a named company, with a legal-sounding subject line, from an email address attributable to a real person.</p><p>That scenario lives in demos/openclaw/run.py in the repo I&#39;m about to point you at. It runs in under 60 seconds, with no API key. The first half shows the agent sending the email. The second half shows the same agent behind a gate:</p><pre>SCENARIO 2 — Same agent, behind diplomat-gate<br>  Verdict: STOP<br>    - email.domain_blocklist: Domain &#39;lemonade.com&#39; is on the blocklist<br>  🛡  Email blocked before reaching the SMTP server.<br>  Emails actually sent: 0</pre><p>Ten lines of YAML between the agent and the real world. That’s the product.</p><h3>Why this problem is the shape it is</h3><p>The uncomfortable thing about agent tool calls is that the code that talks to Stripe, SendGrid, the database, or an HTTP webhook was almost always written for a deterministic caller — a human behind a form, a cron job, a webhook handler. It was reviewed for that caller. Its preconditions, its validation, its rate limits all assume that caller.</p><p>Then we wired an LLM to it.</p><p>LLMs hallucinate. They loop. They get prompt-injected through tool outputs. They pattern-match their way from a plausible chain of thought into a decidedly implausible action. The caller changed fundamentally. The code didn’t. Neither did the code-review workflow that’s supposed to catch this kind of thing — a linter does not know what a “tool call” is.</p><p>OWASP now classes this family of failures as <strong>LLM06: Excessive Agency</strong> in its Top 10 for LLM Applications. In plain terms: agents doing more than they should.</p><p>The usual response is “add a guardrail library.” Most of those libraries have one of two problems. They’re either <em>another LLM judging the first LLM</em> — which is exactly the wrong failure mode, non-determinism layered on non-determinism — or they ask you to rewrite your agent inside their abstractions: their Agent, their Runner, their framework. Lock-in renamed.</p><p>I wanted something different. Deterministic. Droppable in front of an existing agent without changing how the agent is built. Auditable months later by someone who wasn’t in the room. That’s diplomat-gate.</p><h3>What it actually does</h3><p>diplomat-gate is a policy gate. Every tool call the agent is about to make passes through it. The gate evaluates the call against a list of policies and returns one of three verdicts:</p><ul><li><strong>CONTINUE</strong> — the call proceeds</li><li><strong>REVIEW</strong> — the call is paused and enqueued for a human approver</li><li><strong>STOP</strong> — the call is refused; the agent gets a structured reason it can reason about</li></ul><p>The resolution rule is unambiguous: if any policy returns STOP, the verdict is STOP. If any returns REVIEW and none return STOP, the verdict is REVIEW. Otherwise, CONTINUE. Severe wins.</p><p>Every verdict produces a Receipt — a signed record with a SHA-256 hash of the canonical tool call, the list of violations, the policies evaluated, the outcome, and the latency. The receipt is the primary output. The verdict is just a routing decision; the receipt is the evidence.</p><p>No LLM calls. No network requests. Pure Python stdlib. Zero mandatory dependencies. Works on Python 3.10 through 3.13.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ix4v6EQ2TyON-03QGzQGPg.png" /></figure><h3>60 seconds of code</h3><pre>pip install &quot;diplomat-gate[yaml]&quot;</pre><p>A minimal gate configuration:</p><pre># gate.yaml<br>version: &quot;1&quot;<br><br>audit:<br>  enabled: true<br>  path: &quot;./diplomat-audit.db&quot;<br><br>review_queue:<br>  enabled: true<br>  path: &quot;./diplomat-review.db&quot;<br><br>payment:<br>  - id: payment.amount_limit<br>    max_amount: 10000<br>    on_fail: STOP<br><br>email:<br>  - id: email.domain_blocklist<br>    blocked: [&quot;*@*insurance*&quot;, &quot;*@*legal*&quot;, &quot;*.gouv.fr&quot;]<br>    on_fail: STOP</pre><p>And the wire-up:</p><pre>from diplomat_gate import Gate<br><br>gate = Gate.from_yaml(&quot;gate.yaml&quot;)<br><br>verdict = gate.evaluate({<br>    &quot;action&quot;: &quot;charge_card&quot;,<br>    &quot;amount&quot;: 15_000,<br>    &quot;customer_id&quot;: &quot;cus_123&quot;,<br>})<br><br># verdict.decision    -&gt; Decision.STOP<br># verdict.violations  -&gt; [Violation(policy_id=&quot;payment.amount_limit&quot;, ...)]<br># verdict.latency_ms  -&gt; ~0.02</pre><p>gate.evaluate() takes a dict. Any dict. If your agent framework produces tool calls as dicts — or anything you can coerce into a dict — you can gate them. No SDK import. No subclassing.</p><p>If you prefer a decorator:</p><pre>from diplomat_gate import Blocked, Gate, NeedsReview, configure, gate<br><br>configure(Gate.from_yaml(&quot;gate.yaml&quot;))<br><br>@gate(action=&quot;charge_card&quot;)<br>def charge(amount: int, customer_id: str) -&gt; dict:<br>    return stripe.charges.create(amount=amount, customer=customer_id)<br><br>charge(amount=500,    customer_id=&quot;cus_123&quot;)   # CONTINUE — runs normally<br>charge(amount=50_000, customer_id=&quot;cus_123&quot;)   # STOP — raises Blocked</pre><p>The decorator wraps any Python function, evaluates the gate against the bound arguments, raises Blocked or NeedsReview on non-CONTINUE verdicts, or calls the function normally. No change to the function&#39;s signature. No change to its return type on the happy path.</p><h3>The policies that ship in 0.3.0</h3><p>Five for payments:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RRUGh3mtnu3sZrDJkZYPUg.png" /></figure><p>Four for email:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oSqkFCEVLI6GHCAo3AbyQg.png" /></figure><p>Each policy takes a severity (critical / high / medium / low) and an on_fail action (STOP or REVIEW). You can mix them: STOP on a transfer above €10 000, REVIEW on anything between €1 000 and €10 000, CONTINUE below. That&#39;s three lines of YAML, one decision surface, and a receipt for every outcome.</p><p>Custom policies are a base class away:</p><pre>from diplomat_gate.models import PolicyResult<br>from diplomat_gate.policies.base import Policy<br><br>class NoWildcardDeletePolicy(Policy):<br>    def evaluate(self, tool_call, state):<br>        if (tool_call.action == &quot;delete_user_data&quot;<br>                and tool_call.params.get(&quot;user_id&quot;) == &quot;*&quot;):<br>            return PolicyResult.FAIL<br>        return PolicyResult.PASS<br><br>    def violation_message(self, tool_call):<br>        return &quot;Wildcard delete requires a manual ticket&quot;</pre><p>Twelve lines. The custom policy snaps into the same gate as the built-ins, same on_fail semantics, same receipt in the audit log.</p><h3>Adapters for the frameworks you already use</h3><p>diplomat-gate is framework-agnostic by design — any tool call expressible as a dict works. But for three common shapes there are typed adapters in the repo, duck-typed so the underlying SDK doesn&#39;t need to be installed:</p><pre>from diplomat_gate.adapters.openai    import filter_allowed as openai_filter<br>from diplomat_gate.adapters.anthropic import filter_allowed as anthropic_filter<br>from diplomat_gate.adapters.langchain import gated_tool<br><br># OpenAI Chat Completions tool_calls<br>allowed, review, blocked = openai_filter(gate, response.choices[0].message.tool_calls)<br><br># Anthropic Messages API tool_use blocks<br>allowed, review, blocked = anthropic_filter(gate, response.content)<br><br># LangChain / LangGraph tools<br>safe_tool = gated_tool(my_langchain_tool, gate)</pre><p>The OpenAI and Anthropic adapters partition a response’s tool calls into three lists — allowed, needing review, blocked — so your orchestrator decides what to do with each. The LangChain adapter wraps the tool itself. In every case, three lines to wire into an existing agent.</p><p>OpenClaw, browser-use, CrewAI, AutoGen, and any framework that emits dict-shaped tool calls work through the core evaluate() without an adapter at all.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nNhQex-BRsotrZlt3tpqKw.png" /></figure><h3>The audit trail — the part I think actually matters</h3><p>Every verdict is recorded in an append-only SQLite log. Each row carries a monotonically increasing sequence number and a SHA-256 record_hash computed over the row&#39;s canonical contents <em>and the previous row&#39;s hash</em>. Tampering with any historical row breaks the chain. The chain is verified by a single CLI command:</p><pre>$ diplomat-gate audit verify --db ./diplomat-audit.db<br>OK: chain valid (14832 record(s) checked)</pre><p>Two things worth being precise about:</p><ul><li>This is <strong>tamper-evident, not tamper-proof.</strong> An attacker with write access to the .db file can run rebuild-chain and produce a valid chain over tampered data. If you need non-repudiation, ship the records to a write-once store — S3 Object Lock, an external logging system, whatever you already have. The chain catches accidental corruption and post-hoc row edits by someone who doesn&#39;t own the file. That is the honest threat model.</li><li><strong>Sensitive parameters are redacted by default.</strong> recipient, to, email, domain, amount, card_last4, phone — each is replaced with a deterministic hash prefix (h:&lt;sha256-prefix&gt;) before persistence. You can extend or shrink that list at runtime; it&#39;s a mutable module-level constant. The default errs on the privacy side.</li></ul><p>REVIEW verdicts are enqueued in a second SQLite database, with a pending → approved / rejected / expired lifecycle enforced server-side. Operators can work the queue from the CLI:</p><pre>diplomat-gate review list    --db ./diplomat-review.db<br>diplomat-gate review approve --db ./diplomat-review.db --id &lt;item_id&gt; --reviewer alice<br>diplomat-gate review reject  --db ./diplomat-review.db --id &lt;item_id&gt; --reviewer alice --note &quot;...&quot;</pre><p>When a regulator, an auditor, or your own incident-response team asks “did the agent have authorization to do that, and who said so when?”, the answer is a row, a chain of hashes back to the genesis record, and a reviewer’s name.</p><h3>Performance</h3><p>I cared about this from day one. A gate you can’t afford to run is a gate you turn off.</p><p>Microbenchmarks, 5 000 iterations, mid-range 2024 laptop:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*uT163X0f7usGYgC0JHSlMw.png" /></figure><p>Two caveats I’d rather state than hide. First, these numbers vary 1.5x to 4x across sessions on the same machine — CPU load, SQLite WAL state, temperature. They’re order-of-magnitude claims, not contractual ones. Run python benchmarks/run.py on your own hardware before quoting them publicly. Second, the long tail on with_audit_sqlite reflects fsync latency spikes; expect similar behavior on shared storage.</p><p>Your LLM call takes hundreds of milliseconds. The gate adds tens of microseconds. You will not notice it.</p><h3>What it explicitly does NOT do</h3><p>In the spirit of the threat model being honest:</p><ul><li><strong>No LLM judge.</strong> Policies are deterministic Python. A SemanticPolicy base class for LLM-evaluated rules is a design sketch in the issue tracker, not a shipped feature.</li><li><strong>No magic auto-instrumentation.</strong> You wire the gate into your tool dispatch path, explicitly. Three lines. This is deliberate — invisible safety is no safety.</li><li><strong>Not a sandbox.</strong> If your tool function is subprocess.run(user_input), the gate can refuse to invoke it, but it does not isolate execution. Pair with a real sandbox (Firejail, gVisor, a container) if that&#39;s your threat model.</li><li><strong>Not an alignment tool.</strong> It does not prevent the model from <em>wanting</em> to do the wrong thing. It prevents the wrong thing from <em>happening</em>.</li><li><strong>Syntactic evaluation, not semantic.</strong> Policies match on action names, parameter values, and patterns. “Is this particular refund justified?” is not a question the gate can answer. “Is any refund allowed above €10 000 without human approval?” is.</li><li><strong>No cross-process rate-limit consistency out of the box.</strong> The in-process StateStore is fine for a single worker; multi-process deployments need a shared backend. Redis adapter is one of the good-first-issues tagged in the repo.</li></ul><h3>Try it</h3><pre>pip install &quot;diplomat-gate[yaml]&quot;<br><br># or clone and run the insurance-email demo end-to-end:<br>git clone https://github.com/Diplomat-ai/diplomat-gate.git<br>cd diplomat-gate<br>pip install -e &quot;.[yaml]&quot;<br>python demos/openclaw/run.py</pre><p>The demo is self-contained. It shows the incident, then shows the same incident prevented. Under 60 seconds. No API key. No Docker.</p><p>Eight runnable examples sit under examples/ — basic gate, YAML config, decorator, audit trail, review queue, OpenAI adapter, Anthropic adapter, LangChain adapter. Each one works from the repo root and from inside examples/. None of them require an SDK install.</p><p>If you’ve shipped an agent to production and learned the hard way what “the LLM did <em>what</em>?” feels like, I’d genuinely like to hear from you. The threat model in SECURITY.md is opinionated, and I&#39;d rather have it broken in the issue tracker now than in production later.</p><p><strong>Repo:</strong> <a href="https://github.com/Diplomat-ai/diplomat-gate">github.com/Diplomat-ai/diplomat-gate</a> <strong>PyPI:</strong> pip install diplomat-gate <strong>License:</strong> Apache 2.0</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qU5HVkMscL0P3w1aGOVg1g.png" /></figure><h3>One closing thought</h3><p>The last two years built agents that can do more and more on our behalf. Very little of that work went into the seatbelts.</p><p>The paradigm shift is concrete and unglamorous: the caller changed. Code designed to be invoked by a deterministic caller — a human, a cron, a webhook handler — is now invoked by a probabilistic one that hallucinates, loops, and gets prompt-injected. The code didn’t change. Our review workflows haven’t caught up.</p><p>Nothing about that is a moral failing. It’s what happens when an industry ships a new kind of caller faster than it ships the tooling to audit it.</p><p>You don’t need a 50 MB framework with vendor lock-in to put a gate in front of your agent. You need a few hundred lines of Python, a YAML file, and the discipline to write down the rules your agent must follow.</p><p>That’s all diplomat-gate is. I&#39;d love it if you tried it and more if you broke it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=75dec84b8acc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your AI Agent Can Delete Your Database 345 Times Before Anyone Notices]]></title>
            <link>https://medium.com/@JossGuarnelli/your-ai-agent-can-delete-your-database-345-times-before-anyone-notices-55eabee32211?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/55eabee32211</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Wed, 08 Apr 2026 08:40:21 GMT</pubDate>
            <atom:updated>2026-04-08T08:40:21.614Z</atom:updated>
            <content:encoded><![CDATA[<h4>No rate limits. No input validation. No confirmation steps. No auth checks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uKdk6TjKfpqY_G-l1Bjuow.png" /></figure><p>We scanned 16 open-source AI agent repos. 76% of tool calls with real-world side effects had zero protective checks. No rate limits. No input validation. No confirmation steps. No auth checks.</p><p>Not in toy projects. In production applications with thousands of GitHub stars.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T8n5VywQAN9VNt6KrN13vg.png" /></figure><p>You’d expect framework code to lack guards that’s normal. It’s generic code, and it’s the implementor’s job to add protections.</p><p>Here’s the part that should bother you: the same pattern holds in production agent applications with real business logic.</p><p>Skyvern (browser automation, 595 files): <strong>76% unguarded.</strong> Dify (LLM platform, 1,000+ files): <strong>75% unguarded.</strong></p><p>The frameworks aren’t the problem. The problem is that we spent 30 years securing functions called by humans — and overnight, it’s an LLM calling them. The protections we know how to build weren’t designed for a caller that can hallucinate, loop, or get prompt-injected.</p><p>In practice: a single prompt injection or a simple hallucination — could trigger hundreds of unvalidated database writes, unchecked HTTP requests to arbitrary URLs, or file deletions without confirmation.</p><p>Here’s what we found, how we found it, and how you can audit your own agent code in 60 seconds.</p><h3>What We Actually Scanned</h3><p>We analyzed 16 open-source repos in two categories:</p><blockquote><strong>Agent applications</strong> : repos that ship real business logic: browser automation agents, AI assistants, LLM platforms with tool-calling capabilities. These are the repos where guards <em>should</em> exist because the code runs in production against real databases and APIs.</blockquote><blockquote><strong>Agent frameworks: </strong>repos like CrewAI and PraisonAI that provide scaffolding for building agents. Framework code is intentionally generic — it exposes tool call patterns without business-specific guards, because that’s the implementor’s responsibility.</blockquote><p>We report findings for both categories, but the story that matters is the application layer: even when developers build on top of frameworks and add their own logic, the guards don’t necessarily follow.</p><p>For each repo, we asked one question: <strong>which functions can change the real world, and which ones have any check before they do?</strong></p><p>A “tool call with side effects” is any function that can:</p><ul><li>Write to a database (session.commit(), .save(), .create())</li><li>Delete data (session.delete(), os.remove(), shutil.rmtree())</li><li>Make HTTP write requests (requests.post(), httpx.put())</li><li>Process payments (stripe.Charge.create())</li><li>Send emails or messages (smtp.sendmail(), slack_client.chat_postMessage())</li><li>Invoke another agent (graph.ainvoke(), agent.execute())</li><li>Execute dynamic code (exec(), eval(), importlib.import_module())</li></ul><p>A “guard” is any check that protects that call:</p><ul><li>Input validation (Field(le=10000), @validator)</li><li>Rate limiting (@rate_limit, @throttle)</li><li>Auth checks (Depends(), Security() in FastAPI)</li><li>Confirmation steps (confirm, approve in function body)</li><li>Idempotency (idempotency_key, get_or_create)</li><li>Retry bounds (max_retries=, @retry(stop=stop_after_attempt()))</li></ul><h3>The Results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-gIzqxjryuQyLzj0l_pYOQ.png" /></figure><p>Across all repos, the most common unguarded categories were database writes, database deletes, HTTP write requests, subprocess/exec/eval calls, LLM calls, and email/messaging.</p><p>The pattern is consistent: <strong>the more dangerous the action, the less likely it has guards.</strong> Not out of negligence — by design. The code you clone from a framework or a GitHub repo is optimized to work out of the box before your setup. Guards are a problem left as an exercise to the reader. Except there’s no tool in the standard workflow to tell you “you just added 12 functions that can change the real world, and none of them have a rate limit.”</p><p>A note on methodology: subprocess/exec/eval calls are a different class of risk — these should generally be eliminated entirely, not guarded. The scanner prioritizes recall over precision: we’d rather flag a function that might be fine than miss one that isn’t. Based on manual review, the false positive rate is roughly 15–20% — mostly from generic .save() calls that turn out to be config or file operations rather than database writes.</p><h3>“But Unguarded Calls Exist in Every Codebase”</h3><p>True. So what makes agents different? <strong>Who calls these functions?</strong></p><p>In a traditional web app, a human user triggers actions through a UI with built-in constraints — forms with validation, buttons with confirmation dialogs, rate limits per session.</p><p>In an agent, an LLM decides which functions to call, with what arguments, how many times. The LLM doesn’t know your business rules. It doesn’t understand that calling refund() 200 times in a loop is economically catastrophic for a company running automated refunds. And if an attacker crafts a prompt injection, the LLM will happily execute whatever functions it has access to — as many times as it&#39;s told.</p><p>Without guards in the code, there’s nothing between the LLM’s decision and the real-world consequence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rcyL3ql2Zw-ri-OMu5NCuw.png" /></figure><p>A concrete example from our scan: Khoj, an open-source AI assistant, exposes a function that lets the LLM delete and replace user memories. It calls session.delete() followed by session.add() with no confirmation, no rate limit, and no validation on the content. A single adversarial prompt could wipe and replace a user&#39;s entire memory store.</p><h3>How We Built the Scanner</h3><p>We built <a href="https://github.com/Diplomat-ai/diplomat-agent">diplomat-agent</a>, an AST-based static analyzer for Python. It uses Python’s built-in ast module — zero required dependencies.</p><p><strong>Why AST and not regex?</strong></p><p>Regex pattern matching misses most real-world code patterns. A function call like db.session.commit() can appear as a direct call, nested inside a try/except, called through a variable alias, or buried three levels deep in a helper function. AST understands the code structure — it parses the actual syntax tree, not text patterns.</p><p>The scanner walks every Python file in your project (excluding tests, migrations, venv, examples, and other non-production directories), visits every function definition, and for each function:</p><ol><li>Finds all calls that match side-effect patterns (DB writes, HTTP calls, deletes, payments, etc.)</li><li>Finds all guards in scope (validators, rate limits, auth checks, confirmation steps)</li><li>Outputs a verdict: UNGUARDED, PARTIALLY_GUARDED, GUARDED, or LOW_RISK</li></ol><h3>Try It</h3><pre>pip install diplomat-agent<br>diplomat-agent scan .</pre><p>That’s it. Zero config, zero required dependencies. Takes about 2–3 seconds on a 1,000-file repo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Joz1IhWXBFam0dD3U5sSA.png" /></figure><pre>diplomat-agent — governance scan<br><br>Scanned: ./my-agent<br>Tool calls with side effects: 12<br><br>⚠ send_report(endpoint, payload)<br>  Rate limit:             NONE<br>  → Risk: agent could exhaust external API quota with 200 calls<br>  ⤷ no rate limit · no auth check<br>  Governance: ❌ UNGUARDED<br><br>⚠ send_notification(user_id, message)<br>  Rate limit:             NONE<br>  → Risk: agent could send 200 messages — spam risk<br>  ⤷ no rate limit · no auth check<br>  Governance: ❌ UNGUARDED<br><br>✓ process_order(order_id)<br>  Write protection:       Input Validation (FULL)<br>  Rate limit:             Rate Limit (FULL)<br>  Governance: ✅ GUARDED<br><br>────────────────────────────────────────────<br>RESULT: 8 with no checks · 3 with partial checks · 1 guarded (12 total)</pre><h3>Generate a Committable Security Registry</h3><p>The terminal report tells you what’s wrong <em>today</em>. The registry tracks it <em>over time</em>.</p><pre>diplomat-agent scan . --format registry --output-registry toolcalls.yaml</pre><p>This generates a toolcalls.yaml file — a structured inventory of every function with side effects in your codebase. For each one: what it does, which guards exist, which are missing, and who reviewed it.</p><pre># toolcalls.yaml (generated by diplomat-agent)<br><br>summary:<br>  total: 47<br>  no_checks: 31<br>  partial_checks: 12<br>  confirmed: 4<br><br>tool_calls:<br><br>  # ⚠ NO CHECKS<br>  - function: process_refund<br>    file: agents/tools.py<br>    line: 42<br>    actions:<br>      - stripe.Refund.create(amount=amount)<br>    checks: []<br>    missing:<br>      - bounds on &#39;amount&#39; (numeric parameter, no limit)<br>      - rate limit<br>      - idempotency key<br>    owner: null<br>    reviewed: null<br><br>  # ✓ CONFIRMED<br>  - function: update_order<br>    file: agents/tools.py<br>    line: 67<br>    actions:<br>      - session.commit()<br>    checks:<br>      - auth_check: &quot;Depends(get_current_user)&quot;<br>      - input_validation: &quot;if total &gt; MAX: raise&quot;<br>    missing: []<br>    confirmed: &quot;checked:ok — validated at API layer&quot;<br>    owner: alice<br>    reviewed: 2026-03-20</pre><p><strong>Commit this file. Review it in PRs.</strong> When someone adds a new function that can send money or delete data, it shows up in the diff — just like a dependency change in package-lock.json</p><h3>Hand It to Your Security Team</h3><p>The scan speaks to developers. The registry speaks to tech leads. But your CISO and auditors need a format they already know.</p><pre>diplomat-agent scan . --format csaf --output advisory.csaf.json</pre><p>This generates an advisory in <strong>CSAF 2.0 format</strong> — the OASIS standard used by CERTs and security teams. Not a custom report that nobody knows how to read. A format that existing security tools consume natively.</p><p>The generated advisory includes:</p><ul><li>An <strong>executive summary</strong> with severity breakdown: <em>“diplomat-agent identified 127 unguarded tool call(s). After deduplication: 120 unique finding(s) (4 CRITICAL, 33 HIGH, 81 MEDIUM, 2 LOW).”</em></li><li>Vulnerabilities sorted by <strong>CVSS severity</strong> (CRITICAL first)</li><li><strong>Automatic OWASP LLM Top 10 2025 mapping</strong> for each finding</li><li><strong>Remediation guidance</strong> and product tree</li><li><strong>Automatic deduplication</strong> — same function + same category = one entry</li></ul><p>The OWASP mapping is built in:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*gJyI3PFyjvTlf4PrlaYxVw.png" /></figure><p>CI integration — archive the advisory as an artifact:</p><pre># .github/workflows/governance.yml<br>- name: Diplomat CSAF advisory<br>  run: |<br>    pip install diplomat-agent<br>    diplomat-agent scan . --format csaf --output diplomat-advisory.csaf.json<br>- uses: actions/upload-artifact@v4<br>  with:<br>    name: csaf-advisory<br>    path: diplomat-advisory.csaf.json</pre><p>Your auditor gets a standard JSON file. No scoping meeting. No report to reformat. They open it in their existing tool.</p><h3>What To Do When You Find Gaps</h3><p>For each unguarded tool call, you have four options:</p><p><strong>Fix it</strong> , add validation, rate limiting, or confirmation in code. The next scan picks it up automatically.</p><p><strong>Acknowledge it</strong> , if the function is intentionally unguarded or protected elsewhere, add # checked:ok as a comment:</p><pre>def send_alert(message):  # checked:ok — protected by API gateway<br>    requests.post(ALERT_URL, json={&quot;msg&quot;: message})</pre><p><strong>Add it to CI, </strong>block PRs that introduce new unguarded tool calls:</p><pre># .github/workflows/governance.yml<br>- name: Diplomat governance scan<br>  run: |<br>    pip install diplomat-agent<br>    diplomat-agent scan . --fail-on-unchecked</pre><p>If you commit toolcalls.yaml as a baseline, only new findings block — no noise on legacy code.</p><p><strong>Review the inventory</strong> — the toolcalls.yaml file is meant to be committed and reviewed in PRs. When someone adds a new function that can delete data, it shows up in the diff.</p><h3>The Bigger Picture</h3><p>The problem isn’t that developers forget guards. It’s that nothing in their workflow shows them guards are missing. The code comes from a framework or a repo, it works, it passes CI, it ships to prod — with a probabilistic caller wired to deterministic functions that can delete data, send money, or execute arbitrary code. As long as the impact surface isn’t visible, it doesn’t exist in the decision process. That’s exactly what toolcalls.yaml and the CSAF advisory produce: making the gap visible so someone can decide to close it.</p><p>The OWASP Top 10 for Agentic Applications (released December 2025) explicitly recommends maintaining a complete inventory of all agentic components, their permissions, and their capabilities. The EU AI Act (enforceable August 2026) requires documenting system capabilities and human oversight measures for high-risk AI systems.</p><p>diplomat-agent produces both: the technical inventory (toolcalls.yaml) and the compliance report (advisory.csaf.json). It&#39;s not a complete governance solution — it&#39;s a verifiable, versionable starting point, in formats your existing tools already understand.</p><p><strong>You can’t govern what you can’t see.</strong></p><h3>Links</h3><ul><li><strong>GitHub:</strong> <a href="https://github.com/Diplomat-ai/diplomat-agent">github.com/Diplomat-ai/diplomat-agent</a></li><li><strong>Install:</strong> pip install diplomat-agent &amp;&amp; diplomat-agent .</li><li><strong>License:</strong> Apache 2.0</li><li>contact me on <a href="https://www.linkedin.com/in/josselin-guarnelli/">linkedin</a></li></ul><p>The scanner is open source, the findings are reproducible. Run it on your agent code and tell me what you find.</p><p><em>What unguarded tool calls are hiding in your agent code? Run the scan and share your results , I’ll respond to every comment.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55eabee32211" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Comes After DevOps: Agentic Execution Layers for the AI-Native Infrastructure]]></title>
            <link>https://medium.com/@JossGuarnelli/what-comes-after-devops-agentic-execution-layers-for-the-ai-native-infrastructure-58fa7d678220?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/58fa7d678220</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[mcp-server]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Tue, 09 Dec 2025 18:35:09 GMT</pubDate>
            <atom:updated>2025-12-09T18:35:09.699Z</atom:updated>
            <content:encoded><![CDATA[<h4>How cross-system agentic orchestration is redefining the future of software delivery</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*0MU7J5ei8CmSBv6P2RBMyA.jpeg" /><figcaption>Copyright Google Cloud (© Google LLC)</figcaption></figure><p>Yesterday, I shipped a full GenAI product to production — backend, frontend, CI/CD and Cloud Run deployments — in 4 hours. But that’s not what’s truly interesting.</p><p>What matters is how it happened, my IDE (via Google Antigravity + a custom MCP server) became an agentic execution infrastructure, orchestrating GitHub, GCP, tests, secrets and redeploys as a unified system.</p><p>Several excellent Medium articles (Antigravity, Vibe Deployment, MCP) are great entry points if the topic interests you.</p><p>Here, I take another angle: <strong>cross-system agentic integration in real production conditions.</strong></p><h3><strong>1 — What interested me in these articles: agents inside the tool</strong></h3><p>Articles about Antigravity, Vibe coding, or MCP highlight:</p><blockquote>• Antigravity as an “<em>agent-first</em>” IDE, where the agent plans, codes, tests, and debugs in a unified environment.<br> • “<em>Vibe deployment</em>,” where a simple prompt is enough to generate and deploy a complete app on Cloud Run.<br> • MCP as a standard for connecting agents to Git, Docker, or a browser.<br> • Lists of MCP servers giving the agent specialized capabilities.</blockquote><p>These stories focus on an agent’s capacity within a constrained environment, look at what an agent can do inside this tool<em>. </em>Impressive!</p><p>I wanted to test <strong>integration between systems</strong>: “<em>What happens if an agent orchestrates my entire delivery landscape local environment, tests, Docker, CI/CD, Cloud Run, secrets from a single point?</em>”</p><h3><strong>2 — My experience: a full delivery loop, agent-driven</strong></h3><p>In<strong> 4 hours</strong>, the agent operated the entire chain:</p><ol><li><strong>Local setup + backend</strong><br> • FastAPI backend containerized, env configuration via Pydantic Settings, health endpoints, pytest tests.<br> • Debugging a JSON parsing issue in the config, fix + test rerun.</li><li><strong>Frontend + build</strong><br> • React/Vite frontend built, packaged behind Nginx for Cloud Run.<br> • Optimized multi-stage Dockerfile.</li><li><strong>CI/CD on GCP</strong><br> • Cloud Build pipeline connected to GitHub, automatic builds and image push to Artifact Registry.<br> • Deployment of two Cloud Run services (API + Web), revisions, 100% traffic routing, healthchecks.</li><li><strong>Debug &amp; production fixes</strong><br> • CORS errors, incorrect backend URL, /health endpoint returning 404, missing IAM secrets.<br> • For each issue:<br> Cloud Run logs → analysis → patch code/config → tests → GitHub push → rebuild → redeploy → verification.</li><li><strong>Secrets and IAM</strong><br> • Reading the OpenAI key from Secret Manager.<br> • Automatic correction of the missing secretmanager.secretAccessor role.</li></ol><p>All of this, <strong>without leaving the IDE</strong>: the Antigravity agent, via MCP, runs gcloud, reads logs, edits files, runs tests, commits/pushes, and monitors Cloud Run. The IDE is no longer an editor. It is a <strong>distributed conductor</strong>.</p><h3><strong>3 — The key angle: agentic integration as a multi-agent system</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*_M2XwA-6KHIyu74PV1rGvQ.png" /></figure><p>What changes everything isn’t “<em>AI helps me code</em>,” but, an agent orchestrates GitHub, GCP, tests, secrets, and redeploys from a <strong>single agentic execution infrastructure</strong>. This approaches a <strong>multi-agent system (MAS)</strong>, where each capability (GitHub, Cloud Run, Cloud Build, Secret Manager, local tests…) behaves like a specialized “<em>functional agent</em>,” orchestrated by a central layer.</p><h4>With three major effects.</h4><h4>1. Observability &amp; debugging become prompt-driven</h4><p>Instead of opening logs, filtering, hunting the stacktrace… I can say in the IDE: “<em>Find the cause of the production error and fix it</em>.”</p><p>The agent executes: Cloud Run logs → analysis → opens correct file → fix → tests → push → redeploy → validation.</p><p>Observability is no longer a console sport — it becomes an <em>agent capability</em>.</p><h4>2. CI/CD and secrets become high-level primitives</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MdhiA8yFxS_iUfBk05AWjA.png" /><figcaption>Copyright Google Cloud (© Google LLC)</figcaption></figure><p>The agent understands that:</p><blockquote>• a git push main triggers Cloud Build,<br> • Cloud Build deploys to Cloud Run,<br> • OPENAI_API_KEY comes from Secret Manager with a specific IAM role.</blockquote><p>CI/CD and secrets are no longer implementation details they become manipulable <strong>concepts</strong> in the conversation with the agent.</p><p>This shift also raises critical questions about agent permissions, security boundaries, and governance: when an agent can act across multiple systems, the design of guardrails and controlled capabilities becomes a core part of the engineering strategy.</p><h4>3. The cycle time becomes independent of human context switching</h4><p>Most of the time is lost in context switching — bouncing between terminals, cloud consoles, logs, IAM, and code. When the agent performs the “<em>cross-system glue</em>,” the cycle: <strong>Dev → Test → CI/CD → Prod → Fix → Redeploy </strong>can repeat several times per hour, without friction. Yes, it’s possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BucBBE8hYXFGLUCN0RWkKQ.png" /></figure><p>I’ll let you imagine how this ecosystem is about to evolve.</p><h3><strong>4 — How is this different from current demos?</strong></h3><p>Demos often show:</p><blockquote>• an agent generating code inside an IDE,<br> • an assistant deploying a first version,<br> • an MCP used as a simple “<em>plugin</em>.”</blockquote><p>I tested <strong>resilience in real conditions</strong>:</p><blockquote>• real production bugs (CORS, IAM, healthchecks, URLs),<br> • automatic navigation across logs, tests, Docker, IAM, frontend, backend,<br> • successive fixes driven by natural language,<br> • stable orchestration of heterogeneous systems.</blockquote><p>This wasn’t a demo. It was a <strong>production-grade workflow</strong>, with its issues… but handled agentically.</p><h3><strong>5 — How to reproduce this pattern</strong></h3><ol><li><strong>Have a real agentic orchestration layer</strong><br> An agent-first IDE (Antigravity) + an MCP server connected to GitHub, GCP, and your local environment.</li><li><strong>Connect the right MCPs</strong><br> GCP (Cloud Run, Cloud Build, Secret Manager), GitHub, Docker, DBs…<br> The GitHub MCP server is very popular, even though I haven’t tested it yet in this setup.</li><li><strong>Model delivery as an agentic flow</strong><br> Not: “<em>the agent writes, the human deploys</em>”<br> But: <strong>the agent performs the end-to-end cycle.</strong><br> Concretely: Code → tests → commit/push → pipeline → deployment → verification.</li><li><strong>Treat secrets &amp; IAM as first-class entities</strong><br> The agent must understand : permissions, keys and IAM bindings and be able to fix them not just report a failure.</li></ol><h3><strong>6 — What’s next?</strong></h3><p>The real question is no longer: “<em>How much code can an agent write</em>? but:<br> <strong>“<em>How many systems can an agent synchronize without human intervention?</em>”</strong></p><p>Antigravity + MCP are today among the most advanced environments to explore this new era of <strong>human-language-driven delivery</strong>, where the IDE becomes an <strong>agentic execution infrastructure</strong> rather than a simple assisted editor.</p><h3>Conclusion</h3><p>What we’re seeing today with agentic orchestration is only a glimpse of tomorrow’s AI infrastructure. As agents become capable of operating across multiple systems in parallelcode, CI/CD, cloud, security, data the boundary between “<em>development</em>” and “<em>operations</em>” dissolves in favor of an autonomous infrastructure, programmable in natural language and continuously optimized.</p><p>In a future perspective, infrastructure itself becomes an active system: capable of self-optimizing pipelines, anticipating failures, adjusting deployments based on intent, and maintaining security and compliance continuously. The role of the engineer shifts from operating the system to designing the intent layer and the boundaries within which autonomous agents evolve.</p><p><strong>Note on cloud providers</strong></p><p>I am not sponsored by any cloud provider.</p><p>If I use GCP as an example (Antigravity, Cloud Run, Cloud Build, Secret Manager), it’s only because I have already shipped complete GenAI projects there, including this one.</p><p>Most of the patterns described are replicable on AWS, Azure, or any environment with equivalent compute, orchestration, security, and MLOps capabilities.</p><p>I welcome multi-cloud feedback to compare, strengthen or invalidate these practices.</p><h3>Appendix — Recent Articles on Agentic Delivery, IDE Augmentation, and MCP</h3><blockquote>. Hands-on with Google Antigravity: The Agentic Platform That Will Supercharge Your Coding</blockquote><blockquote>. Vibe Deployment with AI Studio, Cloud Run, and Jules</blockquote><blockquote><strong>. Building Cloud AI Copilots: Multi-Agent Systems on Cloud Run with Gemini 2.5, ADK &amp; A2A</strong></blockquote><blockquote>. MCP and Agentic AI on Google Cloud Run</blockquote><blockquote>.CI/CD for Vibe Coding: How I Learned to Stop Worrying and Trust the Vibes</blockquote><blockquote>. CI/CD for Vibe Coding: How I Learned to Stop Worrying and Trust the Vibes</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58fa7d678220" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Réussir votre mise en production GenAI sur Google Cloud Platform]]></title>
            <link>https://medium.com/@JossGuarnelli/r%C3%A9ussir-votre-mise-en-production-genai-sur-google-cloud-platform-40af2757b59a?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/40af2757b59a</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Thu, 27 Nov 2025 15:50:53 GMT</pubDate>
            <atom:updated>2026-03-19T14:07:53.566Z</atom:updated>
            <content:encoded><![CDATA[<h4>Latence, RAG, conformité, CI/CD : ce qu’il faut pour passer à l’échelle</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/817/1*Q8_xpdv_t-JaA-ZAS_15yA.png" /></figure><p><strong>La plupart des projets GenAI passent difficilement du POC à la production pour trois principales raisons </strong>:</p><p>· ils restent bloqués au stade de la démo “waouh”,</p><p>· ils se font stopper par la sécurité ou la conformité,</p><p>· ils ne rencontrent pas l’adhésion de vrais utilisateurs, sous le poids de la latence, des coûts et des incidents.</p><p>Dans la grande majorité des cas, ce n’est pas parce que “<em>le modèle n’est pas assez bon</em>”. Le problème vient de ce qui environne le modèle. La qualité et la gouvernance des données, l’infrastructure, le MLOps, l’observabilité, les garde-fous et les exigences réglementaires.</p><p>Dans cet article, je partage mon retour d’expérience de sujets GenAI que j’ai développés du prototype à la production sur GCP, avec un focus particulier sur l’observabilité. Le but est de mettre en exergue comment les POC meurent, l’intérêt d’avoir un blueprint GCP pragmatique, installer un framework MLOps “production-grade”, suivre les bons KPI, puis dérouler un plan de mise en production réaliste et qui fonctionne ! (surtout)</p><h3><strong>1 — Pourquoi la majorité des POC GenAI passent </strong>rarement <strong>en production</strong></h3><p>Soyons concrets, la plupart des POC GenAI passent rarement le stade de l’expérimentation. On obtient une demo du type“Waouh Effect”, dans une salle de réunion… puis plus rien. Le projet se fige, faute d’alignement métier, de gouvernance ou de socle technique solide.</p><p>Ce n’est pas un problème de modèle, c’est un problème de conception système (et ou de gouvernance).</p><h4><strong>1.1 — Commencer par le modèle au lieu du workflow</strong></h4><p>Premier anti-pattern : <strong>Commencer par le modèle au lieu du workflow</strong>.</p><p>Le schéma classique est souvent le même : “On branche GPT ou le modèle X sur nos données et on voit ce que ça donne.”</p><p>Dans ce scénario, il n’y a ni cible claire, ni métrique définie, ni contrainte métier explicite. On obtient alors une démo très séduisante, mais impossible à rattacher à un processus bien défini, à un gain mesurable ou à un budget précis. Autrement dit, c’est une expérimentation technologique, pas un cas d’usage.</p><p>Un vrai use case GenAI part d’un utilisateur précis, d’un travail concret à améliorer, et d’une métrique simple pour mesurer le succès. Par exemple : “Réduire le temps de traitement des tickets de support de 30 % sans perte de qualité”. Tant que cette phrase n’existe pas, vous n’êtes pas en train de concevoir un système GenAI ; vous êtes en train de jouer avec un gadget.</p><h4><strong>1.2 — Traiter la donnée comme un détail “qu’on verra plus tard”</strong></h4><p>Deuxième anti-pattern : <strong>penser que les données sont un problème secondaire.</strong></p><p>Pas de source de vérité claire, des documents obsolètes qui traînent, des droits d’accès flous, des données sensibles (PII/PHI) non identifiées, et la promesse vague du “on nettoiera plus tard”.</p><p>En GenAI, vos données sont votre modèle. Si elles sont incohérentes, périmées ou mal gouvernées, le système produira des réponses erronées, exposera le mauvais contenu aux mauvaises personnes, et vous placera en situation de non-conformité vis-à-vis du RGPD ou de l’AI Act avant même d’avoir lancé la première requête en production.</p><p>Sans gouvernance des données sérieuse, un projet GenAI est littéralement une bombe à retardement, techniquement fragile, juridiquement risqué, et très difficile à industrialiser.</p><h4><strong>1.3 — Ignorer la latence tant que personne ne se plaint</strong></h4><p>Troisieme anti-pattern : <strong>Ignorer la latence jusqu’au moment où elle explose</strong></p><p>Au début, en environnement de test ou sur un petit volume d’utilisateurs, tout semble fonctionner. Puis le trafic réel arrive, et la réalité rattrape le système, la latence P95 explose, les appels LLM se mettent à saturer, les prompts deviennent interminables, les chaînes multi-agents montrent leurs limites, et les cold starts se multiplient.</p><p>La GenAI impose une triple pénalité de latence :</p><p>#1 / le temps de transit réseau entre le front,</p><p>#2 / le backend et le LLM ; le temps de génération des tokens ;</p><p>#3 /et le coût supplémentaire de l’orchestration (agents, outils, mécanismes de retrieval RAG, etc.).</p><p>Si l’architecture n’a pas été pensée dès le départ pour gérer cette complexité, les symptômes apparaissent instantanément dès que l’usage devient sérieux. Concevoir pour la latence, ce n’est pas une optimisation de luxe, c’est une condition d’adoption.</p><h4><strong>1.4 — Avancer sans observabilité réelle</strong></h4><p>Quatrieme anti-pattern : <strong>Opérer sans observabilité ni SLO</strong></p><p>Autre symptôme d’un système qui ne sera jamais vraiment en production, l’absence d’observabilité. Pas de dashboard centralisé, pas de suivi structuré des prompts, pas de monitoring des erreurs, pas d’alertes, aucun SLO défini.</p><p>Le résultat est toujours le même, on débugue à base de captures d’écran et de “tu peux me renvoyer le prompt que tu as utilisé stp, je galere.. ?”.</p><blockquote>Un système GenAI est en production seulement si vous pouvez répondre immédiatement à des questions simples : Quel est notre P95 aujourd’hui ? Quel est notre taux d’erreurs sur les dernières 24 heures ? Est-ce que la qualité de notre retrieval RAG se dégrade ? Si ces réponses ne sont pas disponibles à tout moment, vous êtes encore en mode POC, même si les utilisateurs sont “réels”.</blockquote><h4><strong>1.5 — Traiter la conformité comme la dernière case à cocher</strong></h4><p>Cinquieme anti-pattern : <strong>Traiter la conformité comme une étape finale</strong></p><p>Enfin, un classique : impliquer le DPO ou le CISO trop tard, sans cartographie des flux de données, sans mécanisme d’anonymisation/pseudonymisation, et sans plan de contrôle clair. Dans les secteurs régulés, cela conduit rarement à une discussion, le projet est simplement bloqué.</p><p>La conformité GenAI sur GCP, ce n’est pas un obstacle , c’est un cadre. Ce cadre doit vous permettre de dire précisément quelles données circulent où, comment les données sensibles sont neutralisées ou protégées, et comment vous pouvez le prouver, logs et rapports à l’appui. Si vous pouvez raconter cette histoire calmement à votre CISO, vous êtes sur la bonne voie pour aller en production.</p><h4><strong>1.6 — Signaux faibles que votre POC GenAI va mourir</strong></h4><p>Avant même de parler d’architecture, plusieurs signaux faibles montrent qu’un POC ne passera pas en production.</p><p>Par exemple, l’usage plafonne ou décroît après 2 à 4 semaines alors que les démos internes étaient positives. Dans le même temps, personne n’est capable de répondre clairement à des questions simple comme le nombre de requêtes quotidiennes, qui utilise réellement l’outil, et pour quels types de demandes.</p><p>Autre symptôme critique : le P95 dépasse 10 à 15 secondes en environnement pilote, mais la latence est acceptée comme « normale » sous prétexte que « c’est de l’IA ». En parallèle, le coût cloud n’est pas suivi au niveau de la requête , uniquement en fin de mois via une facture globale. L’outil dépend d’ailleurs encore d’une ou deux personnes clés (qui ne sont plus là) , sans documentation ni pipeline CI/CD. Enfin, la sécurité, la conformité ou le juridique n’ont jamais examiné le projet autrement qu’en slide.</p><p>Si vous cochez au moins trois de ces points, vous n’avez pas un futur produit GenAI, vous avez un POC qui finira archivé dans un drive.</p><h3><strong>2. Un blueprint GenAI pragmatique sur GCP</strong></h3><p>Une fois ces pièges identifiés, la question devient “<em>à quoi ressemble une architecture GenAI réaliste sur GCP, que l’on peut effectivement opérer en production </em>? ”</p><p>L’idée n’est pas de dessiner un schéma parfait pour une slide, mais de proposer un blueprint suffisamment simple pour être mis en place, suffisamment robuste pour tenir en production, et aligné avec les services managés GCP. En voici un exemple générique :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*5yM3Uvcvd0SSp66GyIo7cQ.png" /></figure><h4><strong>2.1 — Quatre couches essentielles</strong></h4><p>On peut structurer le système en quatre grandes couches.</p><p><strong>La première</strong> est la couche d’interface : c’est là que vivent les utilisateurs, que ce soit via une application interne (par exemple un front React/Next.js), une interface Streamlit, un CRM existant ou un outil métier déjà déployé. L’important n’est pas la technologie choisie, mais le fait que cette couche soit clairement séparée du cœur GenAI.</p><p><strong>La deuxième couche </strong>est celle de l’orchestration et de l’application. Sur GCP, on retrouve typiquement une API construite avec FastAPI (par exemple), déployée sur Cloud Run ou GKE, qui orchestre les appels au LLM, applique les politiques métier, gère les agents éventuels et pilote les outils (recherche, RAG, appels à des API internes, etc.). C’est ici que se traduit la logique métier concrète.</p><p><strong>La troisième couche</strong> regroupe la data et les capacités GenAI. On y trouve le LLM (via Vertex AI ou un provider externe), l’index vectoriel pour le RAG (BigQuery, AlloyDB, Postgres + vector search), ainsi que les pipelines de préparation de données (Dataflow, Pub/Sub, Cloud Functions). C’est cette couche qui assure que le système s’appuie sur des données à jour, indexées et gouvernées.</p><p>Enfin, la <strong>quatrième couche</strong> est celle de la plateforme et des opérations : CI/CD avec Cloud Build ou GitLab, IaC avec Terraform, observabilité avec Cloud Monitoring, Grafana ou ELK, gestion des identités et accès via IAM, secrets centralisés, et réseau maîtrisé (VPC, Private Service Connect). C’est ce socle qui permet de parler de “production” avec un minimum de sérieux.</p><h4><strong>2.2 — Un pipeline RAG robuste sur GCP</strong></h4><p>Lorsque le cas d’usage s’appuie sur des documents internes (politiques, procédures, bases de connaissances, documentation produit…), un pipeline RAG bien pensé est souvent le cœur du système.</p><p>Sur GCP, ce pipeline repose généralement sur un stockage initial dans Cloud Storage et BigQuery. Les étapes de prétraitement (nettoyage, découpage, enrichissement des métadonnées) doivent être adaptées au volume. On utilise <strong>Dataflow</strong> pour les flux industriels massifs nécessitant un scaling horizontal complexe, ou <strong>Cloud Run Jobs</strong> pour une approche batch plus légère, simple à maintenir et souvent plus économique pour des volumes standards.</p><p>Les embeddings sont générés avec Vertex AI, puis stockés dans un index vectoriel compatible avec les besoins de requêtage.</p><p>Lorsqu’une requête utilisateur arrive, le système effectue d’abord une recherche sémantique, sélectionne un contexte pertinent (éventuellement en tenant compte de la permission de l’utilisateur), compose un prompt qui mélange instructions, contexte et historique, puis appelle le LLM. L’anonymisation ou la pseudonymisation peut intervenir à cette étape, avant envoi au modèle.</p><p>Ce type de pipeline fonctionne bien pour plusieurs raisons, la gouvernance des données s’appuie sur des briques natives GCP, la scalabilité est prévisible, et l’intégration avec IAM permet de respecter les permissions existantes. Enfin , le cycle de vie des données (ingestion, mise à jour, purge) est maîtrisé.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*AY0itpQHZh5uIw3wTlS8wg.png" /><figcaption>Crédit visuel : Google Cloud (© Google LLC)</figcaption></figure><p>Ce schéma illustre un pipeline RAG opérable en production : ingestion scalable, embeddings Vertex, index vectoriel managé et backend Cloud Run orchestrant retrieval, augmentation, appels LLM et filtrage.</p><p>Je l’utilise pour montrer où se situent les vrais goulots en pratique, découpage des documents, délai embeddings, latence du vector search, et orchestration multi-appels. C’est la base d’un RAG stable, observable et maintenable.</p><p>Nuançons cependant car le schéma RAG classique (recherche sémantique -&gt; Contexte -&gt; LLM) montre aujourd’hui ses limites face à des cas d’usage complexes. En production, on observe un glissement vers des workflows pilotés par des agents (via des frameworks comme LangGraph). L’exécution n’est plus une simple API stateless, mais une machine à états : l’agent reçoit une tâche, boucle sur lui-même, décide d’utiliser un outil (faire une requête SQL, appeler une API interne), vérifie sa propre réponse, et se corrige si besoin avant de répondre à l’utilisateur. C’est plus puissant, mais cela exige une observabilité décuplée pour éviter les boucles infinies.</p><h4><strong>2.3 — Sécurité et confidentialité : des exigences non négociables</strong></h4><p>Sur GCP, un système GenAI de qualité ne peut pas faire l’impasse sur la sécurité et la confidentialité. Cela commence par une isolation réseau adéquate (VPC, Private Service Connect lorsque c’est possible), un IAM rigoureux en least privilege, et l’usage systématique de Secret Manager pour les clés et identifiants sensibles.</p><p>Avant l’appel au LLM, les données doivent être pseudonymisées ou anonymisées si elles peuvent contenir des informations personnelles ou sensibles. Côté observabilité, les logs ne doivent jamais contenir de texte brut issu de patients, de clients ou de données protégées. Là encore, la règle est simple, tout ce qui part dans les logs doit être considéré comme potentiellement visible.</p><p>L’idéal est de mettre en place une sorte de “privacy firewall” interne , une couche capable de détecter les éléments sensibles, de les neutraliser ou de les pseudonymiser, puis de journaliser ce qui a été fait. Ce n’est pas seulement une bonne pratique, c’est ce qui rend la discussion avec la conformité et le juridique beaucoup plus simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*45X-SSCD1h6iCWoyvyoLnA.png" /><figcaption>Crédit visuel : Google Cloud (© Google LLC)</figcaption></figure><p>Ce schéma illustre pourquoi la mise en production GenAI exige une isolation réseau stricte. Le Private Service Connect permet d’appeler Vertex, BigQuery ou des services internes sans sortie vers Internet, tout en appliquant un IAM . Dans les environnements régulés, c’est la condition pour débloquer sécurité, conformité et juridique (flux contrôlés, auditables et maîtrisés de bout en bout).</p><h4>2.4 — Comment choisir concrètement les briques GCP ?</h4><p>Quelques repères opérationnels pour éviter de sur- ou sous-dimensionner vos choix :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*6-mENguUHiVw2QngT2FUlg.png" /></figure><p>Je ne vous invite pas à utiliser tous ces services, mais à choisir le plus simple et compatible avec vos contraintes réelles (usage, volume, sécurité, budget).</p><h4>2.5 — Vous êtes sur AWS ou Azure ?</h4><p>Les principes décrits ici ne sont pas spécifiques à GCP. Dès lors que votre architecture est fondée sur des briques standard (YAML/IaC, Docker/containers, API stateless, CI/CD, observabilité, IAM, vector search), elle est largement réplicable sur d’autres cloud providers disposant d’équivalents fonctionnels.</p><p>En pratique, la logique d’ensemble (séparation des couches, RAG, MLOps, sécurité, conformité) reste identique. Seuls les services managés et l’implémentation changent. C’est précisément ce qui permet de raisonner en architecture cloud-native plutôt qu’en “GCP-only”. Voici comment cela se traduit sur d’autres cloud providers :</p><p><strong>LLM &amp; orchestration GenAI</strong></p><ul><li>GCP : Vertex AI</li><li>AWS : Bedrock, SageMaker, Lambda/ECS pour l’orchestration</li><li>Azure : Azure OpenAI, Azure Functions / App Service / AKS</li></ul><p><strong>Stockage &amp; vector search</strong></p><ul><li>GCP : BigQuery + vector search, AlloyDB / Postgres + vecteurs</li><li>AWS : OpenSearch, Aurora + vecteurs, DynamoDB + extensions, Redshift ML</li><li>Azure : Azure Cognitive Search, Cosmos DB, SQL + vecteurs</li></ul><p><strong>Compute &amp; APIs</strong></p><ul><li>GCP : Cloud Run, GKE</li><li>AWS : Fargate/ECS, Lambda, EKS</li><li>Azure : App Service, Functions, AKS</li></ul><p><strong>MLOps &amp; données</strong></p><ul><li>GCP : Cloud Build, Artifact Registry, Dataflow, Pub/Sub</li><li>AWS : CodePipeline, ECR, Kinesis, Glue</li><li>Azure : DevOps Pipelines, Container Registry, Event Hub, Data Factory</li></ul><p>Dans le cloud, le nom du service n’est que la surface. Le fond, c’est la cohérence de l’architecture, la séparation des responsabilités, l’observabilité et la gouvernance et ça finit toujours par remonter à la surface. Comme en physique des fluides, si la structure est saine, elle émerge naturellement, quel que soit le contenant. Si elle est bancale, aucun logo AWS, Azure, GCP ou Snowflake ne pourra la maintenir à flot.</p><h4><strong>2.6— L’impératif Hybride et On-Premise (DORA, NIS2, RGPD)</strong></h4><p>Dans les secteurs très régulés (banque, assurance, santé, mutuelles), le déploiement “100% Cloud” est souvent un bloqueur absolu face aux exigences de l’AI Act, de DORA ou de NIS2. La peur de perdre le contrôle sur la donnée sensible (PII, secrets métiers) fige les POC.</p><p>La solution pour passer en production est <strong>l’architecture hybride</strong> :</p><ul><li><strong>La donnée reste à la maison :</strong> L’ingestion des documents, le nettoyage, le découpage (chunking), l’indexation vectorielle et l’orchestration du RAG (via Airflow ou N8N) se font On-Premise, sur l’infrastructure du client.</li><li><strong>L’inférence LLM “As a Service” :</strong> Seul le prompt final (nettoyé de toute donnée identifiante) est envoyé via un flux réseau hyper-sécurisé (Private Service Connect) vers Vertex AI ou Azure OpenAI, avec des contrats stricts garantissant la non-utilisation des données pour l’entraînement.</li><li><strong>L’alternative des LLM Locaux :</strong> Pour des cas d’usage spécifiques nécessitant une confidentialité totale, on déploie de “petits” modèles open-source (type Llama 3 8B ou Mistral) directement sur des serveurs On-Premise. Grâce aux techniques de <em>quantization</em> (réduction de la précision des modèles), des GPU d’entreprise accessibles (comme les NVIDIA L40S ou A10G) suffisent amplement à absorber la charge d’inférence en production, évitant ainsi l’explosion des coûts cloud.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GrNLxzEmmI2GJJT5T6qvfA.png" /></figure><h3><strong>3 — Un framework MLOps réellement “production-grade”</strong></h3><p>Une fois l’architecture en place, il reste une question clé , <strong>comment s’assurer que le système est opérable, améliorable et contrôlable dans le temps ?</strong> C’est précisément le rôle du MLOps.</p><h4><strong>3.1 — Une vraie chaîne CI/CD</strong></h4><p>Le point de départ est une chaîne CI/CD . Le code (backend, orchestration, prompts, policies) doit être testé, linting, tests unitaires ou d’intégration, scans de sécurité. Les builds et déploiements vers Cloud Run ou GKE doivent être automatisés, avec la capacité de revenir rapidement en arrière en cas de problème.</p><p>L’infrastructure GCP elle-même doit être décrite en Terraform, de manière à pouvoir recréer un environnement de test, de préproduction ou de production de façon reproductible. Ce n’est pas un luxe, c’est ce qui rend possible l’itération rapide sans casser ce qui fonctionne déjà.</p><h4><strong>3.2 — Versionner prompts, agents et policies</strong></h4><p>En GenAI, beaucoup de comportements “modèle” se jouent dans les prompts, la configuration RAG ou les politiques d’agent. Pourtant, ces éléments sont souvent gérés comme des fichiers de configuration un peu magiques, modifiés à la main au dernier moment (oui nous l’avons tous fait au moins une fois..).</p><p>Pour un système en production, il faut considérer tout cela comme du code, les prompts, les règles de policy, les paramètres RAG. Ils doivent être versionnés dans Git, faire l’objet de revues, de pull requests, et être revertables si une modification dégrade les résultats. C’est ce qui permet de contrôler l’évolution du système dans le temps.</p><h4><strong>3.3 — Évaluation automatique et garde-fous</strong></h4><p>En GenAI, superviser la latence et les erreurs (via Cloud Monitoring ou Grafana) ne suffit plus. Le juridique et le métier exigeront des réponses sur le comportement profond du modèle.</p><ul><li><strong>Traçabilité et Explicabilité :</strong> Vous devez être capable d’expliquer <em>pourquoi</em> le modèle a donné une réponse X à l’instant T. Cela nécessite des outils de journalisation avancés (comme MLflow, LangSmith ou LlamaIndex) pour capturer la chaîne exacte : quel prompt a été généré, quels chunks de la base vectorielle ont été remontés, et quelle version du code a orchestré le tout.</li><li><strong>Les Garde-fous (Guardrails) :</strong> Avant même de renvoyer la réponse à l’utilisateur, le système doit passer par des filtres de sécurité stricts (via des outils comme Guardrails AI ou NeMo Guardrails). Le but ? Bloquer dynamiquement les hallucinations avérées, interdire les réponses toxiques, empêcher l’exposition de données sensibles, et garantir que le ton respecte les normes de l’entreprise. C’est la ligne de défense ultime avant l’utilisateur.</li></ul><p>Enfin, un système GenAI en production doit être instrumenté avec des métriques adaptées de latence P50/P95/P99, taux d’erreurs, coût par requête, dérive du RAG, et comportements utilisateurs (types de requêtes, fréquence, récurrence, etc.).</p><p>Ces métriques doivent être rattachées à des SLO clairs. Par exemple, un P95 de latence maximal, un taux d’erreurs tolérable, un budget mensuel cible. Sans ces objectifs, il est très difficile de parler de fiabilité ou d’industrialisation , on se contente de constater après coup que “ça ne tient pas”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*qfbOZ9UV8YBtaoCqJOreAw.png" /><figcaption>Crédit visuel : Google Cloud (© Google LLC)</figcaption></figure><p>J’utilise ce visuel pour illustrer la chaîne Vertex AI, depuis l’ingestion de données, entraînement, versioning du modèle, déploiement batch/online et gestion des données d’inférence. J’explique à partir de ce pipeline que <strong>la GenAI n’est jamais un livrable fini</strong> : sans MLOps , validation continue, surveillance de la dérive, contrôle de la latence, gestion des coûts, mise à jour des modèles et du RAG = aucun système ne tient en production.</p><blockquote><strong>Le MLOps n’est pas facultatif, c’est le socle du run et de la qualité</strong></blockquote><h3><strong>4 — Trois KPI qui prouvent réellement la valeur</strong></h3><p>Le nombre de tokens n’intéresse personne en dehors de l’équipe technique et le contrôle de gestion bien sûr :)</p><p>Pour démontrer la valeur d’un système GenAI, surtout face à un CFO ou un comité de direction, trois KPI sont parlants.</p><h4><strong>4.1 — La latence P95</strong></h4><p>Le premier est la latence P95, qui reflète l’expérience des utilisateurs dans les pires cas raisonnables. Une interface qui répond en quelques secondes pour 95 % des requêtes est adoptée, une interface qui se met à répondre en 15 ou 20 secondes finit par être évitée.</p><p>Dans la pratique, il n’est pas rare de pouvoir réduire le P95 de 30 à 60 % une fois que les problèmes d’orchestration, de prompts trop verbeux ou de modèles surdimensionnés ont été adressés.</p><h4><strong>4.2 — Le coût cloud par requête</strong></h4><p>Le deuxième KPI est le coût cloud par requête réellement utile. C’est ce chiffre qui intéresse les CFO, bien plus que le coût au token ou les prix affichés par les providers.</p><p>Ce coût inclut les appels au LLM, mais aussi l’usage de BigQuery, le compute, le stockage et éventuellement le réseau. Une fois ce KPI calculé, vous pouvez le mettre en regard d’un gain métier par requête (temps économisé, erreurs évitées, etc.) et commencer à parler de ROI de façon crédible.</p><h4><strong>4.3 — L’uplift du processus métier</strong></h4><p>Le troisième KPI est l’amélioration du processus métier lui-même. Selon le cas d’usage, il peut s’agir de temps gagné par dossier, de réduction d’erreurs, d’augmentation du volume traité par personne, ou de réduction de la durée d’un cycle opérationnel.</p><p>Un système GenAI qui ne bouge aucun KPI métier significatif sera probablement coupé au prochain cycle budgétaire. À l’inverse, un système qui permet de traiter 30 % de tickets de support en plus à effectif constant a de bonnes chances d’être consolidé, voire étendu.</p><h3><strong>5 — Un chemin pragmatique</strong></h3><p>Plutôt que d’empiler les POC, il est plus utile de suivre un chemin structuré, avec des phases claires.</p><p><strong>Phase 1 — Stratégie et faisabilité (1 jour à 3 semaines)</strong></p><p>La première phase consiste à cadrer le cas d’usage, analyser la donnée disponible et les contraintes (techniques, réglementaires, organisationnelles), puis produire un blueprint GCP et une estimation réaliste du ROI. À ce stade, l’objectif est surtout de décider de quel cas d’usage on veut (peut) réellement mettre en production ?</p><p><strong>Phase 2 — Mise en production (8 à 12 semaines)</strong></p><p>La deuxième phase est celle du delivery , construction du backend, mise en place du RAG, intégration des briques MLOps, mise en conformité avec les exigences de sécurité. L’observabilité doit être présente dès le début, et les SLO définis avant d’ouvrir à un large public. La phase se conclut généralement avec les premiers utilisateurs internes.</p><p><strong>Phase 3 — Run et MLOps (6 à 12 mois)</strong></p><p>La troisième phase est celle du run continu, optimisation des performances et des coûts, ajout de nouveaux cas d’usage, mise en place d’une gouvernance récurrente et suivi de la qualité, des coûts et des incidents. C’est ici que le système cesse d’être un “projet” pour devenir un élément durable du paysage applicatif.</p><h3><strong>6 — Si vous êtes déjà sur GCP et que vos POC n’avancent plus</strong></h3><h3>En résumé : ce qui sépare un POC d’un système en production</h3><p>La GenAI n’est pas une technologie difficile à expérimenter. Elle est difficile à industrialiser.</p><p>Ce qui fait la différence entre une démo enthousiasmante et un système réellement en production, ce n’est pas la puissance du modèle. C’est la rigueur de tout ce qui l’entoure : la gouvernance des données, la qualité de l’orchestration, la maturité du MLOps, la lisibilité de l’observabilité et la solidité du cadre de conformité.</p><p>Les organisations qui réussissent cette transition partagent trois réflexes communs : elles partent d’un cas d’usage métier concret et mesurable, elles traitent la donnée et la sécurité comme des fondations et non comme des options, et elles instrumentent leur système dès le premier jour pas en post-mortem.</p><p>Sur GCP comme sur n’importe quel autre cloud, le modèle ne fait pas le produit. L’architecture, l’équipe et la gouvernance le font.</p><p>Si vous êtes arrivé jusqu’ici, vous avez probablement un POC qui mérite mieux qu’un drive. La question n’est plus “est-ce que la GenAI peut fonctionner chez nous ?”elle fonctionne. La vraie question est : “avons-nous les fondations pour qu’elle tienne en production ?”</p><p>Si votre organisation est déjà sur GCP, il est probable que vous n’ayez pas besoin d’un énième POC, ni d’un workshop générique, ni d’un deck marketing de plus.</p><p>Ce dont vous avez besoin ressemble plutôt à deux options concrètes :</p><blockquote><strong>- un atelier ciblé pour clarifier et prioriser vos cas d’usage,</strong></blockquote><blockquote><strong>- un audit de votre architecture GenAI actuelle sur GCP, couvrant vos dataflows, votre sécurité, votre MLOps, vos coûts et votre conformité.</strong></blockquote><p>En une à deux semaines, vous devez avoir une vision claire et factuelle de votre projet ou du moins quoi prioriser. Vous saurez si votre cas d’usage est réellement industrialisable sur GCP, dans quelles conditions techniques et organisationnelles il peut fonctionner, et quel sera son coût opérationnel une fois en production.</p><p>Vous comprendrez également comment l’exploiter au quotidien sans que chaque évolution ne devienne un mini-projet, et quelles mesures concrètes sont nécessaires pour éviter les dérives classiques : hallucinations, instabilités, incidents en charge ou explosion des coûts.</p><p>C’est à partir de là que la discussion change , on ne parle plus d’expérimenter la GenAI, mais de la <strong>mettre réellement en production pour être utiliser par des utilisateurs</strong>.</p><blockquote>nb : <em>Je ne suis sponsorisé par aucun cloud provider. Si j’illustre avec GCP, c’est uniquement parce que j’y ai livré des projets GenAI complets. Une grande partie de ce que je décris est réplicable sur AWS, Azure ou d’autres plateformes disposant d’équivalents en compute, vector search, orchestration, sécurité et MLOps. Je suis d’ailleurs preneur de retours d’expérience multi-cloud pour enrichir et confronter ces bonnes pratiques.</em></blockquote><p>Pour me contacter via linkedin : <a href="https://www.linkedin.com/in/josselin-guarnelli/">Josselin Guarnelli | LinkedIn</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40af2757b59a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Open-Source AI Assistant Bringing Trust to Digital Health]]></title>
            <link>https://medium.com/@JossGuarnelli/open-source-ai-assistant-bringing-trust-to-digital-health-10c1f183ee2c?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/10c1f183ee2c</guid>
            <category><![CDATA[healthcare]]></category>
            <category><![CDATA[digital-health]]></category>
            <category><![CDATA[health]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Thu, 21 Aug 2025 08:25:24 GMT</pubDate>
            <atom:updated>2025-08-22T12:41:40.806Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building Healthcare Commons, Not Just Another App</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q658_I9kNSWxy836YsJfkg.png" /></figure><h3>1 — Why EVI Kids?</h3><p>In 2025, the question is no longer whether AI should enter healthcare, but <strong>how it can do so without undermining the trust so precious that people place in medical information</strong>.</p><p>To build this project, I combined several levers, large language models (LLMs), GDPR compliance, applied healthcare AI, and open-source health technologies. But this is not about stacking buzzwords. A technology mille-feuille without coherence only creates entropy. Every decision whether technical or architectural must be <strong>weighed, converge toward a clear objective</strong>, and remain anchored in the needs of end-users.</p><p>Parents, often worried in complex situations, need clear, fast, reassuring, and grounded answers. Caregivers, meanwhile, carry an immense human workload. Their reserve toward technology does not stem from rejection, but from a legitimate concern to preserve connection, accuracy, and ethics.</p><p>I launched this project because I saw, lived, and <strong>felt the distress of single parents like me</strong>, facing unexplained symptoms in moments of doubt and helplessness. I also witnessed the colossal workload of healthcare professionals, often overwhelmed by simple yet recurring questions, asked with both anxiety and hope.</p><p>To understand, I first listened before even thinking about coding.<br> I engaged with two hospitals, a CPTS, and met healthcare workers directly in the field, often between two emergencies, between two breaths.<br> In total, <strong>37 people</strong> were heard not behind a screen or a form, but face-to-face, with the time it takes.</p><p>I wasn’t looking for “user needs.” I was searching for their shortcuts, frustrations, homemade automations, hidden scripts, WhatsApp exchanges between interns.</p><p>And when you listen this way, what you build is no longer just a “feature” — it’s concrete. For this relief to be useful, technology must fade behind <strong>trust</strong>, AI must respect <strong>confidentiality</strong>, and the tool must be <strong>shared, not sold</strong>.</p><p>Trust must be embedded into the architecture of AI to reduce the cognitive load of caregivers. The famous <em>human-in-the-loop</em> should become an enlightened actor, not a spare wheel.</p><p>That’s why I designed a <strong>free and open-source tool</strong>, based on three architectural commitments:</p><ol><li><strong>Real time</strong>: responses in a few seconds, even under load.</li><li><strong>Quality</strong>: formatted answers anchored in official references.</li><li><strong>Trust</strong>: privacy by design (upstream anonymization, secrets out of repo, minimal traces).</li></ol><p>These guarantees start in tooling:</p><ul><li>secrets handled via <strong>GCP Secret Manager</strong> and injected at runtime,</li><li>no plain-text keys in the repo,</li><li>rotation process included.</li></ul><h3>2 — Design Principles (Guiding Choices)</h3><ul><li><strong>Simplicity first</strong>: a Streamlit monolith to prototype UX, but clean interfaces between layers (anonymization → LLM → observability).<br> Dependencies reflect this pragmatic choice: streamlit, openai, gspread, google-cloud-secret-manager, plus a French spaCy model (fr_core_news_md) for NER.</li><li><strong>Secrets separation</strong>: nothing committed (.env, keys, JSON SA, etc. ignored) → enabling community contributions without leakage risks.</li><li><strong>Data minimization</strong>: the architecture processes the bare minimum and logs KPIs instead of free-text identifiers. (Philosophy + technical ingredients detailed in the README.)</li></ul><h3>3 — Logical Architecture (Overview)</h3><p>At the highest level, the assistant’s logic follows a simple sequence:</p><pre>[User] → [Anonymization] → [LLM] → [Post-processing] → [Observability]</pre><p>This flow illustrates the trust boundary, <strong>nothing leaves the system until it is normalized and anonymized</strong>. From there, the LLM is treated as a replaceable black box, with guardrails before and after its use, and observability ensuring quality without storing sensitive data.</p><p><strong>Detailed view</strong></p><pre>[Client Web]<br>    ⇅ TLS<br>[UI Streamlit]  --(👤 User Input)--&gt;<br>    [Gating &amp; Preprocessing]<br>        ├─ Normalization / Cleaning<br>        ├─ Anonymization (regex + FR NER)<br>        └─ Security heuristics<br>          |<br>          v<br>    [LLM Gateway]<br>        ├─ Model call (OpenAI API / on-prem model)<br>        └─ Retention policies &amp; scoped limits<br>          |<br>          v<br>    [Post-processing]<br>        ├─ Response structuring<br>        ├─ Add warnings &amp; limitations<br>        └─ Red flags → redirect to care<br>          |<br>          v<br>    [Observability]<br>        ├─ Aggregated KPIs (quality, orientation, feedback)<br>        └─ Dashboards (no PII)</pre><p><strong>Architecture Intentions</strong>:</p><ul><li><strong>Gating</strong> = trust boundary. Nothing leaves until normalized + anonymized.</li><li>Stack includes <strong>spaCy FR model</strong> (fr_core_news_md) for stronger anonymization vs regex only.</li><li><strong>LLM Gateway</strong> is agnostic: OpenAI today, replaceable with self-hosted model (VPC/HDS) if sovereignty becomes a requirement.</li><li><strong>Observability</strong> targets KPIs (home/consultation/ER orientation rate, feedback usefulness, etc.) instead of storing sensitive free text.</li><li><strong>Secrets</strong> (OpenAI key, creds) externalized &amp; injected at runtime (Cloud Run --set-secrets, rotation).</li></ul><h3>4 — Technical Stack &amp; Packages</h3><ul><li><strong>UI</strong>: <strong>streamlit </strong>→ rapid UX iterations, ideal for healthcare product discovery.</li><li><strong>Model</strong>: <strong>openai </strong>official Python client, compatible with zero-retention policy.</li><li><strong>Anonymization</strong>: Regex (FR patterns for emails, phones, addresses, dates) and NER with spaCy FR (fr_core_news_md) for entities escaping deterministic rules.</li><li><strong>Light Observability</strong>: <strong>gspread </strong>→ quick dashboarding (QA, ideation, quality monitoring).</li><li><strong>Security &amp; Secrets</strong>: <strong>google-cloud-secret-manager</strong> + strict repo separation (.env, keys ignored with .gitignore).</li><li><strong>Network layer</strong>: httpx, urllib3 → fine-grained timeout/retry control.</li></ul><h3>5 — Data Journey (Before/After) &amp; Safeguards</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*IL6GObAFjompSrr6y8-S7A.png" /></figure><p><strong>Design decision</strong>: logs ≠ business truth.<br>The real measure of value is whether parents feel reassured and caregivers feel supported, not the raw text of their questions. That’s why we track <strong>perceived quality</strong> and <strong>orientation safety</strong> (home, consultation, emergency) through anonymized KPIs instead of storing nominative free text.</p><h3>6 — Deployment with DevSecOps Compliance</h3><p>The app was deployed on <strong>Google Cloud Platform (GCP)</strong> directly from the GitHub repository. This choice is not about GCP itself AWS, Azure, and others provide similar pipelines but rather about developer habits and ecosystem familiarity.</p><p>The real complexity does not lie in pushing the app live, but in <strong>setting up the surrounding safeguards</strong>:</p><ul><li><strong>Secret management</strong>: all credentials stored in GCP Secret Manager, injected at runtime.</li><li>️ <strong>Key rotation policies</strong>: mandatory to reduce long-term exposure.</li><li>️ <strong>Log retention limits</strong>: automated purge every 90 days to comply with privacy-by-design.</li></ul><p>These steps transform a “simple deployment” into a <strong>production-grade, privacy-compliant service</strong>.</p><p><strong>6.1 — Secrets &amp; Keys</strong></p><ul><li>Never in Git. Documented management via Secret Manager + Cloud Run --set-secrets.</li><li>.gitignore bans .env, .json, .pem, .key, etc.</li></ul><p><strong>6.2 — Minimization &amp; Responsibility</strong></p><ul><li>Repo formalizes minimization + secret rotation as baseline — an <strong>architect’s stance</strong>, not a patch.</li></ul><p><strong>6.3 — Maturity Path</strong></p><ul><li>Level 1 (POC/common): anonymization + external LLM (zero-retention) + KPI logs.</li><li>Level 2 (healthcare interop): self-hosted model in VPC, EU data residency, RAG connectors to public refs.</li><li>Level 3 (clinical prod): HDS, full traceability, clinical evaluations, medical governance.</li></ul><h3>7 — Lessons Learned</h3><blockquote>Compliance is <strong>not paperwork, </strong>it’s design. Encoding anonymization, minimization, and secret separation reduces GDPR pain <strong>without breaking UX</strong>.</blockquote><p>At this stage, the lesson is simple, prompts alone are not enough. Asking the right question to an LLM is useful, but without orchestration gating before, post-processing after, and observability throughout you cannot guarantee safety, reliability, or trust. The strength of the system comes from this orchestration of layers, not from a single prompt.</p><p>And when it comes to making the project open source, rigor matters even more. Open source is like Shaolin discipline 🥷. It looks light and fluid from the outside, but behind it lies strict rules and daily practice. A simple .gitignore, the separation of secrets, and rotation policies may look like details, yet they prevent 90% of the incidents that typically harm community projects. Trust is not a slogan, it is a discipline encoded directly into the workflow.</p><h3>8 — Getting Started (5-Minute Guide)</h3><ul><li>Python 3.10+</li><li>LLM API access (e.g. OpenAI) → never commit the key.</li><li>Install deps:</li></ul><pre>pip install -r requirements.txt<br># streamlit, openai, gspread, google-cloud-secret-manager, httpx, spaCy FR, etc.</pre><p><strong>Secrets (local)</strong></p><ul><li>Export OPENAI_API_KEY according to your OS.</li><li>Verify runtime reads it (snippet in README).</li></ul><p><strong>Secrets (GCP)</strong></p><ul><li>Store keys in <strong>Secret Manager</strong> and mount via --set-secrets (Cloud Run).</li></ul><p><strong>Launch</strong></p><pre>streamlit run app.py</pre><h3>9 — Roadmap (Contribution Areas)</h3><ul><li><strong>Anonymization</strong>: regex + FR NER → next: context-aware rules + transformers.</li><li><strong>Observability</strong>: KPIs now → next: automatic eval (factuality, harm, attacks).</li><li><strong>Knowledge &amp; RAG</strong>: structured instruct now → next: RAG over HAS/WHO refs.</li><li><strong>Sovereignty</strong>: external LLM now → next: self-hosted, HDS compliance.</li><li><strong>Clinical Governance</strong>: disclaimers now → next: editorial board, escalation playbooks.</li></ul><h3>10 — FAQ</h3><p><strong>Q1 — Why Streamlit, not React?</strong><br> Fast learning. UX iteration speed &gt; front sophistication at early stage.</p><p><strong>Q2 — Why keep OpenAI if sovereignty is the goal?</strong><br> To prove usage value first. Design allows <strong>backend swap</strong> later.</p><p><strong>Q3 — Where are the secrets?</strong><br> In Secret Manager, injected at runtime, never committed.</p><p><strong>Q4 — How to avoid sensitive data storage?</strong><br> By design: KPI logging only, minimization documented, feature flags disable text logs.</p><h3>11 — Open Source Governance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tj1xSkdS0BLM5pSv30Bk7w.png" /><figcaption>Image credit © 1985 Warner Bros. Entertainment Inc. / Amblin Entertainment — The Goonies</figcaption></figure><p>Opening EVI Kids = opening the discussion:</p><ul><li><strong>Developers</strong>: advanced anonymization, red teaming, RAG.</li><li><strong>Doctors</strong>: response formats, orientation playbooks.</li><li><strong>Privacy experts</strong>: GDPR audits, consent models.</li><li><strong>Designers</strong>: parent/caregiver UX, alerts, accessibility.</li></ul><p>The humble .gitignore is the cornerstone: it keeps pull requests free of secrets. Combined with pre-commit hooks, it becomes the daily discipline that keeps the repository healthy.</p><h3>Conclusion, Scaling Trust in Tech</h3><p>EVI Kids is not just a prototype it is a proof that digital health can be both innovative and trustworthy when ethics, compliance, and open source discipline are built into the architecture from day one.</p><p><strong>Technically</strong>: simple, clean stack (Streamlit + LLM + anonymization + observability), portable and replaceable.</p><ul><li><strong>Operationally</strong>: secret separation + runtime injection + documented rotation ensure safe replication.</li><li><strong>Ethically</strong>: minimization &amp; privacy by design are <strong>coded into the architecture</strong>, not wishful thinking.</li></ul><p>EVI Kids is a <strong>commons</strong>, a foundation for pediatric healthcare to adopt open, reliable, respectful digital tools. If this vision speaks to you: <strong>fork it, test it, challenge it, and above all contribute.</strong></p><h3>Explore &amp; Connect</h3><ul><li><strong>Repository</strong> → <a href="https://www.linkedin.com/in/josselin-guarnelli/">contact me!</a></li><li><strong>Vision of Evi </strong>→ <a href="https://medium.com/@JossGuarnelli/a-free-pediatric-ai-to-support-families-caregivers-and-communities-a6cdb491ec22">Medium Article</a></li><li><strong>Let’s Connect</strong> → <a href="https://www.linkedin.com/in/josselin-guarnelli/">Josselin Guarnelli on LinkedIn</a></li></ul><p>Tags: Generative AI · LLM · GDPR compliance · Healthcare AI · Open Source HealthTech</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10c1f183ee2c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Free Pediatric AI to Support Families, Caregivers, and Communities]]></title>
            <link>https://medium.com/@JossGuarnelli/a-free-pediatric-ai-to-support-families-caregivers-and-communities-a6cdb491ec22?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6cdb491ec22</guid>
            <category><![CDATA[parenting]]></category>
            <category><![CDATA[ai-ethics]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[healthcare]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Mon, 18 Aug 2025 15:31:01 GMT</pubDate>
            <atom:updated>2025-08-18T15:31:01.839Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ah1h-ZN_xqLSlfxrzapfHg.jpeg" /></figure><h3>Why this choice?</h3><p>Because in 1978, my parents knocked on several doctors’ doors without ever receiving a real answer. Six years later, my brother passed away. I swore that no parent should ever have to walk that cold hallway between doubt and fear again.<br> No family should see their trust betrayed, as mine did — believing they were supporting life through a foundation, only to discover that even hope can be twisted when it isn’t protected by ethics.</p><h3>1 — The context in two numbers</h3><ul><li><strong>1 in 3 children in France</strong> are taken to the ER for cases that do not qualify as medical emergencies. This number reflects our collective difficulty to respond early, simply, without filters or judgment.</li><li><strong>40% of parents</strong> admit they’ve held back from asking a medical question for fear of being judged, not taken seriously, or sent into a complex system. That silence often precedes delayed diagnoses, sleepless nights, and doubts that only multiply.</li></ul><p><em>“Mom, why do I still have this bruise?”</em> — I still hear that question asked at the corner of a table, one Wednesday in 1983. So many families have this vital need: to be heard, to ask without fear, to simply know, to be reassured. To be there. Always there.</p><h3>2 — What we do today, concretely</h3><p>EVI-Kids answers <strong>all pediatric questions, 24/7, with no account and no ads. </strong>If a request even remotely resembles a sign of severity: <em>Call 911 (or 15 in France). No half-measures. </em>Every answer includes a button: <em>“A Question or Suggestion? We’ll reply fast.”</em> Your feedback lands directly in our team Slack — sometimes we react in less than ten minutes.</p><h3>3 — Where we want to be in five years</h3><p>We dream that <em>“Write to EVI”</em> becomes as natural a reflex as <em>“Dial 911”</em> — but <strong>without siphoning your data or pushing a “premium” plan on the day of your panic.</strong></p><h3>4 — Governance</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*unFzm_UlVF0quYbC2tGFww.png" /><figcaption>(Promise: we’ll never make more sections than needed — this one is worth it.)</figcaption></figure><h3>5 — Voluntary limits (because not everything is in the code)</h3><ul><li>Human review before any new content goes live.</li><li>Alert threshold: if the red light flashes, we direct you to a 24/7 pediatric hotline.</li><li>Continuous bias audits — yes, even the ones that hurt our ego.</li><li>Low-tech Plan B: in case of outage or “white zone,” a simple SMS works.</li></ul><h3>6 — What we will never do</h3><ul><li>Sell a single comma of your data.</li><li>Create a “Gold Pass” or any other premium gimmick.</li><li>Slip an insurance ad between two health tips.</li><li>Pretend a chatbot can replace a pediatrician.</li></ul><p><strong>EVI doesn’t examine; EVI enlightens.</strong></p><h3>One test is worth a thousand slides</h3><p>📱 <a href="https://www.evi-kids.health/">https://www.evi-kids.health</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6cdb491ec22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Build a Care Coordination Workflow with AI Agents using LangGraph]]></title>
            <link>https://medium.com/@JossGuarnelli/how-to-build-a-care-coordination-workflow-with-ai-agents-using-langgraph-0dfb9e561290?source=rss-61c405c54317------2</link>
            <guid isPermaLink="false">https://medium.com/p/0dfb9e561290</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[langchain]]></category>
            <category><![CDATA[science]]></category>
            <category><![CDATA[medical]]></category>
            <dc:creator><![CDATA[Josselin Guarnelli]]></dc:creator>
            <pubDate>Thu, 10 Apr 2025 15:23:08 GMT</pubDate>
            <atom:updated>2025-05-01T12:31:32.203Z</atom:updated>
            <content:encoded><![CDATA[<h4>A structured AI workflow for healthcare using LangGraph and LangChain. From protocol analysis to patient updates — all visualized in Streamlit</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8MBwXHLlmDCg-fYLRTyTCw.png" /></figure><h3>1 — Context and Motivation</h3><p>The evolution of agent orchestration frameworks fascinates me. For a care coordination project, I experimented with creating a workflow capable of:</p><ul><li><strong>Analyzing a Medical Protocol:</strong> Extracting the patient’s name, age, treatment details, and exam information from an unstructured text.</li><li><strong>Planning Interventions:</strong> Generating a schedule for treatments (e.g., injections, exams).</li><li><strong>Notifying the Patient and Updating Records:</strong> Sending a summary message to the patient and updating their record.</li></ul><p>To accomplish this, I chose <strong>LangGraph</strong>, a tool that allows for composing workflows of autonomous agents. Beyond the execution logic, I found it essential to visualize the execution path — much like using DAGs in Airflow — to clearly understand, debug, and communicate the operational flow.</p><h3>2 — Workflow Architecture</h3><p>The workflow I built consists of several key steps, each represented by an autonomous agent:</p><blockquote><strong>- Protocol Analysis</strong><br>An agent receives a raw text describing a medical protocol, processes it, and extracts structured information (e.g., patient’s name, age, type of injection, duration and frequency, exam details).</blockquote><blockquote><strong>- Injection Planning</strong><br>A second agent generates a treatment schedule (e.g., “Day 1 at 10AM, Day 2 at 10AM, …”).</blockquote><blockquote><strong>- Exam Scheduling</strong><br>An agent interprets medical exam instructions and schedules the deadlines.</blockquote><blockquote><strong>- Notification</strong><br>An agent dedicated to creating and sending a summary message to the patient.</blockquote><blockquote><strong>- Record Update</strong><br>An agent that updates the patient’s record with all the information.</blockquote><p>Each step in the workflow is implemented as a Python function that accepts a shared state (a dictionary) and returns updates to that state. The shared state is represented by a type called FollowupState (which holds the runtime information of the workflow). Below are simplified code examples for two agent functions, illustrating how they process and update this state:</p><pre>def node_analyze(state: FollowupState) -&gt; dict:<br>    protocol = analyze_protocol(state[&quot;input_text&quot;])<br>    if not isinstance(protocol, dict) or &quot;patient&quot; not in protocol:<br>        raise ValueError(&quot;Invalid analysis: Patient name was not detected.&quot;)<br>    return {<br>        &quot;protocol_info&quot;: protocol,<br>        &quot;patient_name&quot;: protocol.get(&quot;patient&quot;, &quot;Unknown Name&quot;)<br>    }<br><br>def node_injections(state: FollowupState) -&gt; dict:<br>    return {&quot;injection_info&quot;: plan_injections(state[&quot;protocol_info&quot;])}</pre><p>For more details about the type FollowupState and its usage, please refer to the <a href="https://github.com/langchain-ai/langgraph">LangGraph GitHub Repository</a> or relevant project documentation.</p><p>These functions are then assembled in a LangGraph, orchestrating the complete path from text input to the notification and record update.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rEukd-H9daBf4__1lLsaOg.png" /></figure><p><em>Note:</em> At this stage, the workflow is synchronous and relatively linear. For dynamic or asynchronous applications, the architecture would need to be rethought.</p><h3>3 — Visualization Benefits</h3><p>From the initial design phase, I identified a crucial need: to have a graphical representation of the execution path. Such a diagram allows you to:</p><blockquote><strong>- Follow the Information Flow:</strong> Visualize how the state evolves from the initial analysis to the final record update.</blockquote><blockquote><strong>- Quickly Detect Anomalies:</strong> Identify precisely where an error or blockage occurs.</blockquote><blockquote><strong>- Facilitate Communication:</strong> Provide technical and clinical teams with a clear overview that aids in process review, training, or workflow optimization.</blockquote><p>To generate this visualization, I used the draw_mermaid() method from LangGraph, which produces a Mermaid-formatted text diagram. I then integrated this Mermaid string into my Streamlit application using the streamlit-mermaid component.</p><pre>graph TD<br>    __start__([Start]):::first<br>    analyze[Analyze Protocol]<br>    injections[Plan Injections]<br>    exams[Schedule Exams]<br>    notify[Notify Patient]<br>    update[Update Record]<br>    __end__([End]):::last<br><br>    __start__ --&gt; analyze<br>    analyze --&gt; injections<br>    injections --&gt; exams<br>    exams --&gt; notify<br>    notify --&gt; update<br>    update --&gt; __end__<br><br>    classDef first fill:#f0f8ff,stroke:#333,stroke-width:2px<br>    classDef last fill:#e6f2ff,stroke:#333,stroke-width:2px</pre><p>Or directly in your Streamlit app with :</p><pre>from streamlit_mermaid import st_mermaid<br>st_mermaid(diagram_code)</pre><h3>4 — Technical Challenges and Solutions</h3><h4><strong>4.1 Version Conflicts and Dependency Issues</strong></h4><p><strong>Problem : </strong>The LangGraph version I used (0.2.32) requires that <em>langgraph-checkpoint</em> be between 2.0.0 and 3.0.0, while I initially had an older version (1.0.12). This caused import errors (e.g., inability to import StateGraph from langgraph.graph).</p><p><strong>Solution : </strong>I updated <em>langgraph-checkpoint</em> to a compatible version by modifying my dependency file to install a version in the 2.0.0–3.0.0 range.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ncUWKt_nQBY1qvr3wvUT3Q.png" /></figure><h4><strong>4.2 JSON Response Cleaning and Parsing</strong></h4><p><strong>Problem: </strong>The API response included Markdown delimiters (json ... ), which prevented json.loads from parsing the JSON string correctly.</p><p><strong>Solution:</strong> I added a cleaning step in my analyze_protocol function to detect and remove these delimiters before attempting to parse the JSON.</p><h3>5 — Generating and Displaying the Mermaid Diagram</h3><p><strong>Problem : </strong>Initially, despite calling st_mermaid(diagram) in Streamlit, the diagram was not displayed. The method to generate the diagram was not available on the initial graph object; it needed to be called on the compiled object using app.get_graph().draw_mermaid().</p><p>Additionally, the streamlit-mermaid component might encounter issues if the Mermaid code had syntax errors or if it was not up to date.</p><p><strong>Solution : </strong>I modified the code to use the compiled graph object (app) and its method draw_mermaid() to correctly generate the diagram.</p><p>I ensured that the streamlit-mermaid component was updated to version 0.3.0, which is compatible with my version of Streamlit.</p><p>I added debug messages (e.g., st.write(&quot;Generated Diagram:&quot;, diagram)) to verify the Mermaid string content.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/1*L6MGToiEv8AxT6GF5kMTQw.png" /><figcaption>Image showing your Streamlit app with the diagram displayed</figcaption></figure><h3>Conclusion</h3><p>Implementing a workflow of autonomous agents with LangGraph provides a modular approach to orchestrate processes — in my case, care coordination. Visualizing the workflow state using Mermaid not only aids in debugging but also makes the system transparent and easier to understand for technical and clinical teams.</p><p>Through managing dependency conflicts, adapting to OpenAI API changes, cleaning JSON responses, and resolving Mermaid rendering issues, I created an intuitive Streamlit interface that displays both a detailed textual summary and an interactive diagram of the workflow.</p><p><em>Note: As libraries like LangGraph, LangChain, and the associated components evolve rapidly, please ensure you consult their official GitHub repositories or documentation for the latest methods and compatibility guidelines. For example, verify that functions like </em><em>draw_mermaid() are still supported or if there are any changes in dependency requirements.</em></p><h3>Resources</h3><ul><li><strong>Mermaid Live Editor:</strong> <a href="https://mermaid.live/">https://mermaid.live/</a><br> <em>For testing and validating your Mermaid diagrams.</em></li><li><strong>Mermaid Documentation:</strong> <a href="https://mermaid-js.github.io/mermaid/#/">https://mermaid-js.github.io/mermaid/#/</a><br> <em>For details on Mermaid syntax and functionality.</em></li><li><strong>LangGraph GitHub Repository:</strong> <a href="https://github.com/langchain-ai/langgraph">https://github.com/langchain-ai/langgraph</a><br> <em>The official source and documentation for LangGraph.</em></li><li><strong>LangChain GitHub Repository:</strong> <a href="https://github.com/langchain-ai/langchain">https://github.com/langchain-ai/langchain</a><br> <em>For project guides and related resources.</em></li><li><strong>OpenAI-Python Migration Discussion:</strong> <a href="https://github.com/openai/openai-python/discussions/742">https://github.com/openai/openai-python/discussions/742</a><br> <em>For understanding API evolution and migrating your code.</em></li><li><strong>Streamlit Documentation:</strong> <a href="https://docs.streamlit.io/">https://docs.streamlit.io/</a><br> <em>For official best practices when using third-party components.</em></li><li><strong>Streamlit Mermaid Component GitHub:</strong> <a href="https://github.com/okld/streamlit-mermaid">https://github.com/okld/streamlit-mermaid</a><br> <em>For the component used to render Mermaid diagrams in Streamlit.</em></li><li><strong>Mermaid GitHub Repository:</strong> <a href="https://github.com/mermaid-js/mermaid">https://github.com/mermaid-js/mermaid</a><br> <em>For updates on Mermaid and to view the source code.</em></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0dfb9e561290" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>