<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Godel Labs on Medium]]></title>
        <description><![CDATA[Stories by Godel Labs on Medium]]></description>
        <link>https://medium.com/@marketing_83369?source=rss-2727f26ea468------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Godel Labs on Medium</title>
            <link>https://medium.com/@marketing_83369?source=rss-2727f26ea468------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:42:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@marketing_83369/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[When Microsoft open-sourced the Agent Governance Toolkit (AGT) in April 2026, it filled a gap the…]]></title>
            <link>https://medium.com/@marketing_83369/when-microsoft-open-sourced-the-agent-governance-toolkit-agt-in-april-2026-it-filled-a-gap-the-e625ba20026a?source=rss-2727f26ea468------2</link>
            <guid isPermaLink="false">https://medium.com/p/e625ba20026a</guid>
            <category><![CDATA[ai-agent-security]]></category>
            <category><![CDATA[generative-ai-security]]></category>
            <category><![CDATA[agt]]></category>
            <dc:creator><![CDATA[Godel Labs]]></dc:creator>
            <pubDate>Thu, 28 May 2026 13:51:56 GMT</pubDate>
            <atom:updated>2026-05-28T13:51:56.834Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BsRx3yPQcS5Sa2w3AXkgZw.png" /></figure><p>When Microsoft open-sourced the <strong>Agent Governance Toolkit (AGT)</strong> in April 2026, it filled a gap the industry had been circling for two years. As enterprise AI moved from chat to action — agents calling tools, mutating data, talking to other agents — security teams kept asking the same question: <em>who governs what these agents actually do?</em> AGT is Microsoft’s answer, and it’s a serious one.</p><p>This post is a practitioner’s read of AGT. What it is, what it gets right, and — just as important — where it deliberately stops, so teams know what they still have to bring to complete the picture.</p><h3>What AGT Is</h3><p>AGT is an <strong>MIT-licensed, open-source toolkit</strong> for governing the runtime behavior of autonomous AI agents. It’s polyglot (Python, TypeScript, .NET, Rust, Go) and integrates with the major agent frameworks the industry actually uses — LangChain, CrewAI, AutoGen, OpenAI Agents, Google ADK, Semantic Kernel, AWS Bedrock, Microsoft Agent Framework, and 20+ more. Microsoft has stated its intent to move governance of the project into a neutral foundation over time.</p><p>The headline claim is bold and specific: AGT is the first toolkit to address all ten OWASP Agentic AI risks with <strong>deterministic, sub-millisecond policy enforcement</strong>. Every tool call, resource access, and inter-agent message is evaluated against policy before execution — auditable and reproducible.</p><p>That framing matters. AGT is not a guardrail wrapper around prompts. It’s a runtime governance layer that intercepts what the agent actually <em>does</em>.</p><h3>The Modules That Define AGT</h3><p>AGT ships as seven independently installable packages, each addressing a distinct concern in the agent lifecycle:</p><p>Agent OS</p><p>The stateless policy engine that intercepts every agent action before it executes. Supports YAML, OPA Rego, and Cedar policies. Sub-millisecond evaluation. This is the heart of AGT.</p><p>Agent Mesh</p><p>Cryptographic identity for agents and inter-agent trust scoring. One agent calling another? Mesh verifies who’s calling and applies a trust policy. Notably, this gives AGT its own agent-identity primitives without requiring an external IdP.</p><p>Agent Runtime</p><p>Execution sandboxing with privilege rings, saga orchestration for multi-step workflows, and emergency termination. Contains what a misbehaving agent can touch.</p><p>Agent SRE</p><p>Production reliability practices applied to autonomous agents: SLO monitoring, circuit breakers, cost caps, chaos engineering.</p><p>Agent Compliance</p><p>Tamper-evident audit logging via OpenTelemetry, with automated mapping to EU AI Act, SOC 2, HIPAA, and GDPR controls.</p><p>Agent Marketplace</p><p>Plugin and MCP server lifecycle management with supply-chain trust scoring. Stops an agent from loading unvetted extensions at runtime.</p><p>PromptInjectionDetector</p><p>A local, in-memory regex and heuristic scanner that runs at the input boundary and after every tool fetch. Sub-millisecond, fail-closed.</p><p>A few additional packages — Agent Lightning (RL policy enforcement) and Agent Hypervisor (hardware-level isolation) — extend the model further but address more specialized concerns.</p><p>The architecture is deliberately modular. You don’t have to take everything. The policy engine alone is useful; so is the runtime; so is Mesh. They compose.</p><h3>What AGT Gets Right</h3><p>It’s worth being specific about Microsoft’s contribution here, because it’s substantial.</p><p>The license and the neutrality</p><p>AGT is MIT-licensed and explicitly intended for a foundation home. Microsoft could have made this proprietary, Azure-locked, or Copilot-only. They did the opposite. This is a real act of standards-building, and the industry benefits.</p><p>The framework reach</p><p>Most “agent platform” tools assume you’re on one stack. AGT hooks the <em>native</em> extension points of LangChain, CrewAI, AutoGen, Google ADK, AWS Bedrock, and others. You don’t rewrite your agent code to adopt it. That portability is genuinely difficult engineering.</p><p>The policy engine choices</p><p>Supporting YAML, OPA Rego, <em>and</em> Cedar means teams aren’t forced to learn a proprietary DSL. OPA Rego in particular is a CNCF-graduated standard with mature tooling and broad familiarity.</p><p>Coverage of OWASP Agentic Top 10</p><p>This isn’t a marketing claim — the toolkit maps explicit controls to each risk, and the modular packages line up with real categories of harm (tool misuse, prompt injection, supply chain, runaway execution, inter-agent abuse).</p><p>Deterministic enforcement</p><p>The decision to keep the policy gate deterministic and sub-millisecond is the right call. Authorization decisions need to be auditable and reproducible. Probabilistic gates can’t be either.</p><p>This is a credible foundation. Teams building agent platforms should adopt AGT rather than reinvent it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-iOUHk88GBmF571Qi5Eu8w.png" /></figure><h3>A Concrete Walkthrough — Wiring AGT Into an Existing Agent</h3><p>Take a common case: a <strong>LangGraph-powered project management assistant for Jira</strong>. It looks up tickets by key to retrieve summaries, descriptions, statuses, comments, and attachments; runs JQL queries across many issues to aggregate status updates, find blockers, and generate reports; manages a long-term key-value memory; and sends notifications to Slack.</p><p>An assistant like this typically has content guardrails in place — scanners running over user inputs, agent responses, and retrieved Jira attachments to catch prompt injection and malicious payloads. Those handle the <em>data layer</em>: the contents the agent sees and emits. What they cannot handle is the <em>action layer</em>: the fact that the LLM might decide to call a tool — write to memory, post to Slack, run a broad JQL query — for reasons that look benign on the surface but violate policy. The agent needs a deterministic gate on every action.</p><p>That is exactly the gap AGT fills. Here is what wiring AGT into an agent like this looks like in practice.</p><p>Step 1 — Define a governance policy</p><p>AGT policies are declarative. Start with a YAML file that explicitly whitelists the tools the agent is allowed to invoke. Tighter parameter constraints, sequence rules, and conditional logic come next, but the first cut is just the allowlist.</p><pre>name: jira-agent-policy<br>version: &quot;1.0&quot;<br>defaults:<br>  action: allow</pre><pre>rules:<br>  - name: allow-jira-fetch<br>    condition: &quot;tool_name == &#39;fetch_jira_ticket&#39;&quot;<br>    action: allow<br>    priority: 100<br>  # Additional rules for JQL search, memory writes, notifications, etc.</pre><p>The policy file is living code: versioned, reviewed, with a regression suite alongside.</p><p>Step 2 — Import the governance entry point</p><p>Pull govern from the agentmesh package — AGT&#39;s policy-engine integration:</p><pre>from agentmesh.governance import govern</pre><p>Step 3 — Wrap govern in a framework-aware decorator</p><p>This is the step that does the real work. govern itself is generic; LangGraph passes tool calls with its own schema, and the policy engine needs richer context — the tool name, a structured action object — than the default wrapper provides. The clean answer is a small custom decorator that does three things: defines how denials are surfaced back to the agent, wraps the function with govern, and patches the context builder so every call presents the right metadata to the policy engine.</p><pre>def governed(policy: str):<br>    def decorator(fn):<br>        def handle_deny(decision):<br>            return f&quot;Action denied by policy: {decision.reason}&quot;</pre><pre>        wrapper = govern(fn, policy=policy, on_deny=handle_deny)</pre><pre>        original_build_context = wrapper._build_context<br>        def custom_build_context(args: tuple, kwargs: dict) -&gt; dict:<br>            ctx = original_build_context(args, kwargs)<br>            ctx[&quot;tool_name&quot;] = fn.__name__<br>            ctx[&quot;action&quot;] = {&quot;type&quot;: &quot;call&quot;, &quot;tool_name&quot;: fn.__name__}<br>            return ctx</pre><pre>        wrapper._build_context = custom_build_context<br>        wrapper.__name__ = fn.__name__<br>        return wrapper<br>    return decorator</pre><p>This is the kind of glue most teams write once and reuse across every tool. It is also where framework-specific quirks get absorbed so the rest of the agent code stays untouched.</p><p>Step 4 — Stack the decorator alongside LangGraph’s @tool</p><p>Apply both decorators to every tool function. LangGraph keeps doing tool discovery; AGT intercepts the call before it executes.</p><pre>@tool<br>@governed(&quot;agent/policy.yaml&quot;)<br>def fetch_jira_ticket(ticket_key: str) -&gt; str:<br>    &quot;&quot;&quot;Fetch a Jira ticket&#39;s details...&quot;&quot;&quot;<br>    ...</pre><p>The agent code itself does not change. Only the tool boundaries gain a policy gate.</p><p>Step 5 — Optional: post-execution output scanning</p><p>The custom @governed wrapper is a natural place to also invoke post-execution scanning — passing the tool&#39;s raw output back through the existing content guardrails before returning it to the LLM&#39;s context. This closes the perimeter: AGT governs the <em>intent</em> to act; the content scanners govern the <em>results</em> of the action.</p><p>Before and After</p><p>Without AGT, an agent like this relies on probabilistic LLM behavior plus content filtering — a model that usually does the right thing, with guardrails on the inputs and outputs. With AGT in place, every tool call is intercepted by a deterministic, sub-millisecond policy engine. The agent cannot invoke a tool the policy has not authorized, regardless of what the LLM decides. That is the difference between <em>trusting the model</em> and <em>verifying the action</em> — and it is exactly what the action layer is supposed to do.</p><h3>The Gaps Worth Knowing</h3><p>AGT is a foundation, not a complete solution — and Microsoft is open about that. The toolkit scopes itself deliberately, expecting teams to layer in complementary tools for several important concerns. Here are the gaps that matter most when planning a real deployment.</p><p>1. PromptInjectionDetector — pattern-based, by design</p><p>AGT’s built-in injection detector is a local, regex-and-heuristic scanner with a configurable blocklist and risk-score threshold. Its strengths are exactly what its design promises: it’s fast, deterministic, runs in-process, and adds essentially no latency. It catches the obvious — ignore previous instructions, &lt;system&gt; markers, suspicious tool names, hidden Unicode.</p><p>Where it stops is also a function of that design. It cannot reason about meaning. A paraphrased jailbreak, a multi-turn semantic injection, a ROT13- or base64-encoded payload, or a novel attack pattern the rules haven’t seen — these can pass through. In practice, teams have demonstrated bypasses against the default ruleset with simple obfuscation. Microsoft’s own documentation is candid: detection effectiveness depends on threshold tuning against your specific threat model.</p><p>This is a known tradeoff: rules are deterministic and fast; semantic understanding requires a model. For threat-detection on adversarial content, that tradeoff matters. Teams running agents over untrusted retrieved content typically need a semantic, model-based detector in addition.</p><p>2. Azure Prompt Shields — capable, but cloud-bound</p><p>Microsoft does have a model-based prompt-injection detector: <strong>Prompt Shields</strong>, part of Azure AI Content Safety. It’s a genuinely capable detector that handles cases the regex layer can’t.</p><p>It is <em>not</em> part of AGT. It’s an Azure cloud API. That introduces three architectural realities worth planning for:</p><p>Latency</p><p>Prompt Shields is a remote call. Adding it to the hot path of every tool fetch and every retrieved chunk adds round-trip cost to a flow that AGT itself designed to be sub-millisecond.</p><p>Cost</p><p>Prompt Shields bills per text record (up to 1,000 Unicode characters per record). In autonomous agents, scan counts compound fast — a workflow that retrieves 20 chunks, loops 5 times, across 3 tools, can easily generate 300 shield scans per user interaction. At fleet scale, the bill is real.</p><p>Scope</p><p>Prompt Shields analyzes <em>text</em>. It doesn’t process binary artifacts directly — you parse and OCR first, then send the extracted text. Attacks hidden in malformed encodings, tiny fonts, or invisible characters that defeat OCR will defeat the shield. And Spotlighting (the indirect-injection defense) base64-tags untrusted content to lower its trust weight, which increases token usage and sometimes surfaces the tagging in model output.</p><p>None of these are flaws — they’re the natural cost of a centralized cloud classifier. For some deployments they’re fine; for high-throughput or air-gapped environments, they push teams toward local, on-prem alternatives.</p><p>3. Microsoft Purview — designed for the cold path</p><p>A frequent question: can Purview classify content for agents in real time? The honest answer is no, and the architecture documentation says so directly.</p><p>Purview is an excellent <strong>data-at-rest</strong> catalog. It scans SharePoint, OneDrive, SQL, and other repositories asynchronously, applies sensitivity labels, and exposes those labels as metadata. For agent governance, the right pattern is the <strong>hybrid hot-path / cold-path</strong> topology Microsoft recommends:</p><p>Cold path</p><p>Purview pre-classifies documents in the background and writes sensitivity labels to metadata.</p><p>Hot path</p><p>When an agent retrieves a document, AGT’s policy engine reads the pre-cached Purview label — not the file content — and enforces the policy in sub-millisecond.</p><p>Audit</p><p>AGT exports OpenTelemetry logs back to Purview for compliance reporting.</p><p>This works well. What it doesn’t do is classify content that isn’t yet in a Purview-scanned repository — ephemeral tool outputs, real-time user inputs, freshly scraped web content. Synchronously calling Purview APIs from the agent’s execution path is documented as an anti-pattern: high latency, rate-limit exhaustion, runaway cost. For hot-path classification, teams need a separate runtime classifier.</p><p>Purview also does not detect prompt injection or jailbreaks. It is a data classification and governance tool, not a threat detector. Active AI attacks require runtime tools — AGT’s local detector, Prompt Shields, or an alternative.</p><p>4. Authorization — where AGT deliberately stops at the agent</p><p>This is the area teams most often misjudge. AGT’s policy engine governs <strong>what an agent does</strong> — tool calls, inter-agent messages, plugin loads, resource access at the URL or endpoint level. It does this very well.</p><p>What AGT does <em>not</em> do is verify the human user behind the agent or make per-record data-access decisions. There is no IdP in AGT, no source-system ACL projection, no notion of “this user is the manager of the data subject.”</p><p>For the Microsoft estate, <strong>Microsoft Entra (and Entra Agent ID)</strong> closes much of this. Entra provides verified human identity via OIDC, Conditional Access, Identity Protection signals, lifecycle revocation, and — the headline — the <strong>Agent User Account / on-behalf-of (OBO)</strong> flow. With OBO, the agent acts <em>as</em> the user, presents the user’s token to the source, and the source enforces its own ACLs. For SharePoint, OneDrive, Exchange, Teams, and Graph-connected data, this is a complete answer.</p><p>The residual is what lives <em>outside</em> the Microsoft estate: per-record authorization for Workday, Salesforce, GitHub, SAP, ServiceNow, and on-prem databases where OBO doesn’t reach or isn’t faithfully supported. Domain business relationships beyond the basic org chart — hrbp_for, account_owner_for, territory_manager_for. Field-level access (e.g., suppressing a salary field on an otherwise-permitted record). Purpose-bound access and data residency at the resource level. These are layered on by complementary authorization platforms or built in-house.</p><p>There is also the service-account case. Many real agent deployments — multi-tenant agents, background workflows, agents calling legacy non-OAuth APIs — cannot use OBO. In those cases, the source sees only the application identity, and the user-level decision has to be made somewhere else, in the agent runtime. AGT provides the enforcement gate; the per-record decision needs an external Policy Decision Point.</p><p>5. Content classification — AGT is content-blind by design</p><p>A related point: AGT does not natively detect PII, PCI, PHI, secrets, or API keys in the content flowing through an agent. This is intentional — AGT is a policy enforcer, not a DLP scanner. It expects classification signals to come from elsewhere.</p><p>For data at rest, that elsewhere is Purview (cold path). For data in motion — the user’s chat prompt, an ephemeral tool output, a freshly scraped page — a runtime classifier is required. AGT will then enforce policy on the classification tag, but it won’t produce the tag.</p><p>6. Multimodal scope</p><p>The current AGT detection layer is text-oriented. Agents increasingly process documents, images, audio, and other binary artifacts. Attacks hiding in steganographic images, malicious PDFs, or audio prompts are an active threat surface. AGT’s policy engine is binary-agnostic — it’ll enforce whatever policy you write — but the <em>signals</em> needed to drive that policy across modalities require complementary detection layers.</p><h3>How To Think About Using AGT</h3><p>Adopt AGT as the runtime enforcement foundation. Don’t rebuild what it already gives you well — the policy engine, the sandbox, the kill switch, the supply-chain trust scoring, the inter-agent mesh, the audit logging. Reinventing any of these is engineering time spent on commodity.</p><p>Then plan the layers around it deliberately:</p><ul><li><strong>Pair with Entra (or your IdP) for identity.</strong> OBO is the cleanest answer for Microsoft-native data. Conditional Access and Identity Protection give you the device, location, and risk signals AGT’s policy can act on.</li><li><strong>Adopt the hot-path / cold-path topology for classification.</strong> Purview (or equivalent) handles at-rest sensitivity labeling; a runtime classifier handles the in-flight stream. AGT enforces on both.</li><li><strong>For authorization beyond the Microsoft estate</strong>, plan for a Policy Decision Point that handles per-record decisions for non-Microsoft sources, domain business relationships, purpose binding, residency, and the service-account case. This can be a complementary platform or an in-house build.</li><li><strong>For threat detection on retrieved content</strong> — especially in regulated or air-gapped environments where latency and data-egress constraints matter — pair AGT’s local detector with a semantic, model-based scanner that can run where the content lives. On-prem content-integrity platforms are emerging specifically for this layer.</li><li><strong>Treat AGT’s audit log as the spine</strong> of your agent compliance story. OpenTelemetry export to Purview, your SIEM, or your compliance portal gives you the trail regulators are going to ask about.</li></ul><p>The key shift is mindset: AGT is the foundation, not the whole house. Microsoft built it that way on purpose — modular, neutral, and meant to be composed with other tools.</p><h3>Summary</h3><p>The Agent Governance Toolkit is a serious contribution to a problem that needed one. It gives the industry a credible, open-source, framework-neutral foundation for governing what AI agents do at runtime — policy enforcement, sandboxing, supply-chain trust, inter-agent identity, audit. Microsoft’s decision to release it under MIT and move it toward foundation governance is the right move for the ecosystem.</p><p>It is also, by deliberate scope, a foundation rather than a complete solution. Semantic prompt-injection detection beyond pattern-matching, real-time content classification, per-record authorization for non-Microsoft data, and air-gapped deployments all sit outside what AGT itself does — and Microsoft expects teams to layer those in.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e625ba20026a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[When Documents Start Talking Back: How Hidden Instructions Hijack AI Agents]]></title>
            <link>https://medium.com/@marketing_83369/when-documents-start-talking-back-how-hidden-instructions-hijack-ai-agents-e7a341168995?source=rss-2727f26ea468------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7a341168995</guid>
            <category><![CDATA[data-security]]></category>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[ai-security]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Godel Labs]]></dc:creator>
            <pubDate>Tue, 12 May 2026 14:19:09 GMT</pubDate>
            <atom:updated>2026-05-12T14:22:23.376Z</atom:updated>
            <content:encoded><![CDATA[<p>AI agents are being asked to do something traditional software never had to do at scale: read untrusted human content and then act on it.</p><p>That sounds harmless until you realize what “content” now includes. Emails. PDFs. Web pages. Resumes. Reports. Support tickets. Shared docs. CRM notes. Knowledge base articles. Anything an agent can retrieve, summarize, reason over, or pass into a tool becomes part of its operating environment.</p><p>And that creates a new failure mode:</p><p>The document is no longer just data.</p><p>It can also be an attack.</p><h3><strong>The moment documents stop being passive</strong></h3><p>The core problem is simple. An agent is given a goal by a user, but it also consumes external content while trying to complete that goal. If that content contains hidden or malicious instructions, the agent may follow the document instead of the user.</p><p>That is the essence of indirect prompt injection.</p><p>Unlike a direct jailbreak, the attacker does not need a chat box. They only need to place adversarial text somewhere the agent is likely to read. The attack rides into the system inside seemingly ordinary content.</p><p>An invoice can tell the agent to ignore previous instructions.</p><p>A web page can hide text in HTML or metadata that never appears to a human reviewer.</p><p>A resume can include instructions to exfiltrate recruiter notes.</p><p>A report can inject malicious steps into a downstream workflow.</p><p>The user thinks they are asking the agent to summarize, compare, classify, or act. But the agent may be receiving a second set of instructions from the content itself.</p><p>When that happens, documents start talking back.</p><h3><strong>Greshake et al. named the problem</strong></h3><p>The foundational paper here is Kai Greshake et al.’s 2023 paper, *Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection*.</p><p>That paper made a crucial observation: LLM-integrated applications blur the line between data and instructions. Once a model reads retrieved content inside the same context window as system instructions and user intent, malicious text can compete with or override the original task.</p><p>This was not presented as a theoretical curiosity. Greshake et al. showed that indirect prompt injection could be used to remotely exploit real LLM-integrated applications by planting instructions in data likely to be retrieved. The paper also framed the consequences in security terms, including data theft, manipulation of application behavior, and control over downstream API usage.</p><p>That was the first big warning shot: if a system cannot reliably distinguish trusted instructions from untrusted content, then retrieval is not just a relevance problem. It is a security boundary problem.</p><h3><strong>Agents make the problem worse</strong></h3><p>If Greshake et al. showed that retrieved content can influence model behavior, InjecAgent showed what happens when that model is no longer just answering questions, but calling tools.</p><p>In *InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents*, Qiusi Zhan and coauthors moved the discussion from “can retrieved text change outputs?” to “can retrieved text make agents do harmful things?”</p><p>That shift matters.</p><p>Tool-integrated agents do not just summarize a malicious email or read a poisoned webpage. They may also send messages, query systems, write data, trigger workflows, or reveal sensitive information while trying to complete a task.</p><p>InjecAgent gave the field a benchmark for this risk and measured it at scale. The paper evaluates 30 different LLM agents across 1,054 test cases spanning 17 user tools and 62 attacker tools. The results are hard to dismiss as edge cases. The authors report that agents are vulnerable to indirect prompt injection attacks, and that a ReAct-prompted GPT-4 agent was successfully attacked 24% of the time in their evaluation. In an enhanced setting, the attack success rate rose further, nearly doubling for that GPT-4 setup.</p><p>That finding changes the conversation in two ways.</p><p>First, it shows that the issue is not limited to quirky demos or early prototypes. It persists in structured agent settings with tool use.</p><p>Second, it shows that the consequence is not only wrong text generation. The consequence can be harmful action or private-data exfiltration.</p><p>Once an agent can read and act, hidden instructions in content become a path to behavior hijacking.</p><h3><strong>AI Agent Traps broadens the frame</strong></h3><p>The 2026 SSRN preprint *AI Agent Traps* by Matija Franklin and coauthors provides the broader conceptual overlay.</p><p>Its key contribution is to argue that the threat is bigger than prompt injection narrowly defined. The environment itself becomes adversarial. In that framing, malicious content is not just a bad string in a prompt. It is part of a larger class of traps designed to manipulate how agents perceive, reason, remember, and act.</p><p>The paper describes six trap categories:</p><p>- Content Injection Traps</p><p>- Semantic Manipulation Traps</p><p>- Cognitive State Traps</p><p>- Behavioural Control Traps</p><p>- Systemic Traps</p><p>- Human-in-the-Loop Traps</p><p>That taxonomy is useful because it explains why “just filter prompts” is not enough.</p><p>Some attacks hide instructions in the machine-readable version of a page but not the human-visible one. Some bias the agent’s reasoning or verification process. Some poison long-term memory or retrieved knowledge. Some hijack tool use directly. Some create cascading failures across interacting agents. Some even exploit the human reviewer who is supposed to be supervising the system.</p><p>In other words, indirect prompt injection is not the whole story. It is the front door into a much larger attack surface.</p><h3><strong>The real security lesson</strong></h3><p>The deepest lesson across these papers is that enterprise AI systems should not trust content just because they can retrieve it.</p><p>That assumption is built into too many current architectures. If a connector can access the file, and retrieval says it is relevant, the system often treats it as usable context. But relevance is not the same thing as safety, and access is not the same thing as trust.</p><p>For agents, this distinction is critical.</p><p>A document may be:</p><p>- relevant but malicious</p><p>- accessible but unauthorized for this action</p><p>- visible to the user but unsafe to ground on</p><p>- topically correct but stale or superseded</p><p>- clean for human reading but weaponized for machine parsing</p><p>That means the question is no longer just:</p><p>`Can the agent find this document?`</p><p>The real question is:</p><p>`Should this document be allowed to influence the agent at all?`</p><p>That is the control point most AI systems are still missing.</p><h3><strong>Why this matters now</strong></h3><p>The risk is accelerating because enterprise AI is moving from chat to action.</p><p>The moment an agent can read a support ticket, inspect an attachment, search the web, query a CRM, draft an email, and trigger a workflow, every content source becomes part of the runtime attack surface.</p><p>This is why the phrase “documents talking back” matters. It captures the mental model shift.</p><p>In older software, a document was usually an object to be stored, displayed, or parsed.</p><p>In agentic systems, a document can become an active influence over reasoning and action. It can compete with user intent. It can redirect workflows. It can leak secrets. It can turn trusted tooling against the user.</p><p>That is not a UX bug. It is a security architecture problem.</p><h3><strong>What enterprises need to do differently</strong></h3><p>The practical implication is that AI systems need a content qualification layer before retrieved content reaches the model or agent.</p><p>Not every file should be treated as safe AI context just because it is available through a connector or returned by a tool. Systems need to evaluate whether content is:</p><p>- relevant to the task</p><p>- permitted for that user and agent</p><p>- trustworthy enough to ground an answer or action</p><p>- safe enough to expose to the model</p><p>That is the missing boundary.</p><p>Without it, malicious instructions can enter through ordinary business content and steer agents from the inside.</p><p>With it, enterprises can start making explicit decisions about which content should be allowed, flagged, redacted, reviewed, or blocked before it shapes model behavior.</p><h3><strong>Conclusion</strong></h3><p>Greshake et al. showed that retrieved content can smuggle in instructions. InjecAgent showed that tool-integrated agents can be manipulated into harmful behavior and data exfiltration. *AI Agent Traps* showed that this belongs to a broader class of environmental attacks on autonomous systems.</p><p>Taken together, the message is clear:</p><p>the attack surface for AI agents is not just the model and not just the prompt.</p><p>It is everything the agent reads.</p><p>When documents start talking back, context becomes the new security boundary.</p><p>That is why context integrity matters. The future of AI security will depend not only on protecting models and filtering outputs, but on deciding which external content is allowed to become live runtime context in the first place.</p><h3>Sources</h3><p>- Greshake et al. (2023), *Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection*: <a href="https://arxiv.org/abs/2302.12173">https://arxiv.org/abs/2302.12173</a></p><p>- Zhan et al. (2024), *InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents*: <a href="https://aclanthology.org/2024.findings-acl.624/">https://aclanthology.org/2024.findings-acl.624/</a> and <a href="https://arxiv.org/abs/2403.02691">https://arxiv.org/abs/2403.02691</a></p><p>- Franklin et al. (2026), *AI Agent Traps* (SSRN abstract page referenced via DOI): <a href="https://doi.org/10.2139/ssrn.6372438">https://doi.org/10.2139/ssrn.6372438</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7a341168995" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>