TL;DR: Researchers found that eight published indirect prompt injection defenses all broke under adaptive attacks above 50%, while a separate multi-lab study bypassed twelve defenses at more than 90%, according to ARMO and cited research. The pattern is clear: upstream text classifiers collapse together, while capability and action-boundary controls fail in different places.
At a glance
What this is: ARMO argues that indirect prompt injection defenses fail when they rely on upstream text inspection, because adaptive attackers can break every published detector style tested.
Why it matters: This matters because agent security programmes need controls that bound action and authority, not just controls that try to classify hostile text after it is already in the prompt.
Context
Indirect prompt injection is a control problem, not just a text-filtering problem. The failure mode appears when untrusted content can influence an agent’s decision path and then steer tool use, data movement, or state changes. For IAM and NHI programmes, that matters because the real question is whether authority is constrained at the point of action, not whether the input looked suspicious.
ARMO’s analysis places the debate around where a control sits relative to the model’s decision. Upstream detectors inspect text, while capability and behavioral controls constrain what the agent may do with that text. That distinction is central for anyone governing agentic AI, because the security boundary has to follow the decision and execution path, not only the prompt content.
Key questions
Q: What breaks when indirect prompt injection defenses rely on text inspection?
A: They collapse against an adaptive attacker because the control is trying to classify hostile language after it has already reached the model. Once the attacker can vary wording and probe the boundary, detector-style systems, delimiter tricks, and prompt rewriting all become part of the target surface. Durable protection has to move from text classification to authority and action boundaries.
Q: Why do capability boundaries reduce indirect prompt injection risk?
A: They reduce risk because they constrain what the agent can do, regardless of how the text inside the prompt tries to steer it. If the agent cannot reach a tool, change state, or access sensitive data, an injected instruction has far less room to cause harm. Capability controls do not solve intent detection, but they do bound blast radius.
Q: How do security teams know whether per-agent enforcement is working?
A: Per-agent enforcement is working when observed behaviour and allowed behaviour converge. Teams should see the agent using only the APIs, data sources, and output routes it actually needs, with unused permissions removed and unexpected tool calls blocked. If the agent can still reach unrelated financial systems, the enforcement boundary is too loose.
Q: Should organisations use prompt filtering or action-boundary controls for agent security?
A: They should use both, but for different jobs. Prompt filtering reduces opportunistic noise and cheap attacks, while action-boundary controls stop coerced behaviour inside approved workflows. If the system handles untrusted input and can write, call tools, or reach sensitive systems, the action boundary is the control that matters most.
Technical breakdown
Why upstream prompt injection detectors collapse under adaptation
Upstream defenses treat the problem as text classification: decide whether a span is instruction or data before the model acts. That can work against fixed benchmarks, but it is structurally weak against an adaptive adversary who can probe the model, vary wording, and search for the seam. Techniques such as delimiter isolation, perplexity filtering, prompt sandwiching, and fine-tuned detectors all share that failure mode because they inspect the same input surface. The issue is not one algorithm. It is the control position. Once the attacker can iterate, the classifier becomes the target, and the target is cheap to optimize against.
Practical implication: Treat upstream filtering as a rate reducer, not a containment control.
Why capability controls survive where text classifiers do not
Capability controls do not ask whether text is malicious. They answer a different question: what is this agent allowed to do in this session, with this credential set, and against these systems. That is why allowlists, credential stripping, and planner-executor separation are more durable than prompt inspection. They constrain the action space even when the model is coerced. The trade-off is scope. A capability policy can prevent data from flowing or credentials from being used, but it does not by itself tell you whether a permitted action was socially engineered, coerced, or semantically wrong. It bounds blast radius, not intent.
Practical implication: Define the smallest executable scope the agent needs and remove everything else.
What action-boundary enforcement adds beyond data-flow control
Action-boundary controls watch execution rather than text. Behavioral baselines, syscall observation, and per-agent tool enforcement can detect when an agent starts acting outside its normal sequence, even if every individual call is authorised. This is the control class that catches coercion after the prompt has already succeeded. It is also why classification and enforcement are not interchangeable. Data-flow control can stop an attacker from moving sensitive content to an unauthorised destination, but it may still allow a wrong action inside an allowed flow. Enforcement at the action boundary closes that gap by comparing execution against observed normal behaviour.
Practical implication: Instrument tool use and runtime behaviour, not only prompt content.
Threat narrative
Attacker objective: The attacker wants the agent to execute an authorised-looking action that advances the attacker’s goal without triggering ordinary access controls.
- Entry occurs through poisoned or adversarial text that the agent is expected to process as part of retrieval, summarisation, or tool-assisted work.
- The injected instruction coerces the agent into authorising actions, using tools, or moving data inside a scope that appears permitted to existing checks.
- Impact follows when the agent discloses data, drafts malicious content, or performs an unintended action with its own credentials and approved tools.
Breaches seen in the wild
- EchoLeak (Microsoft 365 Copilot) 2025: A crafted email could make Microsoft 365 Copilot leak data from its context with no click, a zero-click prompt injection fixed as CVE-2025-32711.
- SalesBleed Salesforce Agentforce 2026: Three fixed Agentforce flaws let poisoned web leads make AI agents leak CRM data with zero clicks and send phishing under the agent's identity.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Upstream prompt inspection is a brittle governance pattern: once the attacker can adapt, text classification becomes the control under test rather than the control in charge. That means teams are validating a detector against an adversary who can change wording until the detector loses. The practitioner conclusion is straightforward: upstream text inspection can reduce noise, but it cannot carry trust decisions for agentic systems.
Capability boundaries are the first durable answer to indirect prompt injection: the control should constrain what an agent may do, not try to infer attacker intent from every span of retrieved text. This is especially relevant where agents bridge human identity, NHI credentials, and external tools, because the risk lives in delegated authority. The practitioner conclusion is to govern the action envelope before trying to classify the prompt.
Action-boundary enforcement is the missing second layer: behavioural monitoring and per-agent policies give security teams something text filters cannot, namely evidence of what the agent actually executed. That is the right place to look for coercion, because the agent can be tricked without any single policy violation. The practitioner conclusion is to measure runtime behaviour against the agent’s own baseline, not against the wording of the input.
Inductive controls fail where authority is implied, not declared: indirect prompt injection works because the agent is already trusted to read, decide, and act inside one workflow. That is a governance issue, not a parsing issue, and it crosses NHI, AI, and IAM design. The practitioner conclusion is to separate who may decide from who may execute, then remove standing authority wherever possible.
What this signals
Adaptive prompt injection changes the security question from detection to containment: once attackers can iterate against a text classifier, the real programme risk is not whether the payload looks suspicious but whether the agent still has enough authority to be coerced. Teams should expect their strongest gains from limiting tool scope, separating decision from execution, and measuring runtime behaviour rather than prompt content.
Agentic AI security needs a control stack, not a single control: upstream filters, capability boundaries, and behavioural enforcement each cover different failure modes, and none is sufficient on its own. For identity and NHI teams, the important shift is to treat agent credentials and tool access as governed runtime authority, not as static configuration.
Prompt classifier confidence is not a security assurance signal: a low false-positive rate on a static benchmark can still translate into a brittle control under adaptive attack. Practitioners should reframe success metrics around coercion resistance, blast-radius reduction, and whether the agent can be stopped at the action boundary.
For practitioners
- Bound agent authority before input inspection Define the smallest action set each agent needs, then remove write access, external communication, or sensitive data access wherever the task does not require it.
- Separate planning from execution Design agent workflows so untrusted text can influence planning but cannot directly modify state, call tools, or reuse high-value credentials.
- Instrument tool use at runtime Collect per-agent tool calls, destinations, and execution traces so behavioural enforcement can compare actual activity with a normal baseline.
- Keep upstream filters as a volume control Use prompt and retrieval filtering to suppress obvious noise, but do not treat classifier output as a security decision for privileged workflows.
- Review standing credentials in agent workflows Replace persistent tokens and broad delegation with scoped, session-bound credentials where the agent can only act inside an explicit boundary.
Key takeaways
- Indirect prompt injection succeeds because defenders often inspect hostile text upstream instead of constraining what the agent is allowed to do.
- Adaptive attacks broke every published upstream defense tested in the article, showing that static evaluation numbers do not describe production risk.
- The practical control shift is toward capability limits, runtime enforcement, and scoped credentials that reduce blast radius when text inspection fails.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST AI RMF and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Indirect prompt injection becomes dangerous when an agent can abuse delegated identity and privilege. |
| ASI09 — Human-Agent Trust Exploitation | The article centres on coercing a trusted agent through manipulated text. | |
| Recommendation — Constrain agent privilege at issuance time and separate decision authority from execution authority. Treat untrusted content as adversarial and prevent it from directly steering privileged actions. | ||
| NIST AI RMF | MANAGE — AI Risk Management | The article is about managing operational risk from agent behaviour and control placement. |
| Recommendation — Build governance that measures agent risk at runtime and reduces blast radius before deployment. | ||
| NIST CSF 2.0 | PR.AA-05 — Access Permissions, Entitlements and Authorizations | The core issue is whether an agent is authorised to act with the privileges it holds. |
| Recommendation — Review and restrict agent entitlements so untrusted input cannot reach sensitive actions. | ||
| MITRE ATT&CK | TA0006;TA0008 — Credential Access; Lateral Movement | The article describes coerced use of credentials and tools to move through authorised paths. |
| Recommendation — Map coerced agent actions to credential access and lateral movement patterns for detection and hunting. | ||
Key terms
- Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
- Capability Boundary: A capability boundary is the control line around what a non-human actor can do through a specific tool, workflow, or service. For AI agents, it is the practical unit of governance because the boundary determines what action the system can trigger, not just what the model can describe.
- Action boundary enforcement: Action boundary enforcement means blocking, masking, or redirecting a harmful action at the moment it is about to occur. For AI agents, this control matters because post-event alerts do not prevent damage and do not meaningfully constrain an agent that can move through systems in a single session.
- Behavior Baseline: A record of normal activity for a non-human identity, including typical consumers, resources, and actions over time. Baselines help security teams detect when an identity is being used in an unusual way and provide the context needed to enforce least privilege safely in dynamic environments.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity. It is designed for practitioners who need to connect delegated authority, identity lifecycle, and operational control.
Published by the NHIMG editorial team on August 18, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org