What breaks when organisations rely only on prompt filtering to secure AI agents?

Why This Matters for Security Teams

prompt filtering is useful, but it only addresses the visible text layer. AI agents are not passive chatbots; they can act, call tools, read files, and chain steps across systems. That means the real risk often appears after the prompt is accepted, when a malicious instruction is carried through a tool, browser, repository, or API. Guidance from the OWASP Agentic AI Top 10 and NHI research such as OWASP NHI Top 10 both point to the same problem: text moderation cannot contain runtime authority. Security teams that stop at prompt inspection may miss credential misuse, unauthorized tool calls, or lateral movement through connected systems.

NHIMG research on AI Agents: The New Attack Surface report found that 33% of organisations report AI agents accessed sensitive data beyond intended scope, and 80% report actions beyond intended scope. In practice, many security teams encounter abuse only after an agent has already exfiltrated data or changed a downstream system, rather than through intentional monitoring of agent behaviour.

How It Works in Practice

Prompt filtering reduces obvious jailbreaks, profanity, and direct policy violations, but it does not provide a security boundary for an autonomous workload. Once an agent has legitimate execution authority, the attacker no longer needs to win the prompt. They only need to influence the agent’s context so it performs a harmful action through approved tools, services, or workflows.

The stronger pattern is runtime control. Current guidance suggests combining workload identity, short-lived credentials, and policy checks at the moment of action. That means the agent should prove what it is with cryptographic identity, such as SPIFFE-style workload identity or scoped OIDC tokens, then receive only the minimum permission needed for the current task. This is why NHI guidance in the Ultimate Guide to NHIs and agentic security analysis like Analysis of Claude Code Security focus on authorization and secret handling, not just content filtering.

• Use intent-based authorization so the agent is approved for a specific action, not a broad role.
• Issue just-in-time credentials with short TTLs, then revoke them when the task ends.
• Evaluate policy at request time with context such as tool, target system, data sensitivity, and user approval.
• Log agent actions, tool calls, and data access so defenders can reconstruct runtime behaviour.

Standards work reinforces this direction. The NIST AI Risk Management Framework and the CSA MAESTRO agentic AI threat modeling framework both treat system behaviour, not just input text, as the security problem. These controls tend to break down when agents are given broad, persistent tokens and unrestricted tool access because the prompt layer no longer matches the authority layer.

Common Variations and Edge Cases

Tighter prompt controls often increase operational friction, requiring organisations to balance false positives against the need for fast agent execution. That tradeoff becomes sharper in environments where agents must work across email, SaaS apps, internal code repositories, and production APIs.

There is no universal standard for this yet, but current guidance suggests three common edge cases. First, if the agent is embedded inside a browser or desktop workflow, prompt filtering can be bypassed by hidden page content, file payloads, or indirect instructions. Second, if the agent can call other agents or tools, the attack surface expands into multi-step chains where the harmful step appears far removed from the original prompt. Third, if static API keys or long-lived service accounts are reused, filtering offers almost no protection because the compromise happens at the authority layer, not the text layer. Relevant threat patterns are also documented in NHIMG coverage such as CoPhish OAuth Token Theft via Copilot Studio and the Amazon Q AI Coding Agent Compromised case.

For that reason, prompt filtering should be treated as one layer of hygiene, not a primary control. It is most useful when paired with least privilege, strong session isolation, tool allowlists, and real-time policy enforcement. Without those controls, the defender sees only the words, while the attacker abuses the agent’s permissions.

Related resources from NHI Mgmt Group

• What breaks when organisations rely only on provisioning records for AI agents?
• What breaks when organisations rely on output filtering for AI governance?
• What breaks when organisations rely on access control alone for MCP-connected AI agents?
• What breaks when organisations rely only on per-harness monitoring for AI agents?