Notifications
Clear all
Topic starter
19/08/2026 3:41 pm
TL;DR: AI agent red teaming must test what agents do across tools and connected systems, not just what they say, because roleplay-based prompt injection succeeds nearly 90% of the time and chained actions can create real incidents, according to Fiddler. The governance gap is trust debt: autonomy grows faster than oversight, so runtime guardrails and continuous monitoring have to replace one-time validation.
NHIMG editorial — based on content published by Fiddler: AI Agent Red Teaming: Techniques and Attack Surfaces
By the numbers:
- Roleplay-based prompt injection attacks achieved an 89.6% attack success rate, outperforming logic traps at 81.4% and encoding tricks at 76.2%.
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, sharing sensitive data, or revealing credentials.
Questions worth separating out
Q: How should security teams red team AI agents that use tools and memory?
A: Security teams should test AI agents through the same interface and runtime path production uses, then validate the tools, memory stores, and downstream sinks those agents can reach.
Q: What breaks when AI agents are not governed at runtime?
A: Without runtime governance, an agent can shift behaviour after provisioning and still execute actions that were never reviewed in context.
Q: How do security teams know if AI red teaming is working?
A: AI red teaming is working when testing finds real prompt injection paths, over-scoped integrations, and policy gaps before attackers do, and when fixes are re-tested successfully after model or workflow changes.
Practitioner guidance
- Map agent authority before testing begins Document every action an agent can take, every tool it can call, and every data source it can reach.
- Prioritise roleplay and indirect injection scenarios Build red team cases that use persona shifts, hidden instructions in retrieved content, and misleading context instead of simple keyword abuse.
- Enforce runtime policy checks at the point of action Block or approve tool invocations before execution, not after the fact.
What's in the full article
Fiddler's full blog post covers the operational detail this post intentionally leaves for the source:
- Step-by-step red team structure for macro-level and micro-level agent testing
- Attack scenario examples for prompt injection, tool abuse, and context poisoning
- Guidance on converting findings into runtime policy enforcement and continuous monitoring
- Practical advice on scheduling recurring exercises as agent capabilities change
👉 Read Fiddler's analysis of AI agent red teaming techniques and attack surfaces →
AI agent red teaming and trust debt: are controls keeping up?
Explore further
19/08/2026 4:20 pm
AI agent red teaming is now an identity governance problem as much as a model testing problem. Once an agent can access data, call tools, and chain decisions, the central question becomes who or what is authorised to act. That is an NHI governance issue because the agent operates as a software entity with delegated access that must be bounded, reviewed, and revoked like any other machine identity. Practitioners should stop treating agent testing as a niche AI exercise and fold it into IAM and PAM controls.
A question worth separating out:
Q: When should organisations treat an AI agent as a privileged system?
A: Organisations should treat an AI agent as privileged whenever it can reach production data, administrative tools, or sensitive workflows without direct human approval for each step. At that point, the agent is no longer a passive automation helper. It becomes a governed identity whose permissions, logs, and exceptions need the same scrutiny as other high-risk access.
👉 Read our full editorial: AI agent red teaming exposes the trust debt in autonomous systems
