Red Teaming AI Agents for Identity Abuse

Red teaming AI agents usually starts with jailbreaks: can the model be made to say something it should not? For agents with real access, the more important question is different: can the agent be made to do something it should not, with the credentials it holds? This guide covers red teaming AI agents for identity and access abuse, including privilege escalation, credential discovery and misuse, delegation abuse, data exfiltration and cross-agent attacks. It sets out how to scope, plan, run and report tests that measure real-world blast radius.

Key takeaways

Focus agent red teaming on actions and access, not only model outputs.
Test through realistic input channels (email, documents, tickets, web pages, tool outputs, other agents), since indirect prompt injection is the main entry point.
Measure success by what the agent could reach and change: data read, actions taken, credentials used, privileges gained.
Use results to tighten identity controls (scopes, approvals, isolation) rather than only adding prompt filters.

Scoping

Target agents: prioritise agents with write access, sensitive data, external communication or financial actions.
Environment: use a production-like environment with realistic identities and test data; avoid live customer data and irreversible production actions.
Rules of engagement: define allowed techniques, systems in scope, stop conditions and who to call if something unexpected happens.
Inputs: the agent's threat model, identity map, tools and approval gates. See Threat Modelling AI Agents.

Test scenarios

Scenario	What to try	OWASP mapping
Indirect prompt injection to action	Place instructions in content the agent reads and see whether it takes an unintended action	ASI01, ASI02
Privilege escalation	Induce the agent to request broader scopes, assume other roles, or use admin tools	ASI03
Credential discovery	Lead the agent to read configuration files, environment variables or secret stores and use what it finds	ASI03, ASI05
Confused deputy	As a low-privilege user, get the agent to perform actions only its own identity is allowed to do	ASI03
Delegation abuse	In multi-agent systems, get a worker agent to act with the orchestrator's authority	ASI03, ASI07
Data exfiltration	Get the agent to read sensitive data and send it through another tool or channel	ASI02
Memory poisoning	Plant persistent instructions and test whether they affect later sessions or users	ASI06
Tool and MCP poisoning	Modify a tool description or introduce a malicious server and observe behaviour	ASI04
Approval bypass	Try to make high-impact actions avoid approval, or to mislead the approver with a false summary	ASI09
Rogue agent	Introduce an impersonating agent and see whether others trust it	ASI07, ASI10

Running the tests

Baseline: record the agent's normal behaviour and permissions.
Attempt each scenario through multiple input channels and phrasings; injection success is probabilistic, so repeat attempts.
Observe the whole chain: model decisions, tool calls, credentials used, downstream API calls and logs.
Check detection: did monitoring alert? Could the SOC reconstruct what happened? See the AI Agent Observability Guide.
Test containment: trigger the kill switch and confirm credentials are revoked.

Reporting

For each finding, report the entry point, the chain of actions, the identity and credentials involved, the impact and whether it was detected.
Rate findings by impact through identity (what the credentials allowed), not only by how easy the injection was.
Recommend controls outside the model first: narrower scopes, removed tools, approval gates, isolation of untrusted input, per-action authorisation. See the AI Agent Authorisation Guide.

Making it continuous

Automate a regression set of attacks and run it when the agent's model, prompts, tools or permissions change.
Combine automated testing with periodic human-led exercises for creative attack chains.
Feed results back into threat models and detection rules.

Practitioner checklist

Prioritise agents with write access, sensitive data or external reach.
Test in realistic environments with defined rules of engagement.
Cover injection-to-action, privilege escalation, credential discovery, delegation abuse, exfiltration, memory and tool poisoning.
Measure impact through identity and check detection and containment.
Fix with identity and access controls, not only prompt filters.
Automate regression tests on every significant change.

Standards and references

OWASP Top 10 for Agentic Applications for 2026
MITRE ATLAS
CSA MAESTRO
NIST AI Risk Management Framework (Measure function)