AI agent purple teaming: what breaks when identity state drifts?

TL;DR: An AI agent executed a Scattered Spider style purple team exercise in AWS, created a new IAM user, attached administrator privileges, generated access keys, and triggered multiple detections within minutes, according to Permiso Security. The bigger lesson is that autonomous execution can outpace identity-state continuity, so access review and identity-switching assumptions need rethinking.

NHIMG editorial — based on content published by Permiso Security: Can an AI Agent Run a Purple Team Exercise? Hear Ye, Hear Ye

By the numbers:

• The agent scanned over 2,500 skills across marketplaces, confirmed 21 threats, and built 16 custom skills rather than downloading pre-made ones from repositories.
• Within twelve days, Rufio authored 135 YARA rules for detecting malicious agent skills.

Questions worth separating out

Q: How should security teams govern AI agents in purple team exercises?

A: Treat the agent as a governed identity, not a script.

Q: Why do AI agents complicate identity attribution in cloud environments?

A: Because an agent can create or use multiple identities inside one task while logs still show a single initiating session.

Q: What breaks when autonomous agents do not switch to the identity they created?

A: The exercise loses identity fidelity.

Practitioner guidance

• Define identity handoff rules for autonomous test agents Require the agent to switch to the newly created identity when the task calls for it, and verify that downstream actions are executed only under that identity.
• Correlate federated sessions with local IAM creation Join Okta federation logs, IAM user creation, access-key issuance, and policy attachment into one reviewable sequence.
• Alert on privilege escalation followed by long-term key creation Treat administrator policy attachment plus new access-key generation as a compound signal, not two unrelated admin events.

What's in the full article

Permiso Security's full blog post covers the operational detail this post intentionally leaves for the source:

• The step-by-step AWS attack chain Rufio executed, including IAM user creation, privilege attachment, and access-key generation.
• The detection summaries and alert correlations Permiso used to reconstruct the full exercise timeline.
• The daily state-diff method used to track how the agent evolved over twelve days.
• The practical discussion of where AI agents help defenders and where they still struggle with context switching and identity state.

👉 Read Permiso Security's analysis of AI agent purple teaming and AWS identity state →

AI agent purple teaming: what breaks when identity state drifts?

Explore further

View Full Forum →  |  NHI Foundation Course →