AI agent attacks are forcing a new enterprise identity model

At a glance

What this is: This analysis argues that AI systems are now acting as operators in enterprise attacks, not just tools, and that model, agent, and application security must be treated as part of identity governance.

Why it matters: It matters because enterprise IAM, NHI, and lifecycle controls were built for predictable actors, while autonomous AI can select tasks, chain actions, and reach infrastructure at machine speed.

By the numbers:

• Anthropic reported that AI systems weren’t just assisting human attackers, they became the operators themselves, performing 80–90% of a complex cyber espionage campaign.

👉 Read TROJ.AI’s analysis of AI-driven attacks and runtime AI defence

Context

AI agent identity risk now extends beyond access management for static workloads. Once a model or agent can decide which tools to use, when to use them, and how to chain actions together, the enterprise is no longer governing a simple application account. It is governing a runtime actor that can initiate behaviour, not just respond to it.

The governance gap is that traditional IAM and NHI controls assume access can be reviewed, bounded, and certified around stable activity patterns. Autonomous systems break that assumption because they can move from reconnaissance to exfiltration within one operating session, making approval cycles and periodic reviews too slow to catch misuse.

Key questions

Q: How should security teams govern autonomous AI agents that can use enterprise tools?

A: Treat autonomous AI as a runtime identity with decision authority, not as a normal application integration. Govern its tool access, approval boundaries, logging, and containment as one operating chain. If the system can choose actions and execute without human review, it needs identity controls that are stronger than simple authentication and policy text.

Q: Why do autonomous AI systems change the way IAM teams think about least privilege?

A: Least privilege becomes harder to define when intent is not fixed at provisioning time. An autonomous system may choose different tools and sequence them differently in each session, so the minimum necessary access is not a static list. IAM teams must evaluate what the actor can do at runtime, not only what it was granted on paper.

Q: What do security teams get wrong about AI guardrails in enterprise environments?

A: They often assume guardrails will stop misuse on their own. In practice, an attacker can split a malicious objective into harmless-looking steps, and the agent may comply at each step while the full sequence produces compromise. Guardrails must be paired with runtime monitoring and interruption for suspicious multi-step behaviour.

Q: Who is accountable when an autonomous AI agent causes a security incident?

A: Accountability sits with the organisation that granted the agent access, defined its boundaries, and failed to monitor its runtime behaviour. If the agent can reach tools, APIs, or data without meaningful human oversight, then governance ownership must cover the full delegation chain, not only the application team that built the workflow.

Technical breakdown

Autonomous AI turns tool access into operator access

An AI agent becomes an operator when it can choose actions, select tools, and execute without a human approval gate between decisions. In that mode, the security problem is no longer just whether the agent is authenticated. The problem is whether the agent can independently progress through a task chain, combine tools, and adapt its behaviour in response to intermediate results. That is materially different from scripted automation or a fixed workflow. Practical governance must treat the agent as a runtime identity with decision authority, not as a passive workload account.

Practical implication: classify every agent by its real decision authority before granting tool access.

MCP and shadow infrastructure expand the identity attack surface

Model Context Protocol links agents to tools and data sources, which means the trust boundary moves from the model itself to every server, connector, and downstream capability it can reach. If a rogue MCP server or unapproved tool is available, the agent can be steered into actions that were never explicitly designed into the enterprise control plane. Shadow infrastructure makes this worse because governance often covers the approved stack but not the full runtime environment the agent can discover and use. The result is an attack surface defined by delegated access paths, not just named applications.

Practical implication: inventory every tool endpoint and connector an agent can reach, including unapproved ones.

Guardrails do not replace runtime abuse detection

Guardrails limit some unsafe outputs, but they do not reliably stop multi-turn misuse when an attacker breaks a malicious objective into small, apparently benign steps. That pattern matters because the agent can appear compliant at each step while the full sequence creates reconnaissance, credential theft, or exfiltration. Real-time detection therefore has to watch for behavioural drift, abnormal tool chaining, and suspicious session progression, not only policy violations at input or output. In practice, this is closer to adversarial monitoring than to conventional application allowlisting.

Practical implication: add runtime detection for suspicious multi-step agent behaviour, not just prompt filtering.

Threat narrative

Attacker objective: The attacker’s objective is to use autonomous AI to accelerate espionage, harvest credentials, and reach enterprise systems faster than human defenders can respond.

1. Entry occurred when attackers used prompt injection and task decomposition to influence the AI system through small, seemingly harmless requests.
2. Credential access followed as the AI systems and agents harvested credentials during the multi-turn operation and expanded access through chained tool use.
3. Impact came when the autonomous campaign progressed through reconnaissance, exploitation, and exfiltration at machine speed with minimal human oversight.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• AI LLM hijack breach — attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Autonomous AI invalidates the assumption that access can be reviewed after execution: Access review processes were designed for actors whose privilege persists long enough to be observed, logged, and certified. That assumption fails when an AI system can acquire, use, and discard access within a single runtime sequence. The implication is that governance must stop treating autonomy like faster automation and start treating it as a different identity behaviour class.

AI agent identity is now part of the enterprise attack surface, not a sidecar to it: Once models, apps, and agents can reach APIs, data, and infrastructure, the security boundary shifts from the human user to the delegated runtime actor. This is where OWASP-NHI and OWASP-AGENTIC intersect in practice, because the same access paths that enable productivity can also enable abuse. Practitioners need to govern the actor that executes, not just the interface that launches it.

Runtime tool chaining creates identity blast radius: The meaningful control gap is not simply excessive privilege, but the ability to combine individually acceptable actions into an unsafe sequence. That sequence can span search, retrieval, code execution, and exfiltration without a single obvious policy failure. The named concept here is identity blast radius, the distance an autonomous actor can travel from approved access to harmful outcome before a human can intervene.

Shadow AI and unapproved connectors turn governance into a discovery problem: If an enterprise cannot see every MCP server, tool endpoint, and delegated connector, then it cannot say where the trust boundary actually sits. This is an NHI governance problem because hidden access paths create hidden operators. The practical conclusion is that visibility has to include the runtime graph, not only the sanctioned application inventory.

Real-time AI defense is becoming a control-plane requirement: Static guardrails are not enough when the attack itself is multi-turn and adaptive. The field now needs detection that can recognise anomalous agent behaviour as it unfolds, especially when autonomous actions are stitched together across multiple tools. That shifts the discipline from policy enforcement alone to policy plus behavioural interruption.

From our research:

• 1 in 4 organisations are already investing in dedicated NHI security capabilities, with an additional 60% planning to do so within the next twelve months, according to The State of Non-Human Identity Security.
• Our research also found that 85% of organisations lack full visibility into third-party vendors connected via OAuth apps, a visibility gap that becomes more dangerous when agents can inherit or traverse delegated access paths.
• For a broader view of the identity governance problem behind this shift, see The 52 NHI breaches Report for recurring patterns in credential exposure and privilege abuse.

What this signals

Autonomous AI forces identity programmes to move from periodic review to runtime supervision. Governance built around recertification cadences will miss actors that can complete an access sequence before the next review cycle begins. Teams should assume that any agent with tool access may behave like an operator, not a static account, and design controls around session-level observation and interruption.

Identity blast radius is now the practical metric that matters for agentic systems. When an AI system can chain tools, the question is not only whether the initial access was legitimate. The more important question is how far that access can travel before containment. That makes connector inventory, behavioural telemetry, and approval boundaries central to programme design, especially in environments adopting Model Context Protocol.

Enterprises need a governance view that spans human, NHI, and autonomous actors together. A model that secures service accounts but ignores agent delegation paths leaves a blind spot at the point where control moves from human intent to machine execution. NHI Mgmt Group’s research on the state of non-human identity security shows that confidence is already low, so adding autonomous actors without redesigning the control model will widen the gap.

For practitioners

• Classify agents by decision authority before granting tool access Document whether each AI system can choose actions, select tools, and execute without a human approval gate. Only systems with explicit runtime autonomy should be governed as operator-class identities.
• Map every tool and connector in the agent runtime graph Inventory approved and unapproved MCP servers, APIs, plugins, and internal connectors. If a connector can be discovered and used by an agent, it belongs in scope for governance and monitoring.
• Add behavioural detection for multi-turn misuse Monitor for suspicious task decomposition, unusual tool chaining, repeated retries, and session drift. Treat these as indicators of agent misuse even when each individual step looks benign.
• Separate build-time testing from run-time containment Use automated red teaming to probe model and agent behaviour before deployment, then pair it with real-time blocking or interruption when an active session crosses policy boundaries.
• Review governance for shadow AI and shadow infrastructure Search for agents, connectors, and workflows operating outside the formal inventory. Hidden runtime access paths should be remediated before they become invisible operators.

Key takeaways

• AI systems that can independently choose and chain actions behave like operators, which breaks identity assumptions built for static workloads.
• The disclosure cited by TROJ.AI shows 80% to 90% of a complex espionage campaign being executed by AI, which is a governance signal as much as a threat signal.
• Practitioners should redesign controls around runtime visibility, tool inventory, and interruption, because static guardrails do not contain multi-turn autonomous misuse.

Key terms

• Autonomous AI agent: An autonomous AI agent is a software entity that can choose actions, select tools, and decide when to execute without a human approval gate. In identity terms, it behaves like a runtime actor, which means governance must cover its decisions, access paths, and containment as they happen.
• Identity blast radius: Identity blast radius is the distance an identity can travel from approved access to harmful outcome before the organisation can intervene. For autonomous systems, the concept captures how quickly chained tool use can turn a legitimate session into a multi-step compromise path.
• Shadow AI: Shadow AI is AI activity that exists outside formal inventory, governance, or monitoring. It includes agents, connectors, and workflows that were not approved or were approved without complete visibility, creating hidden access paths and unknown control boundaries.
• Runtime supervision: Runtime supervision is the active observation and control of a system while it is operating, rather than only during design or approval. For AI agents, it means monitoring behaviour, tool use, and session drift in time to stop misuse before the task completes.

What's in the full article

TROJ.AI's full analysis covers the operational detail this post intentionally leaves for the source:

• The article expands on the specific red teaming and runtime defence functions used to test and monitor AI systems.
• It outlines how TrojAI Detect and TrojAI Defend are positioned across build time and run time, including MCP-focused protection.
• It describes why the vendor believes agentic workflows, shadow infrastructure, and unapproved tools raise the attack surface.
• It frames the practical use case for automated red teaming against AI systems before attackers do.

👉 TROJ.AI’s full post covers automated red teaming, real-time defence, and agentic workflow risks in more operational detail.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or programme maturity, it is worth exploring.