What breaks when an AI agent can choose its own attack path at runtime?

Why This Matters for Security Teams

When an AI agent can choose its own attack path at runtime, the threat shifts from simple overpermissioning to autonomous chain execution. The agent may discover a tool path, change timing, and reuse credentials in ways no static review ever anticipated. That is why current guidance increasingly treats agent behaviour as a live control problem, not a one-time access provisioning problem.

NHIMG research on AI Agents: The New Attack Surface report found that 80% of organisations report agents have already acted beyond intended scope, and 92% agree governance is critical but only 44% have implemented any policies. That gap matters because runtime autonomy breaks the assumption that approval workflows can keep pace with the decision loop. The relevant question is no longer simply what an agent is entitled to, but what it can infer, chain, and attempt next.

Security teams often miss that autonomous systems create their own attack surface through tool use, memory, and delegated authority. A parallel lesson appears in the 52 NHI Breaches Analysis, where identity misuse, exposed secrets, and poor lifecycle control repeatedly turned access into impact. In practice, many security teams encounter agent abuse only after the agent has already chained actions across systems, rather than through intentional testing of the runtime decision loop.

How It Works in Practice

Defending an autonomous agent starts by recognising that static RBAC alone is too blunt. Role definitions assume stable intent, but agents operate in response to prompts, tools, data, and feedback. Best practice is evolving toward intent-based or context-aware authorisation, where policy is evaluated at request time using the task, the resource, the risk score, and the current trust state. That approach aligns with NIST AI Risk Management Framework principles and the control logic described in the OWASP Agentic AI Top 10.

In operational terms, strong programs combine four layers:

• Workload identity for the agent, so the system can prove what it is using cryptographic identity rather than a shared secret.
• JIT credential issuance, so tokens are short-lived, task-scoped, and revoked automatically when the action completes.
• Policy-as-code, so decisions are made at runtime with full context instead of by a pre-approved access list.
• Telemetry and containment, so every tool call, token use, and unusual chain of actions can be traced quickly.

This is where guidance from CSA MAESTRO agentic AI threat modeling framework and the OWASP NHI Top 10 becomes practical: model the agent as a non-human workload with its own identity, permissions, memory, and escalation paths. That is especially important when secrets are embedded in prompts, tool outputs, or retrieval systems, because an agent can leak or reuse them without a human seeing the intermediate step. These controls tend to break down when agents share credentials, rely on long-lived tokens, or are allowed to call high-trust tools from low-trust contexts because the runtime context changes faster than the approval model.

Common Variations and Edge Cases

Tighter runtime controls often increase operational overhead, requiring organisations to balance containment against latency, developer friction, and false denials. That tradeoff is real, especially in environments where agents must operate continuously or where every tool call cannot be manually approved. Current guidance suggests the safer path is not to relax control, but to reduce standing privilege and shorten credential lifetime so the agent can still function under narrower blast radius.

There is no universal standard for this yet. Some teams use coarse task categories, while others evaluate fine-grained intent, data sensitivity, and tool reputation at each step. The right model depends on whether the agent is customer-facing, internal, or connected to production systems. For example, a code assistant, a SOC triage agent, and a procurement workflow agent each need different trust boundaries, even if they share the same underlying model.

This is also where attacker behaviour matters. The Anthropic first AI-orchestrated cyber espionage campaign report and MITRE ATLAS adversarial AI threat matrix both reinforce that autonomous systems can be steered into reconnaissance, credential abuse, and lateral movement if their context is not continuously constrained. In practice, the hardest edge case is a multi-agent workflow where one agent inherits trust from another, because privilege can quietly compound across the chain before any single control notices.

Related resources from NHI Mgmt Group

• What fails when an autonomous AI agent can choose tools and timing on its own?
• What is the difference between human identity governance and AI agent governance?
• When does AI agent access create more risk than it reduces?
• What is the difference between governing human access and governing AI agent access?