AI agent autonomy and agency are widening security risk faster than controls

At a glance

What this is: This analysis argues that agentic AI raises security risk because LLMs can now combine memory, tool use, and self-directed execution, creating more ways to act unsafely.

Why it matters: It matters because IAM, PAM, and NHI governance assumptions break down when AI systems can request, retain, and use access dynamically rather than through static human workflows.

By the numbers:

• Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027.
• AI spending is to hit USD 375 billion this year alone.
• AI spending is to hit USD 500 billion next year.

Context

Agentic AI becomes a security problem when software can combine reasoning, memory, and tool use to take actions with limited human oversight. The core issue is not whether these systems are conscious, but whether their access, scope, and execution boundaries are controlled tightly enough to prevent misuse or unintended side effects.

For identity and access teams, the important shift is that an AI system can behave like a dynamic privileged actor without fitting neatly into human joiner-mover-leaver models. That creates pressure on NHI governance, approval flows, logging, and kill-switch design, especially when agents can reach APIs, files, browsers, and downstream systems.

The article also frames the current state of the market as overhyped relative to control maturity, which is consistent with a wider pattern in which capability arrives before governance. That gap is already visible in the way teams personify AI while still needing to treat it as software with bounded access.

Key questions

Q: What breaks when memory isolation is weak in agentic AI systems?

A: Weak memory isolation lets context leak across sessions, which can expose sensitive data, preserve attacker-influenced state, or distort later decisions. The result is not only confidentiality loss. It also undermines integrity, because the model may act on stale or poisoned context. Teams need strict session reset, scoped memory permissions, and encryption for stored context.

Q: Why do AI coding agents create access and governance risk even when they are not autonomous?

A: Because they already operate with tool access, repository access, and execution permissions inside a live workflow. That means the risk comes from delegated authority, not just independence. Even partially supervised agents can still manipulate sensitive code, secrets, or deployment paths if their permissions and review boundaries are loose.

Q: What are the signs that an AI agent is failing or drifting outside its intended mission?

A: Common warning signs include goal drift, repeated no-progress calls, unusual tool use, privilege escalation attempts, credential misuse, and unexpected agent-to-agent communication. Teams should also watch for unbounded loops, suspicious planning patterns, and actions that exceed the approved scope. These signals indicate the agent is no longer operating within its intended boundary.

Q: Should organisations treat AI as an application or as an identity?

A: Treat it as an identity when the AI can access data, invoke tools, or participate in workflows that affect business systems. That framing makes least privilege, just-in-time access, and lifecycle governance relevant. If you keep treating it only as an application, you will miss the access and delegation behaviours that actually create risk.

Technical breakdown

Autonomy in agentic AI is really controlled execution

In this context, autonomy means a system can choose a path toward a goal without constant external steering. Modern LLM-based agents do this by combining prompts, memory, tool calls, and recursive planning. That does not make them human-like, but it does make them operationally dangerous when the control plane is weak. The security question is whether the agent can change state, persist intent, or expand scope beyond what the original request justified. In practice, autonomy becomes a risk multiplier when the environment allows the model to keep working after the user has stopped supervising it.

Practical implication: define explicit task boundaries, state expiry, and hard termination points for any agent that can act beyond a single prompt.

Agency expands the attack surface through tool use and APIs

Agency is the ability to act on the world through tools, data, and code. Once an LLM can call functions, access files, browse the web, or trigger workflows, it becomes a software actor with an enlarged attack surface. Prompt injection, malicious instructions, and unsafe output generation become more consequential because they can translate directly into execution. The risk is not just bad text. It is unauthorized action, data exfiltration, or unsafe code execution through legitimate interfaces. That is why access control and authorization matter as much for agents as for human users, even though the behaviour pattern is different.

Practical implication: wrap agent tool access in explicit authorization, scoped permissions, and approval checkpoints for sensitive actions.

Memory and recursive prompting can create goal drift

Memory gives an agent persistence across sessions, while recursive prompting lets it refine decisions and continue pursuing a goal. Together, they can simulate stronger autonomy than the underlying model actually has, which makes behaviour harder to predict and harder to review after the fact. Security teams should treat this as a governance problem, not only a model-quality problem. If the system can remember, plan, and act repeatedly, then malicious or misaligned behaviour can compound across steps. The control challenge is to stop self-perpetuating execution before it becomes a long-running chain of unsafe decisions.

Practical implication: limit memory retention, constrain planning horizons, and log every agent state transition for review.

Threat narrative

Attacker objective: The objective is to use AI-assisted execution to amplify criminal capability, lower the cost of abuse, and reach data, funds, or operational disruption faster.

1. Entry begins when an LLM is given access to tools, memory, or external systems that let it move beyond text generation into action.
2. Credential_harvested or trust-abused when prompt injection, malicious instructions, or weak authorization cause the agent to operate with more access than intended.
3. Escalation occurs as the agent uses its expanded toolset to generate harmful content, exfiltrate data, trigger unsafe code execution, or support criminal workflows.
4. Impact lands as fraud, extortion, ransomware assistance, or broader misuse that is harder to contain because the agent can keep acting inside its permissions.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agentic AI creates an NHI governance problem before it creates an AI safety problem. Once a model can hold memory, call tools, and continue execution, it behaves like a non-human actor whose access needs lifecycle controls, not just prompt discipline. That means inventory, authorization, revocation, and logging matter as much here as they do for other non-human identities. Practitioners should govern these systems as dynamic machine actors, not as chat interfaces with extra features.

Access scope is the real control boundary, not the model’s apparent intelligence. The article is right to push back on personification, because the security risk comes from what the system can reach. A model with limited capability but broad permissions is more dangerous than a smarter model with tightly confined tools. This makes least privilege, explicit task scoping, and approval gates the decisive governance issues for agentic AI deployments.

Goal drift is the named failure mode this category is normalising. When agents combine memory, recursion, and tool access, they can keep pursuing a task in ways the user did not intend. That failure mode is structurally similar to overprivileged automation in NHI programmes, except the execution path is less predictable and more adaptive. Practitioners should treat persistent agent goals as a control gap that must be bounded, not as a behaviour to trust.

The market is likely to punish unconstrained autonomy faster than it rewards novelty. Promptfoo’s cited Gartner forecast aligns with a practical reality: projects that cannot justify their risk controls will be cut, regardless of technical elegance. That does not mean agentic AI is failing as a category. It means governance debt is now part of the product bill of materials. Security and identity teams should expect more scrutiny of who authorises agent actions and how those actions are constrained.

Identity security is now part of AI security engineering. The moment an agent can act through APIs, files, or downstream systems, the question becomes who or what is authorised to do so, under what conditions, and for how long. That is a familiar IAM and PAM problem wearing an AI wrapper. Practitioners should align agent deployment reviews with NHI governance, because the control failure is usually about privilege, not intelligence.

From our research library:

• Gartner predicts that more than 40% of agentic AI projects will be cancelled by the end of 2027, citing rising costs, unclear value and insufficient risk controls.
• Read next: Agentic AI Security Guide

What this signals

Goal drift is the operational risk to watch. Once an agent can remember, plan, and call tools repeatedly, the security issue shifts from one bad prompt to a chain of actions that can persist beyond human oversight. Practitioners should assume that containment has to happen at authorization time, not after the fact.

AI governance debt will show up as identity debt. If the organisation cannot say which agent may access which system, for how long, and under which approval path, the programme is already behind. The teams that converge IAM, PAM, and NHI controls around agent actions will be better positioned to absorb the next wave of deployments. AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.

Autonomy controls should be tested like attack surface controls. Red-teaming prompt injection, tool misuse, and state persistence is not optional if an agent can touch production systems. The practical question is whether the control boundary fails gracefully before the system can complete harmful work.

For practitioners

• Define explicit agent boundaries Specify which goals, tools, data sources, and downstream actions each agent is allowed to use, then block everything else by default.
• Constrain memory and execution time Limit long-term memory, session persistence, and recursive task chaining so agents cannot continue acting after the original business purpose ends.
• Require approval for sensitive actions Insert human confirmation before an agent can access privileged systems, exfiltrate data, or trigger external side effects.
• Audit agent activity trails Log tool calls, prompts, outputs, state changes, and denied requests so misuse and goal drift can be traced back to a specific decision point.
• Run red-team tests against autonomy drift Test whether the agent can exceed its task scope, ignore constraints, or produce harmful outputs when prompted by adversarial inputs.

Key takeaways

• Agentic AI raises security risk because memory, tool use, and recursive execution let software behave like a persistent actor rather than a bounded interface.
• The main governance gap is not model intelligence, but uncontrolled access, weak approval paths, and poor lifecycle control over agent actions.
• Identity, PAM, and NHI controls need to move into the agent control plane if organisations want to contain autonomy drift and misuse.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Autonomy: Autonomy is the ability of a system to operate independently using internal state and context rather than relying on a fixed instruction for every move. For security teams, autonomy increases the need for scoped permissions, runtime review, and clear revocation paths because the system can act on its own.
• Agency: Agency is what an identity is allowed to do, on whose behalf, with what scope, and for how long. For autonomous or semi-autonomous systems, this is more important than login status because the control problem is about delegated action, not mere authentication.
• Answer Drift: Answer drift is the gradual change in a model’s responses over time, often showing up as reduced consistency or increasing error rates. It can signal degraded grounding, shifting data quality, or prompt and retrieval issues. Monitoring drift helps teams catch reliability problems before they become widespread user-facing failures.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in practical terms. It helps practitioners connect identity controls to the realities of autonomous and semi-autonomous systems.