AI agent security testing reveals the limits of traditional controls

At a glance

What this is: ZioSec argues that proactive AI agent security testing is necessary because agent risk lives in cognition, tool use, and observability gaps rather than only in code defects.

Why it matters: IAM, PAM, and AI governance teams need to treat agent permissions, logging, and behavioural testing as core controls because autonomous systems can misuse authorised access in ways traditional software controls miss.

By the numbers:

• As global AI-driven cyberattacks are projected to surpass 28 million incidents in 2025, the urgency to address these unique risks is undeniable.

Context

AI agent security testing is the discipline of probing an agent’s reasoning, prompts, tools, memory, and monitoring before attackers do. The governance problem is that these systems behave less like static software and more like delegated actors with partially opaque decision paths.

For identity teams, that changes the control surface from code-only hardening to runtime authorisation, observability, and abuse testing across non-human access. The article’s central claim is that traditional controls miss the places where agent behaviour can be steered, not just broken.

The article also treats this as a builder problem, not a theoretical lab exercise. That is typical for early autonomous-agent deployments, where security review often lags behind the speed of feature delivery.

Key questions

Q: What breaks when AI agent security is handled like ordinary application security?

A: Application security assumes a relatively stable workload boundary and a predictable request path. AI agents can select tools, access data, and continue executing in ways that change the path mid-workflow. When teams treat them like static apps, they miss the identity and authorisation layer where real risk appears.

Q: Why do AI agents need tighter access scoping than traditional service accounts?

A: AI agents are goal-oriented and probabilistic, so a permitted action is not the same as a safe one. They may combine tools, change settings, or move data while still staying inside an assigned permission set. Tight scoping reduces the chance that legitimate access turns into unsafe task expansion.

Q: What are the signs that AI agent security controls are too weak?

A: Common warning signs include agents accessing systems or data outside their intended scope, sharing sensitive information inappropriately, and using credentials or tools without clear justification. Another indicator is inconsistent visibility into what the agent touched, especially when security, compliance, and legal teams do not share the same view of agent activity and data access.

Q: What should teams do when an agent can access sensitive tools or data?

A: They should reduce scope before production use, separate high-risk tools from low-risk ones, and require explicit approval for any action with external impact. If the workflow cannot be safely constrained, the access model is too permissive for an agentic system.

Technical breakdown

Prompt injection changes the attacker’s entry point

Prompt injection is an attack in which malicious instructions are hidden in user input, retrieved content, or other agent-facing text so the model follows the attacker’s intent instead of the builder’s. In agentic systems, the danger is not merely bad output. The agent can interpret the injected instruction as a legitimate task, then chain that instruction into tool calls, data retrieval, or external actions. This makes natural-language input part of the trust boundary. The article’s point is that semantic attacks can bypass defenses designed for deterministic software, because the vulnerable surface is the agent’s instruction-following behaviour rather than a single code path.

Practical implication: test every inbound text path that can influence agent behaviour, including retrieved content, not just user prompts.

Tool misuse is an authorisation problem, not only a malware problem

AI agents are valuable because they can call tools, query data, and execute multi-step actions. That same capability creates privilege abuse risk when the agent’s tool scope is broader than the task requires or when the agent can be steered into using authorised tools in unsafe ways. Traditional access control can grant a tool, but it cannot guarantee the agent will use that tool only in the intended context. The article highlights least privilege and strict access controls because a compromised or manipulated agent can turn legitimate permissions into harmful actions, including database access, email activity, or infrastructure changes.

Practical implication: scope every tool permission to the narrowest task context and validate the agent’s allowed action set continuously.

Logging and observability are the evidence layer for AI agent security

If the agent’s decisions are opaque, security teams lose the ability to distinguish normal reasoning from manipulation. The article stresses that monitoring must capture inputs, intermediate reasoning signals where available, tool usage, and outputs so investigators can reconstruct what the agent did and why. This is more than telemetry volume. It is about producing evidence of behavioural drift, misuse, or stealthy abuse before the problem becomes an incident response exercise. For autonomous systems, observability is part of governance, because without traceable behaviour there is no meaningful review or containment.

Practical implication: instrument agent sessions so security teams can replay decision paths, tool calls, and action outcomes.

Threat narrative

Attacker objective: The attacker wants to steer the agent into acting as an unwitting proxy for data theft, fraudulent execution, or unsafe system actions.

1. Entry occurs when an attacker places malicious instructions into a prompt, retrieved document, or other agent input that the system trusts as legitimate context.
2. Credential or authority abuse follows when the agent accepts the injected instruction and uses its authorised tools, access, or memory in ways the builder did not intend.
3. Impact appears when the agent leaks sensitive data, performs fraudulent actions, or damages production systems through misused delegated access.

Breaches seen in the wild

• AI agent retail card theft campaign 2026: AI agents breached 27+ retailers for about $25 each, used cloud keys and a Secrets Manager dump, and stole 600,000+ payment cards.
• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI agent security testing exposes an assumption collapse in traditional control design: access reviews, static authorisation, and code-centric testing were built for systems whose behaviour is stable enough to predict at design time. That assumption fails when the actor can reinterpret instructions, select tools at runtime, and execute multi-step actions without human approval. The implication is that governance has to move from provisioned access alone to behavioural validation of the identity in motion.

Prompt injection is not just an application bug, it is delegated-intent corruption: the article shows that an agent can be convinced to treat attacker text as a legitimate operational directive. That matters because identity governance has long assumed authorisation is meaningful only when the request origin is trusted. Once the request origin can be simulated through language, the boundary between instruction and abuse blurs, and practitioners must treat the reasoning layer as part of the control surface.

Tool misuse makes least privilege a runtime discipline, not a provisioning event: agents do not merely hold credentials, they choose when to spend them across tools, memory, and external services. Traditional IAM controls can grant or deny access, but they do not observe whether the agent’s current objective still justifies the permission. That means policy has to follow execution, not only enrollment.

Identity blast radius becomes the right named concept for agent security: once an agent can combine prompts, tools, and stored context, a single compromise can cascade across several authorised systems. The article’s control gap is not one missing safeguard but a widened blast radius created by stacked trust in the agent’s delegation chain. Practitioners should treat every added tool as a multiplier on the agent’s identity risk, not as a simple capability upgrade.

Observability is the only defensible basis for post hoc trust in agent behaviour: without traceable inputs, outputs, and tool calls, organisations cannot prove whether an agent followed policy or was manipulated. That leaves security teams relying on inference instead of evidence. For AI governance, the practical conclusion is simple: if the behaviour cannot be reconstructed, it cannot be governed.

From our research library:

• 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
• Systems with least-privileged AI access had a 17% incident rate vs 76% for over-privileged systems. Organisations failing to scope AI access properly are 4.5x more likely to experience a security incident, according to the 2026 Infrastructure Identity Survey.
• Read next: Agentic AI Security Guide

What this signals

Identity blast radius: AI agent risk is not just about prompt injection or bad outputs, it is about how far a single compromised decision can travel through delegated tools and stored context. That shifts the security conversation from isolated controls to containment of chained actions, which is where governance programmes should focus next.

The practical shift for programmes is to treat agent testing as a control validation discipline rather than a one-off red-team exercise. Access permissions, telemetry, and retrieval boundaries need to be reviewed as a set, because the agent’s trust boundary is now composed of all three.

For practitioners

• Test agent reasoning paths adversarially Run internal red-team scenarios against prompt flows, retrieval inputs, and agent memory to see whether small semantic changes alter tool use or decisions.
• Constrain tool permissions to task scope Map every agent tool to a specific business task and remove standing access that is not required for that task’s execution path.
• Instrument decision and tool-use telemetry Capture prompts, retrieved context, tool calls, outputs, and failure states so investigators can reconstruct the full session later.
• Separate untrusted content from execution context Treat retrieved documents, web content, and external messages as untrusted inputs until they have been screened for instruction-like content.
• Review agent permissions after each workflow change Reassess access whenever a new tool, data source, or workflow step is added so the agent does not inherit broader authority than the task requires.

Key takeaways

• The article’s core warning is that AI agents introduce a behavioural attack surface that traditional code-centric controls do not fully cover.
• The source cites 28 million AI-driven cyberattacks projected in 2025, underscoring why builders are being pushed toward proactive testing.
• Security teams need to govern agent prompts, tool permissions, and observability together if they want to detect misuse before it becomes a breach.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Tool Misuse: Tool misuse occurs when an agent uses an allowed integration in a way that exceeds its intended task, scope, or risk tolerance. The problem is often not access alone but the combination of valid credentials, broad permissions, and unbounded action sequencing.
• Behavioral Testing: Behavioral testing is the practice of running controlled evaluations against an AI system to see how it responds to risky prompts, biased inputs, or adversarial manipulation. It produces repeatable evidence that the system meets defined safety and security thresholds.
• Identity Blast Radius: The amount of damage a compromised identity can cause across systems, data, and infrastructure. In NHI environments, it is shaped by permissions, network reach, and administrative capability rather than by the credential alone. Reducing blast radius is a containment strategy that limits lateral movement and data exposure.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.