AI agent attack detection needs a surface-based framework

At a glance

What this is: This framework explains how to detect AI agent attacks by instrumenting four attack surfaces and a five-layer operating stack.

Why it matters: It matters because IAM, security operations, and platform teams need visibility into agent prompts, tool use, identity, and delegation before agentic workloads scale beyond human-paced controls.

By the numbers:

• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes and as quickly as 9 minutes in some cases.

👉 Read ARMO's analysis of AI agent attack detection and runtime surfaces

Context

AI agent attack detection is the discipline of spotting malicious behaviour across an agent’s prompts, tools, identity, and coordination paths rather than at the container or host layer. The problem is that existing security stacks were designed for workloads that execute predictable requests, not systems that can select tools and chain actions across sessions.

That mismatch leaves security teams with telemetry that is useful for infrastructure events but weak for agent-specific abuse. A detection model for AI agents has to start with the surfaces where malicious intent enters the agent, not with the controls that already exist in CNAPP, EDR, or SIEM pipelines.

Key questions

Q: How should security teams detect AI agent attacks without relying only on container telemetry?

A: They should instrument the agent’s own decision surfaces first. That means capturing prompt context, tool calls, identity events, and orchestration state, then correlating them into one attack story. Container logs alone show what the workload touched, but not whether the agent was manipulated through data, scope expansion, or delegation abuse.

Q: Why do existing IAM controls struggle with autonomous AI agents?

A: Existing IAM controls were designed around human users and predictable workload behaviour. Autonomous agents can make repeated tool calls, chain permissions, and keep acting after the original task context changes. That creates lifecycle, privilege, and accountability gaps that traditional role models do not close on their own.

Q: What breaks when AI agent tool use is monitored only at the infrastructure layer?

A: The infrastructure layer misses the decision that led to the tool call and the context that shaped it. That creates alerts without causation, which makes both triage and containment weaker. Security teams need framework-level tool telemetry and provenance-aware input logging to preserve the attack chain.

Q: How do security teams respond when an AI agent needs to be contained quickly?

A: Containment should include pausing the agent and removing the access it can exercise, not just turning off execution. If permissions remain active, the investigation is still exposed to the same risk. A practical response plan needs quarantine, deprovisioning, and an investigation path that preserves evidence.

Technical breakdown

Input and reasoning signals in AI agents

The input and reasoning surface includes prompts, retrieved context, memory, and tool output that the agent may treat as instruction. Attacks here are often indirect because the payload is not a malicious binary or a noisy network event, but a piece of data that becomes instruction after the model interprets it. That is why WAF-style controls miss much of the risk. The right telemetry is provenance-aware capture of what entered the context window, where it came from, and whether it was marked as data or instruction.

Practical implication: preserve full prompt and retrieval provenance so security teams can distinguish poisoned context from normal agent input.

Tool invocation, scope, and behavioural envelopes

Tool invocation becomes the detection hinge once an agent starts acting. The meaningful signal is not a single API call, but the sequence, rate, and parameter shape of calls relative to the agent’s normal behavioural envelope. Tool misuse and scope expansion usually look authorised in isolation, which is why per-agent baselines matter more than generic allowlists. Framework callbacks, MCP logs, and process-tree telemetry can show which tools were called and what happened next.

Practical implication: baseline tool-call patterns per agent so deviations in sequence and rate are visible before they become abuse.

Identity, action, and cross-agent coordination

Identity and action is where declared access meets observed access. Agents often have a permission set on paper, but the security issue is the gap between that set and the permissions they actually exercise. Cross-agent coordination adds a further layer because delegation graphs can hide malicious instructions moving from one agent to another through ordinary orchestration. IAM events, Kubernetes audit logs, and orchestrator telemetry need to be joined into one causal chain for the story to be visible.

Practical implication: correlate identity and delegation telemetry across agents so a single compromised actor does not hide inside normal orchestration.

Threat narrative

Attacker objective: The attacker wants to turn a legitimate AI agent into a covert execution path that can misuse tools, access data, or propagate malicious instructions across other agents.

1. Entry occurs when malicious content enters the agent’s decision context through prompts, retrieved data, or poisoned memory.
2. Escalation occurs when the agent makes tool calls or permissioned actions that expand the attacker’s reach while still appearing legitimate to infrastructure controls.
3. Impact occurs when chained tool use, identity abuse, or cross-agent delegation produces exfiltration, escape, or downstream compromise.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agent attack detection fails when teams treat AI agents like containers with better logs. AI agents operate on a decision surface, not just a runtime surface, so container telemetry alone cannot capture the sequence of prompts, tool calls, identity use, and delegation that define the attack. That makes detection architecture a governance question as much as an engineering one. Security teams need to think in surfaces, not in tool silos, if they want coverage that survives new agent behaviours.

Tool misuse is the named control gap that most teams miss first. The article makes clear that individual tool calls are usually authorised, while the abuse lives in the combination, timing, and scope of those calls. That means standard identity review logic can approve each step and still miss the attack. Practitioners should treat behavioural envelopes as the unit of detection, not isolated events.

Cross-agent delegation creates a hidden attack path that per-agent baselines cannot see. Once one agent can pass context or instructions to another, the security problem stops being a single identity and becomes a graph. This is where orchestration telemetry and shared-context audit trails become essential evidence. The implication is that agent governance must extend beyond the actor to the delegation chain that carries its instructions.

Runtime telemetry is now a prerequisite for AI agent governance, not an optimisation. The framework shows that telemetry, baselines, correlation, triage, and response are sequential dependencies. If the first layer does not preserve prompt, tool, and identity evidence, every higher layer becomes guesswork. The practitioner conclusion is simple: no source signals, no agent detection capability.

Identity blast radius: the relevant concept is how far a legitimate agent credential can carry malicious intent before detection catches up. That blast radius expands when permissions, context, and delegation are all visible to the agent at runtime. For security teams, the task is to reduce the amount of trusted execution an agent can accumulate before the chain becomes observable.

From our research:

• 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time, according to Ultimate Guide to NHIs.
• Only 5.7% of organisations have full visibility into their service accounts, which shows how often identity inventories lag operational reality.
• For a broader breach pattern view, the 52 NHI Breaches Analysis shows how exposure, standing access, and weak lifecycle governance combine in practice.

What this signals

Agent attack detection now sits at the intersection of IAM, observability, and runtime control. Teams that only extend existing SIEM rules will keep missing the decision layer where the abuse actually happens. The more reliable path is to define what normal looks like for each agent, then correlate prompt, tool, identity, and orchestration signals into one operating picture.

Behavioural baselines will matter more than event matching as agent adoption grows. The relevant question is not whether a tool call happened, but whether the sequence and scope fit the agent’s intended role. That shifts programme design away from static rule tuning and toward runtime governance that can absorb legitimate non-determinism without giving up on containment.

Runtime identity evidence becomes the control plane for agent governance. Without that evidence, teams cannot tell whether a malicious action came from poisoned input, excessive scope, or cross-agent delegation. For practitioners, the next step is to align detection architecture with the actual surfaces of agent behaviour rather than the abstractions inherited from container security.

For practitioners

• Instrument the four agent attack surfaces Capture prompt context, tool calls, identity events, and orchestration state as separate telemetry streams so the attack chain can be reconstructed later.
• Baseline each agent at the deployment level Build behavioural envelopes from two to four weeks of representative activity so legitimate variation does not collapse the signal into noise.
• Join identity and delegation logs Correlate Kubernetes audit data, cloud IAM events, and orchestrator telemetry to expose when a normal delegation pattern becomes a malicious path.
• Keep response tied to the surface of abuse Use tool-scope revocation for tool misuse, permission revocation for identity abuse, and context isolation for poisoned retrieval paths.
• Treat managed agent opacity as a control limit If a managed runtime does not expose tool-call or orchestration telemetry, restrict it to lower-blast-radius tasks until those signals are available.

Key takeaways

• AI agent attacks are surface-driven, so container-centric detection leaves the most important signals out of view.
• Sequence, scope, and delegation are the three signals that turn legitimate tool use into a detectable attack pattern.
• Security teams need runtime telemetry, per-agent baselines, and correlation across identity and orchestration if they want meaningful agent governance.

Key terms

• Agent Decision Plane: The layer where an AI agent chooses which tool to call, in what order, and for what purpose. It sits above cloud APIs, processes, and audit logs, which means conventional security tools may see the resulting action without seeing the decision that caused it.
• Behavioural Envelope: A behavioural envelope is the observed range of normal activity for a specific workload or agent. It includes tools used, destinations reached, volume moved, and processes spawned, giving detection a reference point for spotting deviations that configuration-only checks would miss.
• Cross-Agent Coordination: Cross-agent coordination is the exchange of instructions, context, or work between multiple AI agents through an orchestrator or shared state. It becomes a security concern when a malicious instruction moves from one agent to another through a path that appears legitimate on both sides.

What's in the full article

ARMO's full blog covers the operational detail this post intentionally leaves for the source:

• Concrete telemetry examples for prompt wrappers, framework SDK callbacks, and eBPF collection.
• The five-layer operating stack that turns surface signals into triage and response.
• How the 2x2 maturity grid maps surface coverage against stack depth in production.
• Examples of detection logic for managed runtimes where agent telemetry is partially opaque.

👉 ARMO's full post covers the four-surface framework, layered detection model, and production maturity grid.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.