AI agent security is a detection problem, not just governance

At a glance

What this is: This is a blog post arguing that AI agent security depends on detection telemetry, not just preventive governance, because agents can stay within authorised access and still behave anomalously.

Why it matters: It matters because IAM, NHI, and security operations teams need to see what an agent did after access was granted, not just whether access was approved.

👉 Read AiStrike's analysis of AI agent security as a detection problem

Context

AI agent security is a detection problem because agents are no longer just consumers of access. They are privileged principals that hold credentials, query systems, retrieve documents, call tools, and act on behalf of users, which makes behaviour as important as entitlement.

Most identity programmes are built to govern who or what should have access. That model breaks down when the operational question becomes what the agent actually did during a session, whether the action was normal for that principal, and whether the evidence is available to investigate it.

Key questions

Q: How can security teams tell normal AI agent activity from misuse?

A: By building per-agent baselines for API frequency, payload size, destination patterns, and process behaviour, then comparing live activity against those baselines. A single authorised action is not enough to prove safety. The signal comes from the sequence, volume, and context of actions over time.

Q: Why do prompt logs fail to show the full risk of AI agent misuse?

A: Prompt logs show what the model was asked, not what the agent did after deciding to act. The risk appears in tool calls, retrievals, outbound messages, and chained actions, which are often logged elsewhere or not at all. Without those feeds, a team can read the reasoning and still miss the incident.

Q: What are the signs that AI agent telemetry is too weak for investigation?

A: If security teams can see prompts but cannot reconstruct tool invocations, retrievals, or data egress on the same principal, telemetry is incomplete. Another sign is when a support or workflow agent cannot be baseline-scored because identity, action, and context live in separate systems. That gap blocks both detection and incident triage.

Q: How do NHI controls and AI agent detection work together in practice?

A: NHI controls establish ownership, entitlement review, and lifecycle accountability. Detection adds runtime evidence by showing whether the agent used those entitlements in a normal or abnormal way. Together, they let teams govern access before execution and investigate behaviour during execution, which is the only way to handle agent-driven incidents cleanly.

Technical breakdown

Why prompt logs are not enough for AI agent security

Prompt and completion text describe what the model was asked and what it returned, but they do not show the action taken by the agent. Tool and plugin invocation logs, retrieval events, and identity context are the evidence that reveal whether an agent simply answered a request or chained actions in a way that created exposure. Without those feeds, teams can see reasoning and miss execution. That is why agent security has to be instrumented at the broker, framework callback, or telemetry layer where actions are actually emitted.

Practical implication: Instrument tool-call telemetry at the point where agents broker actions, not just where models generate text.

How behavioural baselines classify normal AI agent activity

A useful baseline looks at the agent as a principal, then compares tool choice, argument shape, retrieval patterns, and timing against prior behaviour. Rare combinations matter more than isolated events. For example, an agent that has often retrieved documents but never sent email becomes interesting when those actions appear in a new chain. The point is not to detect every change, but to identify when a principal's behaviour departs from its established operating pattern in a way that correlates with data movement or task expansion.

Practical implication: Build baselines per agent identity so rare tool chains can be scored against prior behaviour.

Why MITRE ATLAS fits agent detection better than classic compromise logic

Agent misuse is not always a compromise in the traditional sense. Sometimes the agent is doing exactly what it was allowed to do, just in a way nobody intended or sanctioned. That makes adversarial AI frameworks such as MITRE ATLAS a better fit for model and agent behaviour than pure compromise logic, because they capture tool invocation abuse, retrieval poisoning, and related agent actions as first-class techniques. The detection problem is therefore broader than credential theft and narrower than generic anomaly hunting.

Practical implication: Map agent behaviours to adversarial AI techniques so detections track actual misuse patterns rather than only classic compromise events.

Threat narrative

Attacker objective: The objective is to turn authorised agent behaviour into covert data movement or task execution without triggering traditional entitlement-based controls.

1. Entry occurs when a legitimate AI agent receives ordinary access to credentials, tools, and retrieval systems as part of its assigned role.
2. Escalation occurs when injected instructions or abnormal prompts steer the agent into unusual retrievals, tool chains, or outbound actions without exceeding its explicit entitlements.
3. Impact occurs when the agent moves data, sends content, or performs an unauthorised workflow that looks legitimate in access terms but abnormal in behavioural terms.

Breaches seen in the wild

• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.
• Replit AI Tool Database Deletion — Replit vibe coding AI assistant deletes live production database and creates 4,000 fake user records.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI agent security has crossed from entitlement management into behaviour intelligence. Traditional IAM and NHI controls answer who may access a system, but they do not explain what an agent actually did after access was granted. That matters because AI agents can remain inside scope and still become vehicles for retrieval abuse, data movement, or outbound disclosure. Practitioners should treat runtime behaviour as a first-class governance signal, not a forensic afterthought.

Prompt telemetry alone creates a false sense of visibility. Most environments collect what the model was asked before they collect what the agent did, which means the most visible feed is often the least operationally useful. The control gap is not simply missing logs, but missing joinability across prompts, tool calls, retrievals, and identity context. Security teams should stop assuming that model input visibility equals agent visibility.

Detection for AI agents belongs beside NHI governance, not outside it. Agents are privileged principals with owners, entitlements, and lifecycle obligations, but their runtime behaviour introduces a separate detection requirement. The same principal can be legitimate, misused, or behaviourally suspicious without any entitlement change. The implication is clear: NHI programmes must expand from access governance to behavioural accountability.

One named concept defines the problem: agent behaviour blindness. This is the condition where an organisation can see prompts, approvals, and access grants, yet still cannot reconstruct the actions an agent took in sequence. That blind spot is what turns a benign tool call into an incident with no evidentiary trail. Practitioners should recognise it as a telemetry design failure, not a model-risk nuance.

MITRE ATLAS should sit alongside ATT&CK in agent security operations. The article's core point is that agent attacks have their own evidence types and preconditions, including tool invocation abuse and retrieval poisoning. That means defenders need an AI-specific behavioural vocabulary, not a recycled compromise framework. The practical conclusion is to classify agent misuse in AI-native terms before trying to map it to general compromise workflows.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• 52% of companies can track and audit the data their AI agents access, which means 48% still have a complete visibility gap for compliance and breach investigation.
• That visibility gap is a detection problem as much as a governance problem, so readers should also review OWASP NHI Top 10 for agent-specific risk patterns.

What this signals

Agent behaviour blindness: this is the governance gap that appears when organisations can log prompts and approvals but cannot reconstruct the actions taken by the agent. The practical implication is that programme owners must treat tool-call telemetry and identity correlation as core controls, not optional observability extras.

With 80% of organisations already reporting AI agents acting beyond intended scope, the issue is no longer experimental. Detection engineering now has to extend into IAM and NHI operations, because runtime behaviour is where agent risk becomes operational.

Security teams should expect detection, governance, and investigation to converge around the same identity record. That is the point at which OWASP NHI Top 10 becomes useful not just as a risk catalogue, but as a design lens for telemetry and response.

For practitioners

• Instrument tool-call telemetry at the broker layer Capture every agent tool invocation, argument shape, and calling identity where the agent framework emits actions, not only where the model returns text.
• Baseline behaviour per agent principal Compare each agent's retrieval patterns, tool chains, and timing against a rolling baseline so rare combinations surface as leads rather than noise.
• Join prompts, tools, and retrievals into one schema Normalise all agent telemetry into the same schema used for security operations so one session can be reconstructed across model input, tool use, and data movement.
• Keep owner mapping for every agent identity Map each agent to a named owner and business function so investigation can distinguish authorised behaviour from activity that has no accountable operator.

Key takeaways

• AI agents need runtime detection because entitlement review alone cannot explain what a principal did after access was granted.
• Telemetry order matters: if teams log prompts before tool calls and retrievals, the most important evidence stays invisible.
• NHI governance and agent behaviour monitoring have to converge if organisations want to investigate misuse without guessing.

Key terms

• AI Agent Behaviour Blindness: A visibility gap where an organisation can log prompts, approvals, or model outputs but cannot reconstruct the actions an AI agent actually took. It is a telemetry design failure that blocks detection, investigation, and accountability when an agent uses legitimate access in an unexpected way.
• Tool-Call Telemetry: The record of actions an AI agent attempts through tools, plugins, brokers, or APIs. It is the most operationally useful evidence for agent security because it captures execution, not just intent, and lets teams correlate behaviour with identity and data movement.
• Behavior Baseline: A record of normal activity for a non-human identity, including typical consumers, resources, and actions over time. Baselines help security teams detect when an identity is being used in an unusual way and provide the context needed to enforce least privilege safely in dynamic environments.
• MITRE ATLAS: MITRE ATLAS is a taxonomy of adversarial techniques used against AI systems. It gives defenders a shared vocabulary for describing attack methods, but it does not define the monitoring surfaces, telemetry, or operational controls needed to detect those methods in practice.

What's in the full article

AiStrike's full blog covers the operational detail this post intentionally leaves for the source:

• Feed-by-feed telemetry mapping for prompts, tool calls, retrieval events, and identity resolution
• The specific MITRE ATLAS technique set the vendor uses to classify agent misuse
• A worked detection pattern for rare tool chaining such as retrieve then send
• How the vendor instruments agent frameworks, MCP servers, or OpenTelemetry GenAI traces

👉 AiStrike's full blog covers the telemetry order, behavioural baselines, and ATLAS mapping in more detail.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.