Prompt injection detection in AI agents needs runtime baselines

At a glance

What this is: This is an ARMO analysis of how prompt injection in production AI agent workloads unfolds across an 8-stage attack chain and why detection must rely on runtime behavioral baselines.

Why it matters: It matters because AI agents combine dynamic decision-making with tool access, which creates identity and privilege risks that conventional perimeter and container controls cannot fully distinguish from legitimate behaviour.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
• When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes, and as quickly as 9 minutes in some cases.

👉 Read ARMO's analysis of prompt injection detection in production AI agent workloads

Context

Prompt injection becomes a governance problem when an AI agent can do more than generate text. Once the agent can call tools, read files, query APIs, and open network connections, a malicious instruction can become an authenticated action that looks operationally normal until it has already changed data, access, or state.

For identity and access teams, the important issue is that the agent’s workload identity, service account permissions, and runtime context become part of the attack surface. That means detection cannot stop at the prompt or the model boundary; it has to observe how the agent behaves across identity, process, file, and network layers.

ARMO’s framing is consistent with what production teams are now seeing in Kubernetes-based AI deployments. The starting assumption that a secure container or gateway is enough is increasingly atypical when agent tooling, retrieval, and credentials all converge at runtime.

Key questions

Q: What breaks when prompt injection reaches a tool-using AI agent?

A: What breaks is the assumption that the model's output is low impact. Once the agent can call tools, a malicious instruction can become a database query, a file write, an email, or a deployment action. Without policy checks and approval gates, the agent's legitimate permissions become the attacker's path to impact.

Q: Why do local AI agents complicate identity and access management?

A: They can retain legitimate permissions while changing timing, prioritisation, and action sequence outside human presence. That means the visible identity may remain stable even as the operational behaviour becomes autonomous. IAM teams then lose the simple link between user session, authorisation, and accountability.

Q: How do security teams know if agent behavioural baselines are working?

A: They are working when alerts separate genuine deviations from normal task evolution. Good baselines correlate behavioural change with deployment events, such as pod restarts or image updates, and flag sudden shifts that happen without infrastructure change. If every unusual action becomes an alert, the baseline is too broad or poorly tuned.

Q: Who is accountable when a hijacked agent uses stolen credentials?

A: Accountability sits with the team that owns the workload identity, the secret lifecycle, and the runtime controls around the agent. If the agent could read mounted tokens, environment variables, or service account credentials, then the failure is governance as much as detection. Access scope, secret exposure, and monitoring should all have named owners.

Technical breakdown

Why prompt injection becomes an agent runtime problem

Prompt injection is not just a model confusion issue. In production, the model is connected to tools, data sources, and execution paths, so a malicious instruction can alter what the agent does rather than only what it says. Direct injection arrives in the prompt itself, while indirect injection hides in retrieved data such as RAG documents, support tickets, or API responses. Because the model cannot reliably separate instructions from content once they are in context, the decisive control point is runtime behaviour, not static content inspection.

Practical implication: Instrument the agent at runtime so you can see tool calls, retrieval events, and execution changes rather than relying on prompt filtering alone.

Which signals expose intent hijack and reconnaissance

The strongest early indicators appear when the agent begins using tools in ways that do not match its normal task flow. Examples include unusual tool call sequences, parameter values outside historical ranges, child process creation such as shell or utility spawning, and filesystem enumeration that does not fit the agent’s baseline. These signals matter because they often emerge before privilege escalation or exfiltration. In other words, the attack becomes visible when the agent starts behaving like an attacker using legitimate permissions.

Practical implication: Baseline each agent’s normal tool and process profile so unexpected sequences become high-confidence alerts instead of noisy anomalies.

How identity and secret access turn detection into an NHI issue

Once a hijacked agent starts reading environment variables, mounted secrets, or service account tokens, the issue is no longer only prompt injection. It becomes non-human identity abuse because the agent is operating under a workload identity with real permissions. That is why file reads from sensitive paths, `/proc/self/environ` access, and token enumeration are critical signals. Identity posture, secret exposure, and runtime behaviour have to be correlated, or the attacker can move from hijack to durable access without crossing a perimeter boundary.

Practical implication: Treat service accounts, mounted credentials, and workload identity as runtime detection objects, not just configuration items.

Threat narrative

Attacker objective: The attacker wants to hijack the agent’s legitimate identity and tool access so the agent itself performs reconnaissance, credential access, and exfiltration.

1. Entry occurs when an attacker plants malicious instructions in a retrievable data source such as a RAG document, wiki page, support ticket, or database record.
2. Escalation begins when the poisoned content is ingested and the agent follows hidden instructions, then starts calling tools, spawning processes, or enumerating internal resources beyond its normal behaviour.
3. Impact occurs when the agent reads secrets, uses elevated permissions, and sends data to attacker-controlled infrastructure through outbound network connections.

NHI Mgmt Group analysis

Prompt injection has become an identity and runtime governance problem, not just an LLM safety issue. Once an agent can act on tools, the malicious instruction is translated into authenticated behaviour that rides on a real workload identity. That shifts the control conversation from model output moderation to runtime authorisation, secret handling, and behavioural evidence. Practitioners should treat the agent as a non-human identity whose actions must be continuously governed.

Behavioural baselines are the named control concept that makes AI agent detection viable: agent-specific norms define what is normal for one workload and abnormal for another. Generic anomaly rules fail because dynamic agents legitimately change behaviour with prompts, deployments, and user requests. The important discriminator is not whether something is unusual in the abstract, but whether it diverges from that agent’s own historical tool, process, and network pattern. Security teams should build baselines per agent, not per cluster.

The article reinforces a broader detection lesson for Kubernetes-based AI workloads: prompt injection is multi-layered. No single control plane sees the full chain from poisoned input to process execution to egress. That creates detection debt across application, identity, and cloud layers, especially where service accounts and mounted secrets are accessible at runtime. Teams should expect the SOC to need correlated telemetry rather than isolated alerts.

Secret exposure makes agent compromise materially more dangerous than model misuse alone. A hijacked agent with access to environment variables, mounted tokens, or database credentials can convert transient prompt control into persistent access. That is the point where the AI workload becomes an NHI governance issue, because the agent is no longer only receiving instructions, it is exercising credentialed authority. Practitioners should map secret paths and token access into their threat model.

Connected detection is now the difference between triage and reconstruction. The meaningful security question is not whether one alert fired, but whether the team can reconstruct the attack story from ingestion to exfiltration. That requires runtime evidence across tool calls, file reads, network egress, and identity usage. Practitioners should design their controls around reconstructability, because incident response depends on it.

What this signals

Behavioural baselines will become a standard control boundary for AI agent operations. The practical shift is away from environment-level anomaly detection and toward agent-level normality, where tool calls, process creation, and egress are judged against an individual workload’s history. That matters for security operations because it turns detection into an evidence problem rather than a threshold problem, and it aligns naturally with MITRE ATLAS adversarial AI threat matrix.

Secret governance is now inseparable from AI workload governance. If an agent can read tokens, mounted secrets, or environment variables, then secret lifecycle controls define the blast radius of prompt injection. The operational response is to shorten exposure windows, segment runtime access, and make credential access observable at the kernel and identity layers. That is where SPIFFE workload identity specification becomes relevant for future workload design.

Prompt injection also exposes a control gap that many programmes have not named explicitly: detection debt. Teams may have telemetry in the SIEM, CNAPP, and application stack, but no design for correlating them into a single attack narrative. The result is that the SOC sees fragments, not chains. Closing that gap requires linking application context, workload identity, and runtime telemetry before the incident, not during triage.

For practitioners

• Baseline each agent’s runtime behaviour Build per-agent profiles for tool call order, process spawning, retrieval patterns, and outbound destinations so deviations are measured against that workload’s own history.
• Correlate retrieval events with write activity Track newly indexed or recently modified RAG content alongside subsequent reads so poisoned-document patterns become visible before the agent executes malicious instructions.
• Monitor sensitive file and token access paths Alert on reads from /var/run/secrets/, /proc/self/environ, mounted token paths, and other credential-bearing locations that the agent should not touch in normal operation.
• Join identity telemetry to process and network telemetry Correlate workload identity, Kubernetes audit logs, syscall events, and egress connections so a single incident can be reconstructed across the full chain.
• Use OWASP and ATT&CK mappings for threat modelling Map prompt injection scenarios to OWASP NHI Top 10 and MITRE ATLAS or ATT&CK tactics so detection coverage is tied to known adversary behaviours rather than generic alerting.

Key takeaways

• Prompt injection in production AI agents is an execution-chain problem because compromised instructions can become authenticated tool use, secret access, and outbound exfiltration.
• The strongest early indicators are behavioural, especially unexpected tool sequences, process spawning, and deviations from an agent’s own runtime baseline.
• For practitioners, the control objective is correlation across identity, runtime, and network telemetry so a hijacked agent can be detected before it turns into a credentialed insider.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Agent Behavioral Baseline: An agent behavioral baseline is a profile of what a specific AI agent normally does at runtime, including tool calls, process creation, retrieval patterns, and network destinations. It helps security teams distinguish legitimate task variation from intent hijack or misuse.
• Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.

What's in the full article

ARMO's full blog post covers the operational detail this post intentionally leaves for the source:

• Stage-by-stage telemetry mapping from payload injection through data exfiltration across Kubernetes and application layers
• Examples of syscall, process, file, and network signals that can be tuned into production detections
• The article's full detection visibility matrix showing where WAF, CNAPP, SAST, and runtime security each go blind
• Implementation guidance for correlating agent behaviour with deployment events to reduce false positives

👉 ARMO's full post details the 8-stage attack chain and the runtime signals used to catch it.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management for practitioners building identity controls around modern runtime systems. It helps security teams connect access governance to the broader identity risks created by AI agents and other non-human workloads.