AI agent security risks demand observability beyond prompt injection

At a glance

What this is: Fiddler argues that AI agents create a broader security problem than standard LLM apps because autonomy, memory, and tool access turn text manipulation into real-world actions.

Why it matters: This matters to IAM practitioners because agent permissions, service-level credentials, and delegated tool access now need governance patterns that cover NHI, autonomous workflows, and human oversight together.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.

👉 Read Fiddler's analysis of AI agent security risks and observability controls

Context

AI agent security risks are not the same as traditional LLM risks because agents can act, retain state, and delegate work across tools and sub-agents. That changes the control problem from output moderation to governance of what the system can access, execute, and remember.

The identity angle is central here because many agents operate with service-level credentials and inherited permissions. Once an agent can invoke tools or write to memory, IAM and NHI governance have to cover delegated authority, not just authentication at login.

Key questions

Q: How should security teams enforce least privilege for AI agent identities?

A: Start by treating every agent as an NHI with a dedicated identity, a tight permission boundary, and a named owner. Then enforce least privilege in policy, not just in dashboards, so unused access is removed and rare elevation is granted only through JIT workflows. The goal is to shrink blast radius before autonomy creates incident scale.

Q: Why do AI agents create more identity risk than traditional LLM applications?

A: AI agents create more identity risk because they can persist state, choose tools, and carry out actions over time. Traditional LLM applications usually produce outputs inside a single request-response cycle, but agents can act across workflows and reuse access. That makes privilege, traceability, and impersonation much harder to govern.

Q: What do security teams get wrong about memory in agent systems?

A: Security teams often treat memory as if a vector store were enough. In practice, memory includes working state, durable state, and policy-relevant context, each with different access, retention, and audit needs. If those are collapsed into one mechanism, the system becomes hard to govern, hard to troubleshoot, and hard to prove after the fact.

Q: How do security teams know if agent observability is actually working?

A: Observability is working only when teams can tie together token activity, tool calls, and latency for a specific agent session. If those signals are disconnected, the logs may show traffic but not governance. The test is whether investigators can reconstruct the path of a meaningful agent action from start to finish.

Technical breakdown

Why agent tool access turns prompt injection into real action

A standard LLM prompt injection mainly corrupts text output. In an agent, the same malicious instruction can survive into the execution layer and trigger API calls, file operations, email sending, or data export. The core issue is not whether the input looks clean, but whether the agent is trusted to act on it. Once tool access and instruction interpretation are coupled, the attacker’s payload becomes operational rather than merely conversational. Least-privilege tooling reduces that blast radius by limiting the agent’s callable surface to the smallest set needed for the task.

Practical implication: constrain each agent’s tools and scopes so injected instructions cannot translate into broad system actions.

Persistent memory creates a cross-session attack surface

Agent memory changes security from session-scoped risk to stateful risk. If an attacker poisons long-term memory or the retrieval corpus, the payload can survive resets and reappear when a future trigger condition matches. This is especially difficult in RAG-style workflows, where poisoned documents can be retrieved repeatedly and reinsert the malicious context into otherwise valid sessions. The challenge is that the malicious content does not need to dominate the current prompt; it only needs to be present in a memory write or retrieval path that later influences the agent’s next decision.

Practical implication: monitor memory writes and retrieval outputs as security events, not just application data flows.

Sequence-level observability is the missing control in multi-agent systems

Multi-agent architectures create trust chains where one agent’s output becomes another agent’s input. A compromised upstream step may look harmless in isolation, but the full chain can still produce data theft or policy violation. Static guardrails miss this because each intermediate action can appear legitimate. Effective defence needs telemetry across every step, tool call, and handoff so teams can detect drift, unexpected privilege use, and output divergence before the sequence completes. In agentic systems, the attack often emerges only when the whole decision path is visible.

Practical implication: instrument every agent handoff with continuous tracing and anomaly detection, not periodic review.

Threat narrative

Attacker objective: The attacker wants to convert the agent’s authority into unauthorized actions that move data, reveal credentials, or corrupt downstream workflows.

1. Entry begins when an attacker places malicious instructions in content, memory inputs, or other data sources the agent will process.
2. Escalation occurs when the agent interprets that content as a legitimate task and uses its own service-level permissions to call tools or access connected systems.
3. Impact follows when the agent performs unauthorized actions such as data exfiltration, credential exposure, or downstream propagation through other agents.

NHI Mgmt Group analysis

Autonomy changes the security unit of analysis from prompt to execution path. Traditional LLM controls focus on input and output boundaries, but agents create chained decisions, memory persistence, and tool execution. That means the real risk surface is the execution path itself, where one injected instruction can become a sequence of authorised-looking actions. Practitioners should treat agent workflows as governed systems, not chat interfaces.

Agent permissions are now NHI governance problems in disguise. When an agent runs under service credentials or delegated access, its identity behaves like a non-human identity with real operational authority. That makes least privilege, scope limitation, and lifecycle controls central to agent security, especially when autonomous systems can access production data or downstream services. IAM teams should explicitly classify agent identities alongside other NHI estates.

Memory poisoning is a durable failure mode, not a one-off injection event. The article correctly distinguishes transient prompt attacks from persistent state corruption. Once malicious content is stored in long-term memory or retrieval systems, the enterprise inherits a trust gap across sessions and across agents. This is a named concept worth tracking: persistent context poisoning means security teams must govern what is remembered, not only what is prompted.

Multi-agent cascades expose a governance assumption that outputs are trustworthy by default. That assumption fails when one compromised agent can pass plausible but tainted output to another. The problem is not just bad data, but unverified delegation across autonomous steps. Security programmes that already manage identity trust chains, approval boundaries, and evidence logging are better positioned to extend those controls into agent workflows.

Continuous observability is becoming the category-defining control for agent security. Static guardrails still matter, but they cannot see multi-step behaviour that becomes malicious only in sequence. The market is moving toward runtime tracing, policy enforcement at tool boundaries, and auditability of every agent action. Practitioners should expect agent governance to converge with NHI control patterns, because the system identity is now the enforcement point.

What this signals

Persistent context poisoning: once an agent stores malicious instructions in memory or retrieval content, the risk survives session resets and reappears across later workflows. Teams should expect agent security to move from prompt moderation toward evidence-grade tracing of memory writes, retrieval events, and tool-boundary enforcement, with OWASP Agentic AI Top 10 and NIST AI RMF both becoming practical reference points.

The governance signal for identity teams is clear. Agents with service credentials now behave like machine identities with agency, so inventory, ownership, and offboarding need to extend into agentic systems. If an organisation cannot see every tool call, memory write, and delegated action, it cannot credibly claim control over the identity layer that is executing those actions.

For practitioners

• Limit each agent to task-scoped tools Define the minimum callable tool set for every agent, then remove broad API, file, and database permissions that are not required for the specific workflow. Review delegated access the same way you review privileged service accounts.
• Monitor memory writes and retrieval paths Log memory updates, retrieval hits, and corpus changes as security events so poisoned context can be detected before it persists across sessions. Treat instruction-like content in memory as a governance exception, not normal state.
• Trace every agent handoff end to end Deploy continuous telemetry across agent steps, sub-agent delegations, and tool calls so anomalous sequences can be identified before the workflow completes. Pair that telemetry with policy checks at the tool-call boundary.
• Classify agent identities inside NHI inventory Add autonomous agents, service credentials, and delegated tokens to the same inventory and review processes used for machine identities. That lets IAM and PAM teams track scope, ownership, and offboarding for agentic systems.
• Use red teaming for escalation paths Test prompt injection, memory poisoning, and downstream propagation scenarios against production-like agent workflows. The goal is to find where benign-looking steps combine into an unauthorized action chain.

Key takeaways

• AI agents expand the security problem from text abuse to action abuse because they can call tools, retain memory, and chain decisions.
• The strongest evidence in the market is a visibility gap, with many organisations unable to audit what their agents access or do.
• Agent security now depends on least privilege, sequence-level observability, and NHI-style governance for delegated identities.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• External Context Poisoning: The injection of misleading, malicious, or outdated external content into an AI assistant’s working context. In practice, the model may treat that content as trusted reference material and reproduce insecure code, bad instructions, or unsafe decisions without recognising the source is untrusted.
• Sequence-Level Observability: Sequence-level observability means tracing every step in an agent workflow, including tool calls, memory access, and sub-agent handoffs. It is needed because many agent attacks only become visible when individual benign-looking steps are analysed as a complete chain.
• Tool Boundary: The tool boundary is the point where an agent's model output becomes an enforced action. In agentic systems, this boundary acts like a permission layer because tool names, schemas, and implementations determine what the agent can actually do, not just what it appears able to request.

What's in the full article

Fiddler's full blog covers the operational detail this post intentionally leaves for the source:

• Step-by-step observability patterns for tracing every agent action across tool calls, memory access, and sub-agent handoffs
• Examples of security signals teams can watch for in production agent telemetry, including drift, anomalous tool use, and output divergence
• Expanded discussion of the AI Trust Tax and why inline evaluation changes the economics of agent governance
• Regulated-industry considerations for audit trails, explainability, and policy enforcement in autonomous decision systems

👉 Fiddler's full post covers the agent telemetry signals, attack surfaces, and regulated-industry governance details.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in practical terms. It helps IAM, PAM, and security teams apply identity control patterns to autonomous systems and other non-human estates.