AI agent observability gaps are exposing action-level production risk

At a glance

What this is: This is an analysis of AI agent observability and the key finding is that traditional LLM tracing misses tool calls, state changes, and recovery paths that matter most in production.

Why it matters: It matters to IAM and security teams because agents increasingly act like privileged runtime entities, so visibility, authorization, and traceability have to extend beyond prompts and responses.

By the numbers:

• Tool calls fail between 3 and 15% of the time in production, and a single failure can cascade silently across a multi-step workflow.

👉 Read Openlayer's analysis of AI agent observability and tool-call tracing

Context

AI agent observability is the discipline of tracing what an agent actually does, not only what it generates. That distinction matters because agents call tools, write to systems, trigger APIs, and hand work to other agents, which creates governance gaps that standard LLM monitoring does not cover. For identity and access teams, the relevant question is no longer just whether the model produced the right answer, but whether the runtime action was authorised, attributable, and reversible.

The security problem is action visibility across chained executions. When a multi-step workflow spans tool calls, external state changes, and retries, the failure surface looks much closer to privileged automation than to a simple chatbot exchange. That puts observability in the same conversation as NHI governance, because agent runtimes behave like non-human identities with delegated access and persistent side effects.

Key questions

Q: How should security teams trace AI agent failures in production?

A: Trace the whole agent run, not just the final model call. Capture retrievals, tool calls, memory reads, state updates, and handoffs as connected spans so the team can see which step introduced the bad context or decision. That is the difference between guessing at root cause and proving it with evidence from the execution path.

Q: Why do AI agents need more than LLM tracing and logs?

A: Because logs and model-call traces miss the workflow around the model. An agent can fail when retrieval returns stale data, a tool receives the wrong argument, or state changes incorrectly even if the model itself behaved normally. Agent tracing connects those steps so teams can identify the real control failure instead of blaming the last response.

Q: What are the signs that AI agent observability is failing?

A: The main signs are missing tool metadata, disconnected traces between agent hops, unexplained retries, and an inability to reconcile a successful call with the actual state change it caused. If operators can see that a tool ran but cannot prove what it changed, observability is not yet enough for production control.

Q: Should agent observability include runtime enforcement or only logging?

A: It should include both, but logging alone is not sufficient. Logging tells you an unsafe or out-of-policy action happened, while runtime enforcement stops it before the action leaves the inference boundary. For production agents, the control value comes from preventing harmful side effects, not just documenting them after the fact.

Technical breakdown

What AI agent observability must capture

Agent observability needs to record four distinct layers: reasoning traces, tool call behaviour, state changes, and error recovery. A reasoning trace shows how the agent planned, but the tool call record shows which tool it actually used, with what arguments, and what came back. State-change tracing is essential because a successful-looking call can still alter a database, file, or external API in the wrong way. Recovery traces matter because retries and fallback paths can hide systematic failure behind apparent success. Without all four layers, teams are left with partial logs that cannot support root-cause analysis or control enforcement.

Practical implication: Instrument agent telemetry around execution, side effects, and retries, not just prompt and completion logs.

OpenTelemetry GenAI spans and shared trace IDs

Agent observability borrows from distributed tracing. A trace represents the full run, while spans capture the individual LLM calls, tool invocations, memory reads, and routing decisions inside it. The OpenTelemetry GenAI semantic conventions standardise fields such as model name, token usage, and tool name so different frameworks can be compared consistently. In multi-agent systems, the trace must carry a shared ID across every handoff, otherwise a bad context packet passed from one agent to another looks like the downstream agent's fault. That makes correlation, not just capture, a core design requirement.

Practical implication: Adopt shared trace IDs and standard span attributes so you can attribute failure across agent boundaries.

Why action outcome tracing matters more than output logging

An LLM response is just text until an agent uses it to do something. Once the agent writes a record, posts to an API, or changes state in another system, the security and operational consequence becomes real. That is why action outcome tracing is separate from output logging. Teams need to verify whether the intended external state actually changed, not whether the call returned a convenient status code. In agentic environments, observability is incomplete if it cannot answer what changed, when it changed, and whether the change matched the intended goal.

Practical implication: Validate post-action state against expected state before treating an agent step as successful.

Threat narrative

Attacker objective: The attacker or failure mode seeks to make agent actions hard to attribute, hard to stop, and hard to reconstruct before harmful side effects spread.

1. Entry occurs when an agent receives a prompt that leads it to invoke tools and external services with delegated runtime access.
2. Escalation follows when malformed reasoning, silent retries, or wrong tool selection cause the agent to change external state in ways the operator did not intend.
3. Impact appears when those unlogged or poorly correlated actions cascade across a multi-step workflow and the root cause becomes difficult to reconstruct.

NHI Mgmt Group analysis

Action observability is becoming part of identity governance, not just application telemetry. Once an agent can call tools, write records, and chain decisions, it behaves like a non-human runtime identity with delegated authority. That means the governance question is no longer whether output was accurate, but whether the action was authorised, attributable, and bounded. IAM and PAM teams should treat agent traces as control evidence, not as optional debugging data.

Runtime enforcement is the dividing line between visibility and control. Logging that an unsafe action occurred is observation; blocking the action before it leaves the inference boundary is governance. This is the same distinction that separates audit from enforcement in PAM and zero standing privilege models, and it becomes sharper when agents can retry or route around simple checks. Practitioners should align tracing with policy gates, not with post-hoc review alone.

Shared traceability is the named concept this category now needs. In multi-agent workflows, a single trace ID threading through every hop is what preserves causal attribution when one agent hands bad context to another. Without that continuity, teams lose the ability to prove which step introduced drift, which tool mutated state, and where the failure truly started. The practical conclusion is that correlation architecture is now a security requirement for agent governance.

Tool-call failure is a governance signal, not just an engineering defect. A 3 to 15 percent failure band may look like operational noise until you remember that retries, fallbacks, and chained steps can turn one missed call into a larger integrity problem. For identity programmes, the real issue is whether failed actions remain within policy or silently mutate trust decisions. Security leaders should assess agent observability as part of control assurance and incident readiness.

MCP-era agents need identity controls that follow the action path. As agents increasingly rely on tool protocols and delegated access, the identity boundary shifts from login time to execution time. That makes tool-level authorisation, trace continuity, and side-effect logging central to governance. Teams should map agent actions to the same scrutiny they already apply to privileged service accounts and other NHI patterns.

What this signals

AI agent observability is moving into the same control conversation as privileged access and non-human identity governance. If the organisation cannot prove what an agent changed, it cannot confidently claim that access review, audit, or incident reconstruction is complete. The practical shift is toward execution-time controls, shared trace IDs, and policy gates that make agent behaviour governable across the full action path.

Shared traceability will become a design requirement for agent platforms, because distributed handoffs without a consistent execution record create blind spots that audit teams cannot close later. The more agents chain together, the more important it becomes to align tracing with identity and authorisation boundaries rather than with simple application logs.

Teams should expect observability to intersect with standards work around agentic AI and NHI governance, especially where tool protocols such as MCP expand the number of systems an agent can reach. Mapping those connections early helps security leaders decide which controls belong in telemetry, which belong in policy, and which require runtime enforcement.

For practitioners

• Instrument tool-level execution traces Capture the tool name, version, input arguments, raw output, latency, and success or failure status for every invocation so you can reconstruct the real execution path.
• Correlate every hop with a shared trace ID Thread one trace identifier through the full agent chain, including subagents and downstream tool calls, so context handoffs and root cause remain visible across the workflow.
• Verify state after each side effect Check whether a database write, API call, or file change produced the intended external state before the agent continues to the next step.
• Block unsafe actions before inference exit Place policy gates at the runtime boundary so out-of-policy tool use or outputs are stopped before they reach downstream systems or users.
• Review retry and fallback patterns Treat retries as possible signals of prompt ambiguity, tool instability, or hidden failure, and alert when fallback paths become the normal execution path.

Key takeaways

• AI agents create a control problem that standard LLM logging does not solve, because actions and side effects matter more than generated text.
• The production risk is measurable, with tool-call failures, retries, and silent cascades making partial traces unreliable for incident reconstruction.
• Security teams need shared trace IDs, state verification, and runtime policy gates if they want agent observability to support governance rather than just debugging.

Key terms

• AI Agent Observability: AI agent observability is the practice of tracking what an agent does across systems, including actions, permissions, and data access. It provides visibility into behaviour, but it does not by itself establish what the agent was authorised to do or when access should be revoked.
• Tool Call Trace: A tool call trace is a record of a specific external action an agent invoked, including the tool name, input arguments, output, latency, and success status. It provides the evidence needed to reconstruct what happened when an agent interacted with systems beyond the model boundary.
• State Change: A state change is any persistent modification an agent causes in an external system, such as writing a database record, posting to an API, or changing a file. In agent security, state changes matter because they are the point where a model's suggestion becomes a real operational effect.
• Shared Trace ID: A shared trace ID is a single correlation value that threads through multiple agent hops, tools, and subagents so the full execution chain remains connected. Without it, root-cause analysis breaks down when context handoffs or retries obscure which step introduced the failure.

What's in the full article

Openlayer's full analysis covers the operational detail this post intentionally leaves for the source:

• Step-by-step instrumentation guidance for tracing tool calls, retries, and state changes across agent runtimes
• Detailed OpenTelemetry GenAI span mapping for LLM calls, tool spans, and agent spans in production workflows
• Operational examples of how to detect silent failures and reconcile intended state with actual state after execution
• Framework-specific integration detail for teams using agent SDKs and distributed tracing stacks

👉 Openlayer's full post covers trace structure, span design, and production metrics in more operational depth.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It gives security practitioners a common control language for systems that act, not just authenticate.