Agent tracing exposes where AI agent failures actually begin

At a glance

What this is: This guide explains agent tracing and shows how connected spans reveal the exact step that caused an AI agent failure.

Why it matters: It matters because IAM, NHI, and AI governance teams need auditable execution paths for agents that touch tools, data, and state, not just a final answer.

👉 Read Braintrust's guide to tracing and debugging AI agents in production

Context

Agent tracing is the practice of recording a single AI agent run as a connected execution path, so teams can see how prompts, retrievals, tool calls, memory reads, and state changes shaped the final result. For AI agent security and governance, the problem is not just incorrect output, but the loss of evidence about where the failure started and what data or privilege path was involved.

That matters for AI governance, NHI oversight, and broader IAM controls because agents increasingly act through tools, credentials, and stateful workflows. If the trace does not show which step accessed what, security teams cannot reliably separate model error from data error, privilege misuse, or a broken control in the surrounding workflow.

Key questions

Q: How should security teams trace AI agent failures in production?

A: Trace the whole agent run, not just the final model call. Capture retrievals, tool calls, memory reads, state updates, and handoffs as connected spans so the team can see which step introduced the bad context or decision. That is the difference between guessing at root cause and proving it with evidence from the execution path.

Q: Why do AI agents need more than LLM tracing and logs?

A: Because logs and model-call traces miss the workflow around the model. An agent can fail when retrieval returns stale data, a tool receives the wrong argument, or state changes incorrectly even if the model itself behaved normally. Agent tracing connects those steps so teams can identify the real control failure instead of blaming the last response.

Q: How do security teams know if agent observability is actually working?

A: Observability is working only when teams can tie together token activity, tool calls, and latency for a specific agent session. If those signals are disconnected, the logs may show traffic but not governance. The test is whether investigators can reconstruct the path of a meaningful agent action from start to finish.

Q: How should organisations use failing traces to improve agent governance?

A: Turn each confirmed failure into a labelled regression case with source spans, expected output, and a scorer that tests for the same pattern before release. That creates a feedback loop from production evidence to evaluation coverage and makes agent governance measurable rather than anecdotal.

Technical breakdown

What an agent trace records across a full run

An agent trace is a parent-child span tree that captures each unit of work in execution order. Instead of a flat log line, it preserves the inputs, outputs, duration, tool calls, retrieval results, model calls, state updates, and handoffs that occurred between the user request and the final answer. That structure matters because an agent failure often emerges from the interaction between steps, not from one isolated model response. Tracing makes the intermediate path visible, so teams can inspect where the run diverged from expected behavior.

Practical implication: instrument the full agent path, not only the final model call, if you need usable production diagnostics.

Why tracing is different from logging and LLM tracing

Logging tells you that something happened. LLM tracing tells you what prompt went into a model and what came back out. Agent tracing goes further by connecting model calls to retrievals, tool invocations, memory reads, and state changes inside one run. That distinction is crucial when the model behaves correctly but the surrounding workflow fails, such as when an outdated retrieval result, wrong tool argument, or missing state update changes the answer. The trace shows causality across the run, not just isolated events.

Practical implication: use agent tracing whenever the failure could come from retrieval, tools, or state, not just prompt quality.

How regression evals turn failing traces into durable controls

The main operational value of agent tracing is not only debugging. A failed production trace can become a labelled regression case with source spans, expected output, and a scorer that checks for the same failure pattern later. That turns a one-off incident into a repeatable control for future releases. For agentic systems, this is especially useful because the same mistake can recur through different prompts, models, or tool paths. Tracing plus evals creates a feedback loop from production evidence to test coverage.

Practical implication: convert confirmed trace failures into eval cases before you ship the next agent change.

Threat narrative

Attacker objective: The objective is not classic intrusion but the control failure of an AI agent run, where the organisation cannot prove which step produced the bad decision or whether sensitive context was mishandled.

1. Entry occurs when a user request or live context enters the agent workflow and the system records only the final response instead of the full execution path.
2. Escalation happens when a retrieval miss, wrong tool argument, or stale state update shapes the model's decision without being visible in standard logs.
3. Impact is a wrong or unsafe production response that cannot be traced back quickly enough to prevent repeat failure or policy violation.

NHI Mgmt Group analysis

Agent tracing is becoming a governance control, not just an engineering convenience. Once agents can retrieve data, call tools, and update state, the security question is no longer whether the final answer looks plausible. The question is whether the organisation can reconstruct the decision path well enough to explain error, misuse, or credential exposure. That makes traceability part of AI governance, NHI oversight, and incident review.

Trace visibility closes the accountability gap between model output and control failure. A flat log can confirm that an agent replied, but it cannot show whether the problem was stale retrieval, an over-permissioned tool, or an invalid state transition. That gap becomes more serious as AI systems interact with production data and delegated access. Teams need step-level evidence because accountability disappears when only the last message survives.

Connected spans create the named concept of execution-path evidence. This is the practical difference between knowing an agent failed and knowing which control failed. Execution-path evidence lets security, IAM, and governance teams tie behaviour back to the exact span where context, privilege, or retrieval went wrong. The practitioner conclusion is simple: if you cannot inspect the path, you cannot govern the run.

Turning traces into evals is how agent governance becomes repeatable. Production traces should not end as debugging artifacts. They should become regression cases that test the same retrieval, tool, or state failure before release. That shifts agent governance from after-the-fact review to continuous control validation, which is the only sustainable model for fast-changing agentic systems.

What this signals

Execution-path evidence is becoming a practical requirement for AI programmes that use tools, retrieval, and stateful orchestration. As agent populations grow, teams will need to prove where a response came from, not simply that a response exists. The operational signal is whether your trace layer can distinguish model behaviour from workflow failure before the issue becomes a security or compliance event.

The governance question is shifting from whether an agent is powerful enough to whether its decision path is inspectable enough. That maps directly to NIST AI Risk Management Framework expectations around measurement and governance, and it intersects with identity controls whenever the agent uses delegated access, secrets, or machine credentials. If traceability stops at the model boundary, the control stack is incomplete.

Agent tracing also changes how teams should think about sensitive data access by AI systems. If the organisation cannot trace what a system retrieved, used, or forwarded, then policy enforcement becomes hard to prove and incident response becomes slower. For practitioners managing AI, NHI, and IAM together, trace completeness is now a programme-level control signal, not just a developer convenience.

For practitioners

• Instrument the full execution path Capture model calls, tool invocations, retrievals, memory reads, state updates, and handoffs as connected spans so failure analysis does not stop at the final response.
• Separate model issues from workflow issues Use trace review to determine whether the model, the retrieval layer, the tool call, or the state transition caused the failure before you change prompts or policies.
• Convert failed traces into regression cases Label each confirmed failure with a root cause, attach the source spans, and add it to a pre-release eval set so the same pattern is caught earlier next time.
• Trace custom logic outside the framework Wrap business rules, data transforms, and bespoke retrieval functions in manual spans so the trace reflects the complete agent path rather than only framework-managed steps.

Key takeaways

• Agent tracing matters because the root cause of many AI failures sits in retrieval, tool use, or state, not in the final answer.
• Production traces become much more valuable when they are converted into regression evals that block repeat failures before release.
• For AI governance and identity teams, inspectable execution paths are becoming a control requirement, not an optional debugging feature.

Key terms

• Agent Trace: A structured record of an AI agent’s runtime activity, including model calls, tool calls, approvals, and subagent steps. In practice, traces support debugging, evaluation, and governance when they are retained, searchable, and tied to the permissions behind the agent.
• Span: A span is one discrete operation inside a trace, such as a retrieval query, prompt assembly step, or model call. Spans make it possible to pinpoint which part of an AI workflow consumed time, introduced error, or passed bad context to the next stage.
• Regression Eval: A regression eval is a repeatable test case built from prior failures. It uses captured inputs, expected outputs, and a scorer to verify that a previously observed problem does not reappear after a change to prompts, models, tools, or retrieval logic.
• Execution Path: An execution path is the chain of systems, roles, functions, and permissions that an identity uses to reach a target service. For AI agents, this matters more than the label on the agent itself because effective authority often comes from the path, not the object.

What's in the full article

Braintrust's full guide covers the operational detail this post intentionally leaves for the source:

• Framework-specific setup steps for OpenAI Agents SDK, LangGraph, and Mastra instrumentation
• Code examples for adding manual spans around custom logic and data transformation steps
• OpenTelemetry export configuration for teams already using distributed tracing
• Workflow details for converting a failed trace into a regression eval with scorers and release gates

👉 Braintrust's full guide covers trace instrumentation, nested spans, and regression eval workflows in implementation detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps security practitioners connect identity controls to the systems now making autonomous or semi-autonomous decisions.