Agent observability is becoming the control plane for AI agent quality

At a glance

What this is: Agent observability records the internal execution path of AI agents, and Braintrust says that is now necessary because traditional APM can miss silent agent failures.

Why it matters: IAM and AI security teams need this shift because agent behaviour, handoffs, and memory use create governance gaps that request-level telemetry cannot reliably surface.

Context

Agent observability is the gap between knowing that an AI agent returned a response and knowing how it reached that response. Traditional monitoring was built for deterministic services, so it can report latency, errors, and span counts but not the semantic steps that explain wrong-tool calls, looping, or stale memory use. For AI programmes, that means quality failures can remain invisible until they affect users or downstream decisions.

The governance issue is that agents behave like runtime systems with changing plans, tool choices, and memory state. That makes them harder to validate with conventional observability and harder to govern with static release checks alone. Where agents perform work inside customer-facing or operational workflows, trace-level visibility becomes part of quality control rather than optional debugging.

Key questions

Q: How should teams implement agent observability across agent frameworks?

A: Start by standardising trace capture for tool calls, reasoning steps, memory operations, and handoffs, then normalise those spans across frameworks through a shared schema. If native adapters exist, use them; otherwise instrument through OpenTelemetry so evaluation and alerting work consistently across stacks.

Q: Why do traditional monitoring tools fail to capture AI failure modes in production?

A: Traditional monitoring assumes deterministic behavior, where crashes and error spikes reveal the problem. AI systems can return a 200 status while producing wrong, unsafe, or irrelevant answers. The failure often sits in retrieval, prompt regression, model drift, or tool selection, so teams need quality and context signals, not just infrastructure health metrics.

Q: What are the signs that an agent trace is missing critical context?

A: Look for repeated retries without a clear cause, tool calls that do not match the user task, unexpected plan changes, or handoffs that lose the original intent. Those patterns usually mean the trace is incomplete, the state is not preserved correctly, or the agent boundary is hiding the real failure.

Q: How should teams investigate recurring failures in agent production traces without losing the thread across code, evals, and monitoring?

A: Teams should treat production observability as a continuous workflow, not a one-off triage exercise. Start with a recurring behavior, attach evidence from traces, then carry that context into datasets, evaluators, dashboards, and monitors. The key is to preserve the investigation thread so findings can be tested, validated, and turned into measurable improvements instead of remaining isolated observations.

Technical breakdown

Why traditional observability misses agent failure modes

Traditional APM assumes relatively stable execution paths, but AI agents branch, retry, and change course based on model output. A request can succeed at the transport layer while the agent still picks the wrong tool, passes the wrong arguments, or drifts from the original plan. Agent observability solves this by turning each step into a typed span that preserves tool calls, reasoning steps, memory operations, and state transitions. That makes the failure visible at the point where the agent actually made the decision, not only at the point where the request completed.

Practical implication: Instrument agent runs at the step level so quality problems are visible before they become user complaints.

Nested spans and handoffs in multi-agent systems

Multi-agent workflows add a second failure class: one agent can pass incomplete or incorrect context to another, and the downstream agent will continue on the wrong assumption. Parent-child span propagation keeps the handoff payload, the receiving agent run, and the final response in the same trace. That matters because the apparent failure in Agent B may actually originate in Agent A. Without nested spans across boundaries, the team loses the causal chain that explains the error.

Practical implication: Propagate trace IDs and handoff context across agents so root cause analysis survives agent-to-agent delegation.

Trace-to-evaluation loops as an operating model

Observability alone tells teams what the agent did, but not whether the outcome was correct. The article’s important pattern is the coupling of live trace scoring with offline eval suites, so production failures become reusable test cases. That creates a feedback loop between runtime behaviour and release quality. In practice, the trace becomes both evidence and training data for the next evaluation cycle, which is a stronger control than dashboards alone.

Practical implication: Feed failed production traces into eval suites and block releases that regress on the same cases.

Threat narrative

Attacker objective: The objective is not classic intrusion but uncontrolled agent behaviour that escapes normal monitoring and degrades decisions or actions without immediate detection.

1. Entry occurs when a user request triggers an AI agent workflow that can branch through multiple tool calls, memory reads, and sub-agent handoffs.
2. Credentialed or contextual abuse appears when the agent selects the wrong tool, reuses stale state, or follows a corrupted handoff into an incorrect action path.
3. Impact is silent quality failure, where the system appears healthy at the request layer while producing wrong answers, bad actions, or broken workflows.

Breaches seen in the wild

• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.
• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agent observability is becoming the control plane for AI quality: once agents can take multi-step actions, quality control can no longer sit at the request boundary. A 200 response or a completed task says little about whether the agent used the right tool, preserved the right context, or stayed within the intended decision path. That is why trace-level governance is moving from debugging aid to operational control for AI programmes.

Trace capture changes the governance model because it turns behaviour into evidence. AI teams have historically certified prompts, model settings, or test results, but agent systems need proof of what happened at runtime. Structured traces create auditability for tool use, memory access, and handoffs, which are the exact places where silent failures hide. Practitioners should treat those traces as control records, not just observability output.

Multi-agent systems create a new accountability problem: the failure may be upstream of the symptom. When one agent hands off incomplete context to another, the downstream action can look like a local defect even when the root cause is delegation failure. That makes nested spans and lineage a governance requirement for AI operations. Teams that cannot reconstruct the chain of action cannot reliably assign responsibility or improve controls.

AI quality now depends on release discipline, not only runtime alerting. The most useful control pattern in the article is the loop from production traces into eval suites and CI gates. That is a clear sign that AI governance is converging with software delivery governance, where regressions are caught before they reach users. Practitioners should expect agent observability to sit inside quality assurance, change control, and incident review processes.

Memory drift is the named concept teams should watch for in agent programmes. Stale reads, wrong-entity retrieval, and leaking state across sessions are not minor telemetry issues, they are governance failures in the agent's decision surface. When memory becomes part of execution, the quality of retrieval and state transition becomes as important as model accuracy. The control question is whether the agent can prove what it knew at the moment it acted.

What this signals

Trace-based control will become a baseline expectation for AI operations. Teams that treat agent output as sufficient will miss the real governance surface, which is the sequence of tool use, memory access, and delegation decisions that produced the output. The programme implication is that observability, evaluation, and release gating now need to be designed together, not bolted on later.

Agent handoffs are a control boundary, not just an implementation detail. Once one agent can pass work to another, the quality of the context transfer becomes a governance issue in its own right. Practitioners should watch for delegation chains that cannot be replayed end to end, because those are the places where accountability and remediation break down.

For practitioners

• Instrument tool-call and reasoning spans Capture tool name, arguments, return values, retries, and decision branches for every agent step so the team can reconstruct why a result happened.
• Propagate parent-child trace IDs across handoffs Keep a single trace through agent-to-agent delegation so the upstream context, downstream action, and final response remain linked for analysis.
• Convert failed production traces into eval cases Use live failures to expand the evaluation suite, then run the same checks in CI so regressions are caught before release.
• Record memory reads and writes as typed events Log retrieval queries, returned entries, freshness, and state changes so stale context and cross-session leakage are visible during review.
• Gate releases on quality scores, not just latency Set thresholds for trace-based evaluation and block deployments when agent quality drops even if infrastructure metrics still look healthy.

Key takeaways

• Agent observability exposes the internal steps that traditional monitoring cannot see, including tool choice, reasoning changes, memory access, and multi-agent handoffs.
• The operational risk is silent failure, where an agent appears healthy at the request layer while still acting on the wrong branch or stale context.
• The strongest control pattern is to connect traces to evaluation and release gates so production failures become repeatable tests before they recur.

Key terms

• Agent observability: Agent observability is the collection and correlation of traces, logs, metrics, and evaluations across an AI agent’s execution path. It explains what happened during model calls, tool use, handoffs, and downstream effects, but it does not itself authorize, deny, or revoke access.
• Nested Span: A nested span is a child record inside a larger trace that preserves the relationship between one step and the next. In agent systems, nested spans show how a planning step leads to tool use, how one agent hands work to another, and where a failure first appeared.
• Trace-to-Eval Loop: A workflow that converts production traces into reusable test cases for future evaluation. It closes the gap between live failures and regression testing, allowing teams to improve prompt, retrieval, and model quality using evidence from real usage.
• Handoff Propagation: Handoff propagation is the practice of carrying context, identifiers, and trace lineage from one agent to another when work is delegated. It is essential in multi-agent systems because the downstream action can only be understood if the upstream context is preserved intact.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It is designed for practitioners who need to connect identity controls to the broader security programmes they run.