Continuous evaluation for AI agents needs live trace classifications

At a glance

What this is: Braintrust says AI agent evaluation must move from pre-release test sets to live production traces because curated checks miss failures that only emerge in real use.

Why it matters: For IAM and AI security teams, the key issue is governance over runtime behaviour: classification-driven scoring turns production traces into evidence for monitoring, review, and regression control.

Context

Continuous evaluation is the production counterpart to pre-release testing for AI agents. Pre-deploy and CI checks can only judge known cases, while live traces expose user behaviour, drift, and failure patterns that were not in the original test set. For AI governance teams, the gap is not just model quality. It is whether runtime behaviour is being observed and acted on in a way that can survive scale.

In identity terms, the article is really about control points around agent behaviour rather than human sign-in. Trace classifications create a policy layer for deciding which production interactions become reviewable signals, which matters when AI systems are making decisions or taking actions on behalf of users and services.

Key questions

Q: What breaks when AI agent evaluation relies only on static test cases?

A: Static test cases miss behaviour that appears under load, ambiguity, or changing context. They can also hide failures in tool dependencies, multi-agent handoffs, and adversarial inputs. Without dynamic testing and production monitoring, teams may approve agents that look reliable in the lab but drift, hallucinate, or fail in real workflows.

Q: Why do live trace classifications matter for AI governance?

A: They give teams a structured way to decide which production interactions should be scored, reviewed, or promoted into regression tests. Without classifications, continuous evaluation becomes a collection of brittle one-off rules. With them, governance can focus on recurring patterns instead of isolated traces.

Q: How do security teams keep online scoring from becoming evaluation sprawl?

A: Start with a small, durable set of trace facets and let scorers read those labels instead of building a new scorer for every failure mode. That keeps the control surface manageable, reduces maintenance burden, and makes it easier to explain why a trace was flagged.

Q: What should teams do after a production trace is flagged by a scorer?

A: Assign each flagged trace a clear next step, such as alerting, human review, or promotion into a regression dataset. A score without a downstream action only creates backlog. The value of continuous evaluation comes from making production failures part of an operational feedback loop.

Technical breakdown

Why live trace classifications change production evaluation

Continuous evaluation works because production traffic is different from curated test data. A golden dataset can only cover the cases a team already anticipated, while live traces capture new intents, drift, and edge cases as they occur. Trace classifications add structure to that stream by labelling each trace with facets such as Task, Sentiment, or Issues, so scorers can target patterns instead of individual examples. That reduces the maintenance burden that usually appears when teams try to write one scorer per failure mode. The mechanism is simple: classify first, then evaluate only the traces that match a meaningful predicate.

Practical implication: teams should build scoring logic around stable trace labels, not around one-off examples from a static test set.

How classification-driven scoring avoids scorer sprawl

The article’s core architectural point is that classifiers and scorers should be separated. A small set of facets tags traces first, and scorers then apply rules to those tags. That means a single scorer can cover a broader class of failures, such as negative checkout experiences, without hardcoding a bespoke rule for every symptom. This matters operationally because evaluation sprawl is a governance problem as much as an engineering one. If every failure mode requires a new scorer, the evaluation stack becomes too fragile to maintain, and production monitoring degrades into noise.

Practical implication: define shared classification facets before adding more scorers, or production evaluation will become unmanageable.

Why online scoring is a governance layer, not just a metric

Online scoring does more than produce a score. It creates a repeatable decision path from trace to alert, review queue, or regression dataset. That makes production evaluation part of an operational control loop, not a dashboard metric. The article also distinguishes trace scope from span scope, which matters because some behaviours need full conversational context while others are best evaluated at a single tool-call level. In practice, that means governance teams need to decide what evidence is scored, how often it is sampled, and what downstream action follows a flagged trace.

Practical implication: route every online scoring rule to an explicit response path so flagged traces cannot accumulate without action.

NHI Mgmt Group analysis

Live evaluation is the missing control plane for AI agent quality. Pre-release test sets can validate known behaviour, but they cannot absorb the volume of real user variability that appears after deployment. Classification-driven scoring turns production traces into governable evidence, which is the only practical way to keep evaluation aligned with actual agent use. For practitioners, the lesson is that runtime observability is now part of AI control design, not an optional add-on.

Evaluation sprawl is a governance failure, not an engineering inconvenience. If every failure mode requires a separate scorer, teams will either stop maintaining the rules or stop trusting the output. The classification layer gives practitioners a smaller, more durable policy surface that can be reviewed, tuned, and explained. That is the difference between a controlled monitoring programme and an expanding pile of ad hoc checks.

Production traces need to become policy evidence before they become metrics. The article shows a useful pattern for AI governance: classify, score, then route the result into alerts, review, or regression tests. That sequence matters because it prevents scoring from becoming a passive report. For AI programmes, the right question is not whether you can measure live behaviour, but whether you can turn it into accountable action.

Trace classification is the named concept this article sharpens. It is the idea that live AI agent traces should be labelled first so a smaller set of scoring rules can govern many failure patterns. That approach fits AI governance debt: the longer teams wait to structure production signals, the harder it becomes to explain or defend runtime decisions. Practitioners should treat classifications as part of the control surface, not just telemetry.

For identity and access governance, the deeper lesson is that agent oversight must move closer to runtime authorisation. When an AI system can act across workflows, the control question becomes what it did, under what classification, and whether that behaviour should trigger human review or a regression gate. That makes trace labels and scoring predicates relevant to broader identity governance even when no human login is involved. Practitioners should design policy around agent behaviour at execution time, not only around static permissions.

From our research library:

• 62% of organisations are at least experimenting with AI agents, but only 23% have scaled one into a single business function, according to McKinsey's 2025 State of AI survey.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Trace classification is becoming the practical control point for AI agent oversight. A live agent can generate behaviour that never appeared in pre-release testing, so teams need a way to label production interactions before they decide what to score or escalate. That shifts AI governance from static test coverage to runtime policy enforcement, which is where most real failure handling now has to happen.

AI governance debt builds quickly when every failure pattern gets its own scorer. The article’s classification-first model is a useful answer because it concentrates review logic into a smaller set of reusable labels. For practitioners, the programme implication is clear: structure production signals early, or the evaluation stack will fragment faster than it can be maintained. according to the 2026 Infrastructure Identity Survey

For practitioners

• Define stable trace facets first Establish a small set of trace classifications such as task, sentiment, and issue type before writing production scorers. Use those labels as the policy surface so one scorer can cover a meaningful failure class instead of a single symptom.
• Separate trace scope from span scope Use trace-level scoring for multi-turn conversations and span-level scoring for single operations or tool calls. Match the scope to the behaviour you need to evaluate so the scorer sees the right context.
• Route every flag to a downstream action Connect each online scoring rule to an alert, a review queue, or a regression dataset so a failed trace cannot sit idle. Treat scoring output as the start of triage, not the end of governance.
• Review predicates when classifications change Re-check scoring rules whenever facet prompts shift or clusters are regenerated. A predicate that matched one label yesterday may miss renamed or reweighted classifications after the taxonomy changes.
• Use sampling to keep production scoring sustainable Score critical flows more heavily and high-volume flows at a lower rate so evaluation stays cost-aware. Sampling should preserve coverage of important behaviour without forcing every trace through the same workflow.

Key takeaways

• Production AI agents need live evaluation because curated test sets cannot anticipate every user path, edge case, or drift pattern.
• Classification-driven scoring reduces maintenance burden by letting a smaller set of labels drive broader failure checks across production traces.
• Continuous evaluation only becomes operationally useful when flagged traces feed alerts, review, or regression tests instead of lingering as passive scores.

Key terms

• Continuous Evaluation: Continuous evaluation is an ongoing measurement loop that checks whether an AI system still performs correctly as data, language, and requirements change. For regulated workflows, it is essential because static test sets quickly become stale and can hide drift in accuracy or ranking quality.
• Trace Classification: Trace classification is the process of assigning structured labels to an agent trace, such as task, sentiment, or issue type. Those labels let scorers apply rules to groups of traces rather than evaluating every event individually, which reduces scorer maintenance and makes production monitoring more scalable.
• Online Scoring: Online scoring is the practice of evaluating live AI traffic against predefined quality or safety criteria. It helps teams detect quality drift in production and compare live behaviour with offline test results using the same scoring logic where possible.
• Regression Test Promotion: Regression test promotion is the practice of converting a production failure into a permanent test case for future releases. It closes the loop between live incidents and pre-deploy validation so that a problem found in production becomes part of the next release gate.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps security and identity practitioners connect runtime control decisions to broader governance programmes.