Production usage patterns are the right starting point for evals

At a glance

What this is: This is an analysis of how to turn production AI agent traces into evaluation datasets by clustering them into task, sentiment, and issue patterns.

Why it matters: It matters because teams building and governing AI systems need tests that reflect real production behavior, including failure modes that manual suites and random samples often miss.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.

👉 Read Braintrust's guide to turning production usage patterns into eval datasets

Context

AI agent eval quality depends on whether the test data reflects how the system is actually used in production. The primary gap is that random trace sampling and hand-written tests often miss rare failures, while production usage patterns reveal the workflows, frustrations, and breakpoints that matter most. In agentic AI programmes, that same gap becomes an identity and governance issue because traces may expose access patterns, data access scope, and tool use that should be reviewed as part of AI control design.

Braintrust's article frames topics, sentiment, and issues as a practical way to convert trace logs into evaluation slices. That approach is especially relevant for teams responsible for AI governance, security review, and the operational boundaries of AI agents, because the same production traces that improve testing can also reveal where agent behaviour is drifting beyond intended scope. In this case, the starting position is broadly typical for teams that have telemetry but not yet a disciplined eval pipeline.

Key questions

Q: How should teams turn production AI agent traces into eval datasets?

A: Start with clustered production traces, not random samples. Group interactions by task, sentiment, and issue type, then promote the clusters that represent common workflows, concentrated frustration, or concrete failures. This keeps evals anchored in real usage and reduces the chance that the test suite reflects only pre-launch assumptions.

Q: Why do random samples fail to give good AI agent eval coverage?

A: Random samples scatter rare failures too thinly to notice and often miss the exact edge cases that create production risk. They may also overrepresent routine traffic and underrepresent the interactions that frustrate users or break the agent. Clustering makes those patterns visible before dataset promotion.

Q: What do security and AI teams get wrong about trace-based evals?

A: They often assume any log sample is good enough. In practice, the value comes from reviewing trace groups that have clear behavioural meaning, validating the labels, and only then turning them into repeatable tests. Otherwise the eval suite becomes a noisy mirror of logs rather than a reliable control.

Q: How do organisations know if their AI eval rubric is actually useful?

A: They should calibrate it against human review on a sample of cases and measure disagreement. If the automated scorer diverges from practitioner judgment too often, the rubric is optimizing the wrong thing. The best evals are domain-specific, explainable, and tied to the outcomes the team actually cares about.

Technical breakdown

Why production traces outperform random sampling for eval coverage

Production traces are the closest thing to a ground truth record of what users actually asked the agent to do, how the agent responded, and where the interaction broke down. Random samples spread those traces thinly and often underrepresent rare but expensive failures. A better evaluation pipeline starts by clustering traces so that repeated workflows, concentrated frustration, and concrete breakpoints become visible before dataset selection begins. For AI governance teams, that matters because a trace corpus can also reveal where data access, tool calls, or delegated actions need closer review. The dataset should reflect observed behaviour, not the developer's expectation of it.

Practical implication: build eval datasets from clustered production traces before you decide which failures deserve test coverage.

How task, sentiment, and issue facets work together

Task groups traces by user intent, sentiment captures the tone of the exchange, and issues identify concrete failure conditions such as tool-call errors, incomplete answers, or repeated retries. Each facet answers a different question, but the value comes from reading them together. A high-volume task cluster tells you what matters most, sentiment shows where users are unhappy, and issues pinpoint what broke technically. In agentic AI environments, that combination helps separate normal usage from risky behaviour and operational failure. It also gives reviewers a way to detect when the system is being pushed into edge cases that may affect access, policy adherence, or tool use.

Practical implication: use the three facets as a triage system, not as isolated labels.

Why custom facets turn logs into domain-specific eval signals

Custom facets extend the built-in pattern by extracting product-specific signals that generic task or sentiment labels will miss. A team can define labels for churn risk, regulated workflows, endpoint-specific usage, or compliance flags, then cluster traces along those dimensions. The key architectural point is that the extractor turns unstructured interaction history into stable review categories that can be reused in regression testing. For AI systems with identity or access implications, custom facets can also surface whether the agent is operating inside a permitted scope or repeatedly touching sensitive workflows. That makes the eval dataset more representative and more governable.

Practical implication: add custom facets when built-in trace labels do not capture the control question you are trying to test.

NHI Mgmt Group analysis

Production-derived evals are becoming a governance control, not just a QA convenience. When AI agents operate in live environments, the traces they produce describe behaviour, scope, and failure patterns that should inform control design. That makes eval datasets part of operational governance, especially where agent decisions interact with data access, tool use, or delegated actions. The broader lesson is that teams can no longer separate test quality from control assurance. Practitioners should treat trace-derived evals as a standing part of AI governance.

Task, sentiment, and issues expose a useful named concept: evaluation signal collapse. That is what happens when teams rely on a single dimension such as random sampling, expected-user test cases, or generic success metrics. The article shows that each facet surfaces a different risk signal, and only the combined view reveals which traces deserve promotion into a dataset. The concept matters because production AI failures rarely appear as a single clean label. Practitioners should design for multi-signal review rather than single-metric comfort.

Clustering before promotion reduces blind spots in AI agent oversight. Rare failures can disappear in routine traffic, and hand-written tests often overfit the team's assumptions about how the system should behave. Clustering creates a visible review surface for low-frequency, high-severity behaviours before they are lost in noise. In governance terms, that improves evidence quality for model review, regression control, and incident analysis. Practitioners should validate clustered samples before turning them into eval datasets.

Custom facets are where AI evals become operationally relevant to identity and access risk. Once teams can label traces by compliance flags, regulated workflows, or access-sensitive actions, they can test not only output quality but also control boundaries. That intersection matters for agentic AI programmes because the same trace that reveals a bad answer can also reveal a bad delegation chain or excessive data reach. Practitioners should align facet design with the actual control question, not with abstract model quality alone.

What this signals

Trace clustering is becoming an operational input into AI governance. Teams that can convert logs into repeatable eval slices gain a better view of how agents behave in production, which failures recur, and where control boundaries need review. For agentic AI programmes, that means evaluation quality and governance evidence increasingly depend on the same underlying telemetry.

Evaluation signal collapse: when teams rely on one label, one metric, or one sample strategy, they miss the multi-dimensional behaviour that production agents actually exhibit. The practical response is to combine task intent, sentiment, and issue patterns, then add custom facets when the control question is more specific than generic quality.

As production AI usage grows, the review burden shifts from building more tests to building better selection logic. Teams should expect more pressure to justify why particular traces were promoted, why certain clusters were excluded, and how their eval datasets map back to current user behaviour.

For practitioners

• Build eval slices from clustered production traces Start with Task, Sentiment, and Issues to identify high-volume workflows, concentrated frustration, and concrete failures, then promote only the trace groups that represent real regression risk.
• Review rare failures before they disappear in sampling noise Use clustering to pull low-frequency but high-severity traces into visible review groups, especially where a small failure rate could still affect production outcomes.
• Add custom facets for governance-relevant labels Create domain-specific facets for regulated workflows, compliance flags, or access-sensitive interactions when the built-in labels do not answer the control question you are testing.
• Validate cluster labels before promotion Inspect sampled traces inside each cluster to confirm the model-generated label is accurate before turning the group into a permanent eval dataset.

Key takeaways

• Production traces are a better source for eval datasets than random sampling because they preserve the workflows and failures that matter in live use.
• Task, sentiment, and issue clustering gives teams a practical way to find high-value test slices before they promote traces into regression suites.
• Custom facets make eval design more governable by aligning trace labels with the specific risk or control question the programme needs to answer.

Key terms

• Evaluation Dataset: An evaluation dataset is a structured set of representative cases used to test whether an agent behaves acceptably before or after release. It helps teams measure output quality, tool selection, escalation behaviour, and consistency so that production decisions are based on repeatable evidence rather than intuition.
• Trace Clustering: The process of grouping production traces by similarity so recurring behaviours become easier to review. It helps teams find high-volume workflows, concentrated frustration, and distinct failure modes before deciding which trace groups should become tests or monitoring signals.
• Custom Facet: A domain-specific label extracted from traces to classify behaviour along a dimension built around the product or control question. It extends generic task or sentiment analysis by making logs useful for governance, compliance, or specialised operational review.
• Issue Pattern: A repeated failure signal in trace data, such as tool-call errors, incomplete answers, or retry loops. Issue patterns are valuable because they identify concrete breakpoints that can be translated into targeted evals and regression tests.

What's in the full article

Braintrust's full article covers the operational detail this post intentionally leaves for the source:

• Step-by-step Topics workflow for clustering traces from raw logs into reviewable datasets
• Example SQL and API patterns for promoting filtered trace groups into datasets
• Custom facet prompt design examples for domain-specific trace classification
• Daily regeneration and review cadence for keeping eval data aligned with production behaviour

👉 The full Braintrust article covers clustering workflow, dataset promotion, and custom facet examples.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps practitioners connect identity control thinking to the broader programmes that depend on trustworthy automation.