AI agent trace facets: when built-in labels are too broad

TL;DR: Built-in Topics facets such as Task, Sentiment, and Issues can be too coarse for production AI agent review, and custom facets can turn recurring questions into stable, workflow-linked labels for dashboards and evals, according to Braintrust. The practical issue is governance: without narrowly scoped, repeatable classifications, teams cannot reliably track quality, failure modes, or product signals across traces.

NHIMG editorial — based on content published by Braintrust: How to design custom facets for AI agent traces (2026)

Questions worth separating out

Q: How should teams design custom facets for AI agent traces?

A: Start with one operational question, define mutually exclusive labels, and make sure each label routes to a concrete workflow.

Q: Why do broad trace labels fail in AI agent governance?

A: Broad labels collapse too many behaviours into the same bucket, so the classifier cannot separate meaningful patterns from noise.

Q: What do security teams get wrong about observability labels for AI agents?

A: They often treat labels as reporting categories instead of decision tools.

Practitioner guidance

• Define one review question per facet Start with a single operational question such as churn risk, tool reliability, or citation quality, then build labels only for that decision.
• Use the narrowest trace slice that contains the signal Choose user-only, error-window, or metadata-projected input based on the label you need.
• Tie every label to a downstream workflow Make sure each label routes to a concrete next step, such as review, escalation, filtering, or scoring.

What's in the full article

Braintrust's full article covers the implementation detail this post intentionally leaves at the governance level:

• Preprocessor patterns for user-only traces, failure windows, and metadata projection in production facets
• Prompt structures and label boundaries that improve consistency across repeated reruns
• Exclusion regex handling for NONE and N/A outputs so empty classifications do not pollute clustering
• Worked examples for support, coding, multilingual chatbots, and sales or PLG agents

👉 Read Braintrust's guide to designing custom facets for AI agent traces →

AI agent trace facets: when built-in labels are too broad?

Explore further

View Full Forum →  |  NHI Foundation Course →