Behavior specs for long-horizon agents: what controls are missing?

TL;DR: Long-horizon AI agents need process supervision, not just outcome evaluation, because hundreds of decisions across a single trajectory can hide failure even when the final result looks correct, according to Braintrust and Basis. The implication for practitioners is that agent governance must become trajectory-based, with explicit behavior specs, judgeable evidence, and standing evals for drift.

NHIMG editorial — based on content published by Braintrust: Behavior specs, an open standard for supervising long-horizon agents

Questions worth separating out

Q: How should teams govern long-horizon AI agents without over-relying on outcome checks?

A: Teams should evaluate both the result and the path the agent took to reach it.

Q: Why do long-horizon agents create governance risk for IAM teams?

A: Because the agent behaves like a runtime subject with access to tools and data, but its policy memory can decay over time.

Q: What do security teams get wrong about evaluator design for agents?

A: They often treat evaluators as fixed infrastructure instead of versioned instruments that can drift.

Practitioner guidance

• Define trajectory-based controls for critical agent workflows Identify the few agent behaviours that carry the most risk, then write explicit specs for intent, evidence, execution, recovery, and failure modes so they can be judged consistently.
• Bind agent evaluations to standing review cycles Create repeatable evals that replay real trajectories after model, prompt, or tool changes, and require a pass before expanding production scope.
• Treat tool-using agents as governed software identities Limit standing authority, constrain tool access to the minimum required for each task, and review whether the agent retains privileges longer than the workflow needs.

What's in the full article

Braintrust's full blog covers the operational detail this post intentionally leaves for the source:

• The exact behavior spec structure used to define intent, evidence, decision, execution, recovery, and failure modes
• The judgeable verdict model for production trajectories, including true, false, and NA outcomes
• The open-source workflow for writing and testing behavior specs on your own traces
• The collaboration details between Braintrust and Basis on agentbehavior.dev

👉 Read Braintrust's blog on behavior specs for supervising long-horizon agents →

Behavior specs for long-horizon agents: what controls are missing?

Explore further

View Full Forum →  |  NHI Foundation Course →