Behavior specs for long-horizon agents expose the supervision gap

At a glance

What this is: Braintrust argues that behaviour specs are needed because outcome-only evaluation cannot reliably supervise long-horizon agents that make hundreds of decisions across a single run.

Why it matters: IAM and AI security teams need process-level governance for agents because trajectory risk, not just final output, determines whether delegated actions remain trustworthy.

Context

The core governance gap is that long-horizon agents can look correct at the end of a task while having taken unsafe, ungrounded, or inefficient steps along the way. That breaks outcome-only supervision, because the control is trying to certify a result rather than the behaviour that produced it.

In agentic AI programmes, this is an identity and authority problem as much as a quality problem. Once an agent can decide when to act, which tools to use, and how long to continue, the organisation needs a way to define and judge acceptable behaviour across the full trajectory.

Behaviour specs are Braintrust's answer to that gap, but the underlying issue is broader than one implementation. Any team running long-horizon agents needs a durable way to express expected conduct, measure adherence, and detect drift before unsafe delegation becomes normal.

Key questions

Q: What fails when long-horizon agents are judged only on final output?

A: Final-output-only evaluation misses the path the agent took, including whether it grounded decisions properly, used the right evidence, or recovered safely from ambiguity. A result can look correct while the trajectory contains unsafe or unapproved behaviour, so the control is too weak to support trust in delegated execution.

Q: Why do long-horizon agents need trajectory-based supervision?

A: Because the security and governance risk sits in the sequence of actions, not just the endpoint. Long-running agents can make many tool calls and judgments before completion, so organisations need a way to evaluate the behaviour that produced the result, not only the result itself.

Q: What do security teams get wrong about evaluator design for agents?

A: They often treat evaluators as fixed infrastructure instead of versioned instruments that can drift. If evaluator scores do not correlate with real user outcomes, the programme is optimising for the wrong signal and may miss slow degradation, false confidence, or miscalibrated quality thresholds.

Q: How should organisations govern agent behaviour as models and tools change?

A: They should keep a sparse set of authoritative behaviour specs, tie each one to a standing eval, and treat disagreements between runtime prompts and the spec as a governance defect. That gives the team a durable control for drift, rather than a static document that quickly becomes stale.

Technical breakdown

Why outcome evals fail on long trajectories

Outcome evaluation checks the final answer, not the path taken to get there. That works for short, verifiable tasks, but long-horizon agents can make hundreds of intermediate decisions, use tools, and revise state before producing a result. If the outcome is hard to verify, expensive to reproduce, or correct for the wrong reasons, a pass tells you very little about process quality. This is why a trajectory needs its own supervision model: the organisation must be able to judge whether the agent grounded its decisions properly, used the right evidence, and followed the expected sequence of actions. In agentic systems, correctness at the finish line is not the same thing as trustworthy conduct throughout the run.

Practical implication: move evaluation from output-only checks to trajectory evidence that can be judged step by step.

What a behaviour spec actually defines

A behaviour spec is a written control description for an agent's intended conduct in a specific situation. It is not a prompt and it is not runtime code. Instead, it states what evidence the agent should inspect, what decision it should make, what execution it should perform, what recovery path it should follow, and which failure modes it is meant to avoid. That structure matters because it makes the behaviour judgeable by a human or an automated evaluator reading the trace. In identity terms, it is a way to constrain delegated authority with explicit expectations rather than implicit trust. The spec becomes a governance object, not just documentation.

Practical implication: write specs for the behaviours that carry risk, then judge traces against those expectations.

Why standing evals matter for agent governance

Standing evals turn a behaviour spec into a recurring control rather than a one-off review. When the agent, the prompts, the tools, or the context change, the organisation can test whether the behaviour still holds. That is important because agentic systems drift in ways traditional application testing does not catch: a model may still produce the right answer while quietly degrading in how it reasons, verifies, or escalates. For agent governance, the question is not only whether the system works today, but whether the intended behaviour survives future changes. A standing eval is how teams keep the control alive.

Practical implication: attach every high-risk behaviour spec to a recurring eval so drift becomes measurable.

NHI Mgmt Group analysis

Behaviour-level supervision is becoming the missing control plane for agentic AI. Outcome metrics can confirm that an agent reached a useful end state, but they cannot show whether the agent used acceptable evidence or respected the intended decision path. That leaves a governance gap between delegated action and verified conduct, which matters whenever an agent can persist through many steps without human review. Practitioners should treat process supervision as a control requirement, not a documentation preference.

Trajectory-based evaluation is the right response to delegated autonomy. When an AI system can choose tools, timing, and intermediate actions, the risk lives in the sequence, not just the result. That makes the behaviour spec concept especially relevant to agentic AI identity governance, because the organisation must define what the agent is allowed to do as it acts, not merely what it may produce at the end. Practitioners should align authorisation, evaluation, and audit around the full trajectory.

Specs become governance only when they are sparse, stable, and enforced. The article is right to warn that teams already encode behaviours across prompts and tools, but most of those controls are too diffuse to govern consistently. A named concept emerges here: trajectory supervision gap: the distance between a correct final output and an unverified chain of decisions that produced it. Practitioners should close that gap with durable specs that outlive model updates and prompt churn.

The control objective is not more instructions, but more judgeable intent. Long-horizon agents need fewer vague directions and more testable behaviour requirements that can be evaluated against traces. That shifts AI governance toward repeatable assurance, where policy, runtime context, and evaluation criteria are kept in sync. Practitioners should use behaviour specs to make delegated authority observable before it becomes difficult to unwind.

From our research library:

• 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
• 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Trajectory supervision gap: the real governance problem is not whether an agent can produce a correct answer, but whether the organisation can prove the path was acceptable. That pushes AI programmes toward traceable evidence, standing tests, and sharper delegation boundaries before autonomy expands further.

For practitioners, the key change is to treat behaviour definitions like durable controls rather than disposable prompts. Once agent systems can complete long tasks with limited oversight, the programme needs recurring evaluation that survives prompt churn, tool changes, and model updates.

For practitioners

• Define trajectory-level behaviours for high-risk agent tasks Write behaviour specs for tasks where an agent can make many decisions before a final output is produced, especially when evidence quality, sequence, or recovery paths matter.
• Attach standing evals to every critical behaviour spec Run repeated trace-based evaluations whenever prompts, tools, models, or context change so drift is detected as a control failure rather than a quality issue.
• Separate outcome checks from process checks Keep final-result scoring, but add judgeable evidence requirements for intermediate steps so a good end state does not hide unsafe execution.
• Keep the spec sparse and authoritative Limit each spec to the behaviours that materially affect trust, then make the spec take precedence over runtime context when the two disagree.

Key takeaways

• Long-horizon agents can appear reliable at the output layer while still taking unsafe or ungrounded steps during the run.
• Behaviour specs turn expected agent conduct into something that can be judged repeatedly against real trajectories.
• Standing evals are the mechanism that keeps agent behaviour from drifting as models, tools, and prompts change.

Key terms

• Behavior Spec: A behavior spec is a written standard that defines how an AI agent should act across a task, not just what result it should produce. It describes intent, evidence, decision points, execution steps, recovery, and failure modes so the behaviour can be judged consistently from a trajectory.
• Runtime Supervision: Continuous observation of what an identity or agent actually does while it is operating. For NHI governance, runtime supervision helps detect scope creep, unusual tool chaining, and behaviour that is technically permitted but operationally unsafe.
• Standing Eval: A standing eval is a repeatable test that remains active after deployment and is rerun whenever the system changes. In agent governance, it helps teams detect drift in model behaviour, prompt behaviour, or tool usage before the changes expand into production risk.
• Trajectory: A trajectory is the full sequence of actions, tool calls, intermediate decisions, and recoveries an agent takes to complete a task. It is the right unit of analysis for long-horizon agents because the final output alone does not reveal whether the work followed an acceptable path.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore the NHI Foundation Level course, the industry's only accredited NHI security programme, if your programme needs a deeper control model for delegated access and agent behaviour.