Topic starter
18/08/2026 2:47 pm
TL;DR: Long-horizon AI agents need process supervision, not just outcome evaluation, because hundreds of decisions across a single trajectory can hide failure even when the final result looks correct, according to Braintrust and Basis. The implication for practitioners is that agent governance must become trajectory-based, with explicit behavior specs, judgeable evidence, and standing evals for drift.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “Behavior specs, an open standard for supervising long-horizon agents”.
Key questions
Q: What fails when long-horizon agents are judged only on final output?
A: Final-output-only evaluation misses the path the agent took, including whether it grounded decisions properly, used the right evidence, or recovered safely from ambiguity.
Q: Why do long-horizon agents need trajectory-based supervision?
A: Because the security and governance risk sits in the sequence of actions, not just the endpoint.
Q: What do security teams get wrong about evaluator design for agents?
A: They often treat evaluators as fixed infrastructure instead of versioned instruments that can drift.
Practitioner guidance
- Define trajectory-level behaviours for high-risk agent tasks Write behaviour specs for tasks where an agent can make many decisions before a final output is produced, especially when evidence quality, sequence, or recovery paths matter.
- Attach standing evals to every critical behaviour spec Run repeated trace-based evaluations whenever prompts, tools, models, or context change so drift is detected as a control failure rather than a quality issue.
- Separate outcome checks from process checks Keep final-result scoring, but add judgeable evidence requirements for intermediate steps so a good end state does not hide unsafe execution.
Bottom line: Long-horizon agents can appear reliable at the output layer while still taking unsafe or ungrounded steps during the run.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:30 pm
Behaviour-level supervision is becoming the missing control plane for agentic AI. Outcome metrics can confirm that an agent reached a useful end state, but they cannot show whether the agent used acceptable evidence or respected the intended decision path. That leaves a governance gap between delegated action and verified conduct, which matters whenever an agent can persist through many steps without human review. Practitioners should treat process supervision as a control requirement, not a documentation preference.
A few things that frame the scale:
- 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
- 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: How should organisations govern agent behaviour as models and tools change?
A: They should keep a sparse set of authoritative behaviour specs, tie each one to a standing eval, and treat disagreements between runtime prompts and the spec as a governance defect. That gives the team a durable control for drift, rather than a static document that quickly becomes stale.
👉 Read our full editorial: Behavior specs for long-horizon agents expose the supervision gap