Topic starter
18/08/2026 2:47 pm
TL;DR: Long-horizon agents can look reliable at the output layer while still taking unsafe or ungrounded steps, because hundreds of decisions across one trajectory can hide failure, according to Braintrust. Supervising the trajectory itself makes drift, evidence use, and recovery behavior something teams can judge repeatedly.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “Behavior specs, an open standard for supervising long-horizon agents”.
Key questions
Q: Why do long-horizon agents need trajectory-based supervision?
A: Because the security and governance risk sits in the sequence of actions, not just the endpoint.
Q: What do security teams get wrong about evaluator design for agents?
A: They often treat evaluators as fixed infrastructure instead of versioned instruments that can drift.
Q: What fails when long-horizon agents are judged only on final output?
A: Final-output-only evaluation misses the path the agent took, including whether it grounded decisions properly, used the right evidence, or recovered safely from ambiguity.
Practitioner guidance
- Define trajectory-level behaviours for high-risk agent tasks Write behaviour specs for tasks where an agent can make many decisions before a final output is produced, especially when evidence quality, sequence, or recovery paths matter.
- Attach standing evals to every critical behaviour spec Run repeated trace-based evaluations whenever prompts, tools, models, or context change so drift is detected as a control failure rather than a quality issue.
- Separate outcome checks from process checks Keep final-result scoring, but add judgeable evidence requirements for intermediate steps so a good end state does not hide unsafe execution.
Bottom line: Long-horizon agents can appear reliable at the output layer while still taking unsafe or ungrounded steps during the run.
Glossary terms: Behavior Spec, Runtime Supervision, Standing Eval, Trajectory
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
This topic was modified 10 hours ago by NHI Mgmt Group
18/08/2026 3:30 pm
Behaviour-level supervision is becoming the missing control plane for agentic AI. Outcome metrics can confirm that an agent reached a useful end state, but they cannot show whether the agent used acceptable evidence or respected the intended decision path. That leaves a governance gap between delegated action and verified conduct, which matters whenever an agent can persist through many steps without human review. Practitioners should treat process supervision as a control requirement, not a documentation preference.
A few things that frame the scale:
- 53% of security leaders expect AI to run major portions of their infrastructure autonomously within the next three years, according to the 2026 Infrastructure Identity Survey.
- 19% of organisations give AI systems dramatically more access than human employees, nearly one in five granting unrestricted privilege, according to the 2026 Infrastructure Identity Survey.
A question worth separating out:
Q: How should organisations govern agent behaviour as models and tools change?
A: They should keep a sparse set of authoritative behaviour specs, tie each one to a standing eval, and treat disagreements between runtime prompts and the spec as a governance defect. That gives the team a durable control for drift, rather than a static document that quickly becomes stale.
👉 Read our full editorial: Behavior specs for long-horizon agents expose the supervision gap