Fixed LLM judges break when the important evidence is spread across many tool calls, state transitions, and retries rather than concentrated in one final answer. They can miss decisive context, fail to verify the underlying system state, and keep grading against a rubric that no longer matches production behaviour.
What actually breaks in fixed-judge evaluation of long-horizon agents
Fixed judges assume the thing being judged is visible in a short artifact. Long-horizon agents are different: the real behaviour unfolds across planning, tool use, state updates, retries, and partial failures. Once the evaluation target becomes distributed over time, a judge that only sees the final answer stops measuring the system that actually produced it.
That mismatch is most obvious when a judge rewards a polished endpoint while ignoring the path that got there. An agent can recover from a wrong intermediate step, succeed after an unnecessary retry, or quietly violate a constraint before correcting itself. A fixed rubric often cannot tell the difference between robust execution and accidental success.
The result is not just lower fidelity, but a change in what the evaluation selects for. Teams may optimise agents to produce outputs that satisfy the judge, while hiding process weaknesses that matter in production. In other words, the judge becomes part of the system’s feedback loop, and the metric starts shaping behaviour instead of reflecting it.
Why spread-out evidence defeats a static rubric
Long-horizon agents create evidence that is temporally fragmented. One tool call establishes a precondition, another mutates state, a later retry compensates for an error, and the final response looks normal. A fixed LLM judge is weak at reconstructing that causal chain unless the full trajectory is supplied in a structured form and the rubric is designed for stateful reasoning.
That is why “final answer quality” is often the wrong proxy. In agentic workflows, the important question is whether the sequence of actions was safe, necessary, and consistent with the task constraints. If the judge cannot inspect intermediate state transitions, it may miss a dangerous escalation, a hidden dependency, or a silent failure that later got papered over.
For this reason, evaluation has to distinguish outcome correctness from trajectory correctness. A system can be useful and still be unsafe, or be unsafe and still look correct at the end. When a judge only scores the end state, those two cases collapse into the same grade.
How evaluation drift shows up in practice
The first sign of drift is rubric mismatch. Early in development, a judge may track the intended workflow, but once the agent is upgraded, the tooling changes, or the task length increases, the rubric no longer captures the production behaviour. The judge still appears stable because it is grading the same surface pattern, while the agent has moved on.
Another common failure is context loss. Long-horizon agents often accumulate enough evidence that no single prompt can hold it all. Fixed judges may overlook an important branch because it is buried several steps back, or because the decisive state was only visible in a tool response, log line, or retry condition. That creates false confidence in both success rates and safety claims.
When evaluations are used for release decisions, this drift can be operationally expensive. Teams may promote agents that look reliable in benchmark form but fail under real orchestration pressure, where delays, errors, and partial recovery are normal. The more the environment changes, the more a static judge becomes a snapshot of yesterday’s workflow.
Risk and Threat Considerations
Static judging creates a false sense of control in systems whose behaviour is path-dependent. The risk is not only bad scoring, but bad governance: teams may miss unsafe tool use, unreviewed state changes, or recovery behaviour that masks a real control failure. That matters because long-horizon agents often operate with delegated authority and can produce apparently successful outputs even after violating policy earlier in the run.
Failure mechanism: The judge evaluates a compressed artifact instead of the full action trajectory, so intermediate mistakes, retries, and state mutations never enter the score. The rubric then optimises for end-state plausibility rather than execution integrity.
Impact: Production agents can be released with hidden failure modes, including unsafe actions that are only visible in the sequence of steps, not in the final answer. Over time, benchmark performance and real-world reliability diverge, and incident detection gets harder because the evaluation system itself failed to preserve the evidence needed to explain the behaviour.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST AI RMF, NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | ASI08 — Cascading Failures | Long-horizon agent failures accumulate across chained steps and retries. |
| ASI03 — Identity & Privilege Abuse | Agent evaluation must catch unsafe delegated actions across tool use and state changes. | |
| Recommendation — Evaluate trajectories for cascading failures, not only final outputs. Review delegated actions and privilege use across the full run. | ||
| NIST AI RMF | Govern | The question is about evaluation governance and alignment of metrics to production behaviour. |
| Recommendation — Align evaluation metrics to the actual operational behaviour being governed. | ||
| CSA MAESTRO | Threat modeling and risk analysis | Long-horizon agents need structured analysis of orchestration, retries, and emergent failures. |
| Recommendation — Model multi-step agent workflows to expose emergent failure paths. | ||
| NIST CSF 2.0 | GV.OV-01 — Oversight | Evaluation must provide oversight that matches the system’s real behaviour. |
| ID.RA-05 — Threats, Vulnerabilities and Impacts are Used to Inform Risk Response | Missed intermediate failures distort the risk picture for agentic systems. | |
| DE.CM-09 — Configuration and System Monitoring | Distributed agent actions require monitoring beyond the final output artifact. | |
| Recommendation — Use oversight mechanisms that verify agent behaviour across the whole workflow. Incorporate trajectory-level failures into risk assessment and response decisions. Monitor agent tool use and state changes, not only end results. | ||
| NIST SP 800-53 Rev 5 | AU-6 — Audit Record Review, Analysis, and Reporting | Trajectory-aware judging depends on reviewing logs and intermediate records. |
| AU-12 — Audit Record Generation | Intermediate steps must be recorded if judges are to assess long-horizon behaviour. | |
| Recommendation — Review audit records to validate agent behaviour across the full sequence. Generate audit records for tool calls, retries, and state transitions. | ||
Practitioner Guidance
What to verify: Judge the trajectory, not just the destination. If your evaluation cannot reconstruct tool calls, state transitions, and retries, it is not yet measuring long-horizon agent quality.
Decision rule: Use a fixed judge only for narrow, stable tasks where the final answer fully represents the work. Once the task involves planning, branching, recovery, or mutable state, move to trajectory-aware evaluation and make intermediate evidence part of the score.
What practitioners underestimate: The biggest error is treating judge quality as a model problem instead of a systems problem. In long-horizon agents, the failure is often in the evaluation boundary, not the model’s language ability.
Practitioner takeaway: The more an agent’s work is distributed across time, the less a single end-state rubric can be trusted to represent actual performance.