Topic starter
20/08/2026 5:40 pm
TL;DR: Harness reliability depends on model behaviour and task shape, according to Arize, with GPT-4o still producing two false finishes in a 117-run benchmark even after broader model convergence. The practical lesson is that agent loops need adaptive exit checks, configurable tunings, and multi-model evals because static assumptions about narration, tool use, and completion can fail quietly.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “Agent harnesses have an expiration date”.
Key questions
Q: How should teams prevent AI agents from exiting early when models narrate before acting?
A: Use an explicit or adaptive finish condition that requires evidence of completed work, not just a text response that sounds final.
Q: Why do AI agent harnesses break when model behaviour changes?
A: Harnesses often encode unstated assumptions about when narration ends and tool use begins.
Q: What are the signs that an AI agent finish rule is too brittle?
A: Frequent silent exits, mismatches between announced intent and executed tools, and different outcomes across model versions all point to a brittle finish rule.
Practitioner guidance
- Implement adaptive finish logic Add exit checks that distinguish narrated intent from actual task completion so the harness does not stop on text-only responses.
- Test harnesses across multiple models Run the same task suite against more than one model checkpoint and compare completion, false finishes, and token cost over time.
- Surface hidden tunings as controlled variables Move iteration caps, retry budgets, and stop conditions out of implicit defaults and into versioned configuration reviewed in evals.
Bottom line: Agent harnesses can fail silently when completion logic assumes a model will always separate narration from action in the same way.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
20/08/2026 6:24 pm
Adaptive finish logic is now a governance requirement for agentic AI, not an optimisation choice. The article shows that completion semantics are model-dependent, which means a harness can be correct for one checkpoint and wrong for another without any visible code change. That is a control-plane issue for agentic systems because it determines when delegated actions stop. Practitioners should treat exit conditions as a governed policy boundary, not a convenience shortcut.
A few things that frame the scale:
- Claude Code-assisted commits leaked secrets at a rate of 3.2%, more than double the human-only baseline of 1.5%, with peaks reaching 31 secrets per 1,000 commits in August 2025, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should teams use explicit finish steps or adaptive exit checks for agents?
A: Use the stricter control when agents can perform consequential actions, but choose the version that fits the model mix and token budget. Explicit finish is clearer, while adaptive exit checks can catch narrated false finishes with less overhead than a fully manual workflow.
👉 Read our full editorial: AI agent harnesses need adaptive finish logic to survive model drift