Topic starter
20/08/2026 5:27 pm
TL;DR: AI agents improve less through better prompts than through a better harness around the model, according to Arize, where traces, targeted evals, failed-span review, and safe change gates turn opaque behaviour into a repeatable engineering loop. The governance lesson is that observability, policy, and review now matter as much as model quality for reliable agent systems.
Editorial analysis by NHI Mgmt Group, based on content published by Arize: “How to build a better agent harness with traces and evals”.
Key questions
Q: What breaks when teams cannot trace what an AI agent did?
A: Governance breaks first, because teams lose the evidence needed to decide whether the issue was behavior, identity, or control.
Q: Why do evals need to be built from real agent traces?
A: Because evals should measure the behaviour that actually produced the outcome, not an abstract guess about what might matter.
Q: What are the signs that an agent improvement loop is failing?
A: The main signs are either every run passing, which suggests the eval is too weak, or every run failing, which suggests the rubric is misaligned.
Practitioner guidance
- Define a trace-first agent workflow Record tool calls, intermediate outputs, model calls, and stop decisions for every meaningful agent run so failures can be replayed at the span level.
- Tie each eval to one observable behaviour Start with one targeted evaluation, such as priority scoring or groundedness, and map it to a specific span rather than the full output.
- Separate diagnosis from deployment Require human review before any prompt, tool, routing, or evaluator change reaches production, especially when eval logic itself changes.
Bottom line: The article’s core message is that durable agent improvement comes from the harness around the model, not from prompt tuning alone.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
20/08/2026 6:16 pm
Harness observability is becoming a governance control, not a debugging feature. Agent programmes fail when teams can see the output but not the path that produced it. That leaves no defensible way to explain why a decision happened, whether the evaluator was aligned, or what changed between one run and the next. For agentic AI security, observability is therefore part of control design, not a post-hoc troubleshooting aid.
A few things that frame the scale:
- According to PwC, 79% of organizations have already adopted AI agents to some degree.
A question worth separating out:
Q: How should teams govern prompts that drive agent tool use?
A: Treat those prompts as high-impact workflow controls, not just text templates. Teams should require ownership, controlled testing, replayable traces, and approval criteria that reflect the downstream actions the agent can take, especially when tools touch sensitive data or external systems.
👉 Read our full editorial: Agent harness traces and evals are the real improvement loop