Topic starter
18/08/2026 2:09 pm
TL;DR: Web and browser agents can appear successful while clicking the wrong element, stopping early, or extracting values that never appeared on page, so Braintrust argues evaluation must score trajectory, state changes, and final output together. The underlying issue is not just response quality but grounded browser behaviour, which is now a governance problem for agentic AI systems.
Editorial analysis by NHI Mgmt Group, based on content published by Braintrust: “How to evaluate web and browser agents”.
Key questions
Q: What breaks when browser-agent traces do not include page state and action history?
A: You lose the ability to tell whether a failure came from the model, the website, or the execution layer.
Q: Why do browser agents need separate scores for task completion and action correctness?
A: A browser agent can choose the right task but execute the wrong action, such as clicking the wrong element or typing into the wrong field.
Q: How do security teams know whether a browser agent evaluation is trustworthy?
A: A trustworthy evaluation records the page evidence, the selected action, and the resulting browser state at each step.
Practitioner guidance
- Define a browser-agent acceptance contract Specify the starting state, permitted actions, required evidence, and the conditions that invalidate a run even if the final response looks correct.
- Capture step-level browser traces Record the observation shown to the model, the action selected, and the resulting page state so failures can be replayed and diagnosed.
- Score task completion separately from response quality Fail runs that fill a form without submitting it, click the wrong element, or extract a value that is not grounded in the page state.
Bottom line: Browser agents can produce plausible outputs while still failing the underlying task, which is why final-response scoring is not enough.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:02 pm
Trace evidence is the governance boundary for browser agents. In a browser task, the real control problem is not whether the model can generate a plausible answer. It is whether the system can prove the action sequence that produced that answer, including the page state before and after each step. For agentic AI programmes, that turns evaluation into a release-control issue, because unverified browser behaviour can create hidden operational failures.
A question worth separating out:
Q: When should teams use browser traces in agentic AI governance?
A: Use browser traces whenever the agent takes actions in a live interface and the result affects a user, system, or business process. The trace becomes the control evidence for release gating, regression testing, and post-incident review. Without it, teams cannot prove what the agent actually did.
👉 Read our full editorial: Web and browser agent evaluation needs action-level trace evidence