Notifications
Clear all
Topic starter
18/08/2026 2:09 pm
TL;DR: Web and browser agents can appear successful while clicking the wrong element, stopping early, or extracting values that never appeared on page, so Braintrust argues evaluation must score trajectory, state changes, and final output together. The underlying issue is not just response quality but grounded browser behaviour, which is now a governance problem for agentic AI systems.
NHIMG editorial — based on content published by Braintrust: How to evaluate web and browser agents
By the numbers:
- The average success-rate decrease was 31.6% from easy to medium tasks, followed by a further 15.4% decrease from medium to hard tasks.
- WebJudge variants reported overall precision of 73.7%, 75.7%, and 82.0% in AgentRewardBench.
Questions worth separating out
Q: How should teams evaluate browser agents that can complete a task with the wrong click path?
A: Use a contract that scores both the end state and the trajectory that produced it.
Q: Why do web agents create more governance risk than standard LLM outputs?
A: Because they act on a live interface, not just on text.
Q: What breaks when browser-agent traces do not include page state and action history?
A: You lose the ability to tell whether a failure came from the model, the website, or the execution layer.
Practitioner guidance
- Score the full browser trajectory Record the task, every observation, each selected action, and the resulting page state so you can separate grounding failures from execution failures and response hallucinations.
- Require external completion evidence Pass runs only when the final URL, confirmation number, database row, or other independently verifiable signal proves the task reached the intended end state.
- Separate action correctness from element grounding Check whether the action type was right first, then verify that the chosen element, typed value, or selection matched the intended target in the trace.
What's in the full article
Braintrust's full guide covers the operational detail this post intentionally leaves for the source:
- Code examples for action-type and action-value scorers used in browser-agent evaluation
- Trace structuring patterns for screenshots, DOM snapshots, and browser execution spans
- Browserbase and Browser Use instrumentation details for production-style agent traces
- Dataset and regression workflow examples for turning confirmed failures into repeatable eval cases
👉 Read Braintrust's guide to evaluating web and browser agents →
Web agent evals and browser traces: what teams need to measure?
Explore further
18/08/2026 3:02 pm
Browser agents create a delegated-access problem, not just an evaluation problem. When an agent can click through a live interface, the security question becomes whether the system grounded the right action against the right page object. That is close to an identity governance issue because the agent is operating with borrowed authority inside a browser session. If the trajectory is not traced, the organisation cannot prove that the delegated action matched the intended control boundary.
A question worth separating out:
Q: How should organisations validate browser-agent success before production release?
A: Require an independently verifiable end state such as a confirmation number, final URL, or written record, and compare it with the trace. If the run only looks successful in the response text, it should fail. That prevents false completion claims from entering release criteria.
👉 Read our full editorial: Web and browser agent evaluation needs action-level trace evidence
