Visible tests check only the cases included in the prompt or harness, while withheld tests probe requirements the model could not directly optimize against. Visible tests can overstate performance because they miss boundary cases and hidden constraints. Withheld tests are essential when you want to measure whether the agent solved the actual task, not just the exposed examples.
Visible tests measure the exposed sample, not the full task
Visible tests are the examples the model can inspect directly, so they mainly measure how well it fits the stated prompt or harness. That makes them useful for quick iteration, but they are also easy to overfit. In evaluation terms, they tell you whether the agent can satisfy the seen cases, not whether it can handle the broader requirement set.
The practical limitation is that visible tests tend to reward pattern matching. If the prompt, rubric, or scaffold makes the expected shape obvious, a model can optimise for those exact cases and still fail on boundary conditions, alternate phrasing, or hidden constraints. That is why visible scores often look stronger than real-world robustness.
Withheld tests check whether the agent solved the underlying objective
Withheld tests are kept out of the prompt and harness, so the model cannot directly tune to them during generation. They are used to probe generalisation, including edge cases, constraint combinations, and failure modes that are not obvious from the visible examples. When designed well, they reveal whether the agent learned the task structure rather than memorising the test surface.
For evaluation design, withheld tests are the stronger signal of task mastery because they reduce benchmark leakage and expose brittle behaviour. They are especially important when the task includes hidden requirements, safety constraints, or operational conditions that should not be inferable from a few examples alone.
That distinction aligns with the broader security lesson that exposed examples are never enough to validate resilience. In practice, hidden checks are what surface over-permissive or brittle behaviour, just as hidden failure modes often reveal whether a control really works under stress. For related reading on how hidden conditions change security outcomes in agent systems, see the agentic AI applications guide and AI Agent Identity Security: The 2026 Deployment Guide. For a broader framework view, compare the control and risk logic in OWASP Top 10 for Agentic Applications 2026 and NIST AI Risk Management Framework.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 address the attack and risk surface, while NIST AI RMF and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A1 — Agentic Prompt Injection and Instruction Hijacking | Hidden tests detect prompt overfitting and instruction bypass in agent evals. |
| Recommendation — Use hidden evaluations to test whether the agent resists instruction hijacking beyond visible examples. | ||
| NIST AI RMF | GOVERN — AI Risk Governance | Withheld tests support trustworthy AI evaluation and governance over real model performance. |
| Recommendation — Define evaluation gates that require withheld-test success before approving model deployment. | ||
| CIS Controls v8 | 8 — Audit Log Management | Evaluation harnesses need traceable evidence that hidden cases were exercised and reviewed. |
| Recommendation — Record evaluation runs and outcomes so hidden-test failures are visible in the review trail. | ||
Practitioner Guidance
What to verify: Treat visible tests as a smoke check and withheld tests as the real acceptance gate. If the model performs well only when the examples are explicit, the evaluation is probably measuring prompt fit instead of task competence.
Common mistake: Teams sometimes add more visible examples and assume coverage has improved. In reality, that can increase overfitting unless the withheld set includes qualitatively different cases, such as boundary values, alternate formats, conflicting constraints, or adversarially chosen inputs.
What good looks like: A strong evaluation has visible tests that help developers iterate quickly, plus withheld tests that independently confirm the agent can generalise. The score gap between the two should be small enough that you trust the result, not just the demo.
Practitioner takeaway: If visible and withheld performance diverge sharply, trust the withheld result and revisit the prompt, tool policy, or task specification before trusting the apparent win.
Related resources from NHI Mgmt Group
- What is the difference between code-based scorers and LLM-as-a-judge scorers for agent evaluations?
- What is the difference between human identity governance and AI agent governance?
- What is the difference between governing human access and governing AI agent access?
- What is the difference between managing human access and managing agent access?
