A testing approach that measures an AI agent against declared behavioural rules rather than ad hoc expectations. It is useful for identifying failures in tool use, routing, and policy compliance before those failures reach production.
Expanded Definition
Spec-driven evaluation is a structured way to test an AI agent, workflow, or model-assisted application against explicit behavioural specifications. Instead of judging outputs informally, teams define the expected actions, constraints, and prohibited behaviours in advance and then measure whether the system complies under repeatable test conditions. For agentic systems, this can include tool selection, refusal behaviour, routing decisions, escalation logic, and whether the agent stays within approved policy boundaries. The approach is increasingly used because many failures are not obvious from a single prompt-response check; they appear only when the system chains tasks, calls tools, or handles ambiguous instructions.
Definitions vary across vendors, but the core idea is stable: evaluation is driven by the spec, not by subjective reviewer judgement after the fact. That makes it closer to assurance testing than simple benchmark scoring. In security-led environments, the specification may reflect policy, access boundaries, or workflow rules aligned to frameworks such as the NIST Cybersecurity Framework 2.0. The most common misapplication is treating a loose checklist or prompt template as a formal specification, which occurs when teams never define pass or fail criteria before testing begins.
Examples and Use Cases
Implementing spec-driven evaluation rigorously often introduces more upfront design work, requiring organisations to weigh faster ad hoc testing against the cost of writing and maintaining testable rules.
- An AI support agent is evaluated on whether it refuses requests to reveal secrets, even when a user supplies persuasive or conflicting context.
- A routing agent is tested to confirm that high-risk tickets are escalated to a human instead of being auto-closed or reassigned incorrectly.
- An internal coding assistant is checked against a specification that limits which repositories, packages, or deployment actions it may touch.
- A retrieval-augmented generation system is measured on whether it cites approved sources and avoids fabricating unsupported claims when evidence is missing.
- An enterprise tool-using agent is validated against a policy spec that aligns with security expectations described in the NIST Cybersecurity Framework 2.0, especially where action boundaries matter.
These examples show why the method is useful in agentic AI security. It does not just ask whether a model is “good enough”; it asks whether behaviour remains inside defined limits when the system is under pressure, confused, or prompted to overreach.
Why It Matters for Security Teams
Security teams care about spec-driven evaluation because many AI incidents stem from behaviour that was never formally tested against a policy. A model may appear safe in demos yet fail when it receives chained instructions, conflicting context, or access to tools that expand its impact. When the specification is explicit, teams can detect broken escalation logic, unsafe delegation, weak refusal handling, and policy bypass before those failures become operational incidents.
This matters especially for agentic AI, where the system can take actions rather than merely generate text. In that setting, spec-driven evaluation helps translate governance into testable checks: what the agent may read, what it may change, when it must stop, and when it must hand off. That makes the approach relevant to identity-bound controls, privileged workflows, and NHI oversight when non-human identities or agent credentials are involved. It also supports clearer auditability because results can be traced to declared requirements rather than subjective review notes. Teams that ignore this discipline often discover unsafe behaviour only after a bad tool action, a policy breach, or an unintended external call, at which point spec-driven evaluation becomes operationally unavoidable to explain and contain the failure.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10, CSA MAESTRO and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AIRMF frames AI governance and measurement around explicit risk-based expectations. | |
| NIST AI 600-1 | The GenAI profile supports structured evaluation of generative AI behaviours and controls. | |
| OWASP Agentic AI Top 10 | Agentic AI guidance emphasizes testing tool use, routing, and unsafe action paths. | |
| CSA MAESTRO | MAESTRO covers governance and testing for agentic AI systems with action authority. | |
| OWASP Non-Human Identity Top 10 | NHI guidance is relevant where evaluations cover non-human credentials and privilege boundaries. |
Test whether agents stay within their non-human identity permissions and secret-handling rules.
