A property of an evaluation result where the agent both completes the task and remains within the allowed access boundary. In practice, this means a correct answer is not considered valid if it was produced through unauthorized reads, writes, or delegated actions.
Why authorization-aware validity matters
Authorization-aware validity separates a merely correct result from a trustworthy one. If an agent reaches the right outcome by reading data it should not see, writing where it should not write, or exercising delegated power beyond its boundary, the result is operationally useful but security-invalid.
This distinction matters because autonomous systems can appear successful while silently violating policy. In practice, the quality check is not just “did it work?” but “did it work through permitted access paths and authority only?”
How to judge a result against the allowed access boundary
The boundary is defined by what the agent was allowed to read, change, invoke, or delegate at the time of execution. A valid result stays inside those permissions even if the task spans multiple steps, external tools, or intermediate retrievals.
That means the evaluation must consider the full action path, not only the final answer. A response can be semantically correct and still fail validity if it depended on overbroad retrieval, an unauthorized API call, a privileged write, or a borrowed session that was never meant for that task.
Where validity breaks down in practice
Authorization-aware validity usually fails when task success is achieved through hidden privilege expansion, cross-context leakage, or delegated actions that were never intended for the agent’s operating scope. The problem is especially sharp in systems that blend retrieval, tools, and action execution because the boundary can be crossed before the final result is produced.
It also breaks when downstream systems treat outcome quality as proof of legitimacy. A correct completion can mask an access-control failure, which means the evaluator must inspect both the result and the path used to obtain it.
What this property changes in security evaluation
Authorization-aware validity turns access control into part of the success criterion. It pushes teams to assess not only accuracy or task completion, but whether the agent stayed within its allowed authority throughout the workflow.
For governance, that means evaluation logic, logging, and review criteria need to record the permissions in effect at each step. Without that, an apparently good outcome may be accepted even though it depended on unauthorized reads, writes, or delegation.
Risk and Threat Considerations
When this property is missing, organisations can reward unsafe agent behaviour because the system appears to have performed correctly. That creates a blind spot where unauthorized access is normalised as long as the end result looks useful.
Failure mechanism: The agent completes the task by stepping outside its permitted read, write, or delegation boundary, and the evaluation pipeline fails to detect that the path was unauthorized.
Impact: Sensitive data exposure, unintended state changes, privilege abuse, and false confidence in the agent’s operational safety can all follow from accepting a result that was only “correct” because it overreached.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-05 — Overprivileged NHI | Evaluates whether an agent used excess authority to complete a task. |
| Recommendation — Limit agent permissions to the minimum scope needed for each action. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | Covers agent success that depends on misused identity or excess privilege. |
| Recommendation — Constrain agent authority and verify each tool action against policy. | ||
| NIST SP 800-53 Rev 5 | AC-6 — Least Privilege | Directly addresses outcomes that become invalid because access exceeded need-to-know. |
| AU-6 — Audit Record Review, Analysis, and Reporting | Supports review of how an apparently valid result was produced. | |
| IA-5 — Authenticator Management | Covers credential and token handling that can enable unauthorized delegated actions. | |
| Recommendation — Enforce least privilege so task execution cannot rely on unauthorized access. Review logs to confirm the action path stayed within approved authority. Control and rotate credentials so delegated actions remain attributable and bounded. | ||
Practitioner Guidance
Why practitioners should care: Treat this as a control property, not a reporting preference. If a system cannot prove that task success stayed inside the allowed boundary, then result quality and security assurance are being measured as if they were the same thing.
What to watch for: Pay attention to evaluations that only score final answer correctness, especially when the workflow includes retrieval, tool use, delegated actions, or ambient credentials. Those are the cases where hidden boundary violations are most likely to be missed.
Related resources from NHI Mgmt Group
- How should teams implement authorization-aware filtering in data queries?
- Who should own data-aware authorization in an enterprise?
- What breaks when authorization is not tenant-aware?
- How should security teams implement authorization-aware search without repeatedly traversing large permission graphs?