Evaluator

Expanded Definition

An evaluator is a runtime control point inside an agent workflow that inspects a prompt, tool request, or tool output and decides whether execution can continue. In NHI and agentic AI security, evaluators are the moment where policy becomes enforceable action, not just design intent.

Evaluators are closely related to filters, guards, and policy checkpoints, but the term is used more precisely when the decision happens during execution rather than during static review. That distinction matters because an evaluator can observe live context such as sensitive tokens, prompt injection patterns, unusually broad tool arguments, or unsafe output that would not be visible in a pre-deployment review. Definitions vary across vendors, but the security function is consistent: inspect, decide, and either allow, block, or route for higher assurance. This maps well to zero trust thinking described in the NIST Cybersecurity Framework 2.0, where access decisions are continuous rather than one-time. The most common misapplication is treating an evaluator as a logging component, which occurs when teams record risky actions without actually enforcing a runtime decision.

Examples and Use Cases

Implementing evaluators rigorously often introduces latency and false-positive tuning overhead, requiring organisations to weigh stronger runtime control against smoother agent execution.

• A prompt evaluator blocks instructions that try to exfiltrate secrets or override policy before the agent reaches a tool call.
• A tool-request evaluator checks whether the requested action matches the agent’s approved scope and rejects excessive permissions.
• A tool-output evaluator inspects returned data for sensitive records, unsafe code, or policy-violating content before it is passed onward.
• An incident-response workflow uses evaluators to pause execution when a high-risk pattern appears, then escalates for human review.
• Security teams studying agent governance can pair this concept with the Ultimate Guide to NHIs to understand how runtime decisions fit into broader NHI control design, while standards-oriented teams can align implementation with NIST Cybersecurity Framework 2.0.
• In higher-risk architectures, a second evaluator may be placed after tool execution to verify that the returned result still satisfies policy before downstream use.

Why It Matters in NHI Security

Evaluators matter because they reduce the gap between entitlement and actual execution. Without them, an agent or service identity may hold legitimate access but still perform unsafe actions when prompted, manipulated, or misrouted. This is especially important in environments where NHI sprawl already creates exposure: NHI Mgmt Group reports that 97% of NHIs carry excessive privileges, increasing unauthorised access and broadening the attack surface, as documented in the Ultimate Guide to NHIs. Evaluators are therefore a practical control for limiting blast radius when agentic systems touch credentials, APIs, and downstream automation.

They also support zero trust by forcing every high-impact action to earn its way through policy, not merely through authenticated identity. In mature implementations, evaluators complement secret governance, permission scoping, and audit logging rather than replacing them. They are most valuable when an organisation must prove that agent actions are constrained at the point of use, not just documented in a policy file. Organisations typically encounter evaluator requirements only after a prompt injection, unsafe tool call, or data leakage event, at which point runtime enforcement becomes operationally unavoidable to address.