The share of prompts a model declines to answer. In AI security evaluation, it is a useful but incomplete signal because a model can refuse unsafe tasks while still failing to produce correct answers on permitted tasks.
Expanded Definition
Refusal rate is a behavioural metric used in AI security evaluation to measure how often a model declines to respond to a prompt. In safety and governance discussions, it is often treated as a proxy for a model’s willingness to avoid harmful, policy-violating, or out-of-scope requests, but that interpretation is incomplete. A high refusal rate can indicate stronger boundary enforcement, yet it can also conceal overblocking, poor task understanding, or brittle prompt handling. A low refusal rate does not necessarily mean a model is safe; it may simply be over-permissive.
Definitions vary across vendors and evaluation labs, especially around what counts as a refusal, a partial answer, or a softened response. For that reason, refusal rate should be read alongside accuracy, calibration, and harmful completion rates rather than in isolation. NHI Management Group treats it as a governance signal, not a standalone assurance measure. For broader evaluation context, see the NIST Cybersecurity Framework 2.0, which emphasises risk-based outcomes over single-point metrics.
The most common misapplication is using refusal rate as proof of model safety when the model is actually refusing benign prompts or still answering unsafe prompts in disguised form.
Examples and Use Cases
Implementing refusal rate rigorously often introduces a measurement tradeoff, requiring organisations to weigh safety sensitivity against user utility and evaluation consistency.
- A model refuses direct requests for malware instructions, and evaluators track the refusal rate to confirm that safety controls are active.
- A customer-support assistant rejects legitimate account-recovery prompts because the policy classifier is too broad, revealing that a high refusal rate can reduce service quality.
- A red team tests whether a model refuses harmful content when prompts are rephrased, translated, or wrapped in benign context, since refusal patterns can change with wording.
- An AI governance team compares refusal rate with task success on permitted prompts to identify whether safety tuning has introduced avoidable overblocking.
- A security team reviewing agentic workflows checks whether the system declines tool-use requests it should not execute, using refusal behaviour as one signal among many in a control assessment. For a governance-oriented lens on risk and accountability, the NIST Cybersecurity Framework 2.0 remains a useful reference point.
Why It Matters for Security Teams
Refusal rate matters because security teams need to know whether a model is actually resisting unsafe behaviour or merely suppressing output in ways that create false confidence. In AI security, a poorly interpreted refusal metric can hide two opposite failures: unsafe compliance on harmful prompts and unnecessary refusal on legitimate prompts. Both are operationally relevant. The first increases exposure to misuse, policy bypass, and harmful content generation. The second degrades productivity, frustrates users, and can push work into unmanaged channels where controls are weaker.
For teams governing AI systems, refusal rate also intersects with identity and authorization design. In agentic environments, a refusal to answer may need to align with a refusal to act, especially when an AI agent has tool access, delegated permissions, or access to sensitive data. That makes refusal behaviour part of broader control design, not just model tuning. It is best assessed together with NIST Cybersecurity Framework 2.0 outcome thinking and, where relevant, AI risk governance practices.
Organisations typically encounter refusal-rate problems only after users report blocked legitimate work or after a safety review shows the model still answered a prohibited prompt, at which point the metric becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST AI RMF, NIST AI 600-1 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | AI RMF frames trustworthy AI evaluation, including safety behaviour and risk metrics. | |
| NIST AI 600-1 | GenAI profile addresses model behaviour evaluation, including safety and misuse resistance. | |
| NIST CSF 2.0 | GV.RM-01 | CSF risk management supports interpreting refusal rate as an outcome metric within governance. |
Compare refusal behaviour with permitted-task performance to spot overblocking or unsafe compliance.
