Baseline Refusal

Baseline refusal is the behaviour of a model when exposed to a benchmark prompt without any jailbreak strategy applied. It helps reveal whether the test set is genuinely adversarial or whether the benchmark is inflating attack success through ambiguous or mislabeled prompts.

Expanded Definition

Baseline refusal is the model’s native response to a prompt before any jailbreak, prompt injection, or other adversarial manipulation is introduced. In AI security evaluation, it acts as the control condition that shows whether a model is refusing because the prompt is genuinely unsafe, or because the benchmark has been written in a way that triggers refusal for ordinary reasons. That distinction matters because an inflated refusal rate can make a system look more resilient than it is, while a poorly constructed test set can make a model appear vulnerable when it is simply following its safety policy.

NIST’s NIST Cybersecurity Framework 2.0 is useful here as a governance anchor because baseline refusal is ultimately about measurement discipline, repeatability, and evidence quality. In practice, the term sits near adjacent ideas such as false positive refusal, policy compliance, and adversarial test validity. Usage in the industry is still evolving, and definitions vary across vendors when they blend baseline refusal with safety refusal or content filtering outcomes.

The most common misapplication is treating any refusal as evidence of successful defense, which occurs when evaluators do not compare the model’s response to the unmodified benchmark prompt.

Examples and Use Cases

Implementing baseline refusal rigorously often introduces evaluation overhead, requiring teams to separate harmless prompts from truly adversarial ones and to maintain clean prompt labels across test runs.

• A red team compares a model’s answer to a medical prompt with and without jailbreak phrasing to determine whether refusal is driven by policy or by prompt ambiguity.
• An AI security team measures benchmark prompts against the untouched baseline before scoring attack success, reducing the risk of overstating adversarial impact.
• A procurement review asks for baseline refusal evidence to validate that a vendor’s safety claims are based on clean test design rather than mislabeled prompts.
• A model evaluation lab uses baseline refusal to separate content moderation behaviour from actual resistance to prompt injection, which improves test credibility.
• Researchers cross-check baseline responses with guidance from the NIST Cybersecurity Framework 2.0 to ensure the test process preserves traceability and repeatability.

Why It Matters for Security Teams

For security teams, baseline refusal is a quality gate on AI evaluation. Without it, attack success rates can be misleading, benchmark comparisons can become unstable, and governance decisions may be made on flawed evidence. This is especially important where large language models are embedded in agentic workflows, because a false sense of safety at the model layer can cascade into unsafe tool use, poor escalation handling, or overconfident automation. Baseline refusal also helps identity and access teams distinguish between an agent refusing because of policy and an agent failing because the request was malformed or outside its operating envelope.

That distinction supports better incident analysis, better red teaming, and better vendor accountability. It also helps teams avoid overfitting controls to one benchmark style while missing real-world prompt patterns. Security leaders should treat baseline refusal as part of the evidence chain, not as a standalone safety claim. Organisations typically encounter the operational cost of weak baseline testing only after a model passes a benchmark but fails in production, at which point baseline refusal becomes unavoidable to explain the gap.

Related resources from NHI Mgmt Group

• Why does leaving Linux outside the passwordless baseline increase identity risk?
• When should organisations expand beyond the baseline controls in NIST 800-53?
• How should teams govern internal mTLS when TLS 1.3 becomes the baseline?
• Behavior Baseline