Notifications
Clear all
Topic starter
01/08/2026 10:43 pm
TL;DR: Attack success rate in jailbreak research can shift by 50 percentage points or more depending on attempt budget, prompt set, and judge design, according to Promptfoo and the NeurIPS 2025 position paper it cites. The metric is useful, but only when teams report the measurement unit, stopping rules, and scoring rubric alongside it.
NHIMG editorial — based on content published by Promptfoo: Attack success rate is not comparable without a shared method
By the numbers:
- We track published jailbreak research through a database of over 400 papers, which we update as new work comes out.
Questions worth separating out
Q: How should security teams interpret jailbreak attack success rate in AI testing?
A: Treat ASR as a conditional measurement, not a universal vulnerability score.
Q: Why do different jailbreak papers report such different attack success rates?
A: Because they often test different things under the same label.
Q: What do AI security teams get wrong about benchmark-based red teaming?
A: They often assume the benchmark is objective when the prompt set and rubric may be doing most of the work.
Practitioner guidance
- Define the ASR unit before benchmarking State whether ASR is measured per attempt, per prompt, or per goal category, and record the exact attempt budget, decoding settings, and stopping rule for every run.
- Separate clear violations from borderline prompts Review each benchmark prompt against the governing policy and mark ambiguous items explicitly.
- Calibrate judges with explicit rubrics Write pass and fail criteria that define actionable harm, refusal quality, and acceptable high-level guidance.
What's in the full article
Promptfoo's full article covers the operational detail this post intentionally leaves for the source:
- The full methodology checklist for reporting ASR with explicit attempt budgets, prompt sets, and judge rules.
- Worked examples showing how best-of-K testing changes the meaning of a jailbreak result.
- A reproducible evaluation setup for promptfoo red teaming runs, including baseline and attack configurations.
- The paper references and benchmark comparisons used to explain judge variance and prompt-set distortion.
👉 Read Promptfoo's analysis of why jailbreak attack success rate is hard to compare →
Attack success rate in jailbreak papers: what are teams actually measuring?
Explore further
01/08/2026 10:53 pm
ASR is a measurement construct, not a universal risk number. The article shows that identical attacks can produce wildly different results when the unit of analysis changes from one attempt to many. That means the field still lacks a common measurement language for jailbreak evaluation. For AI security governance, reproducibility is the real control objective, because unmanaged variance turns research claims into untrusted signals.
A question worth separating out:
Q: How can organisations make adversarial AI testing results reproducible?
A: Use fixed prompt sets, documented attempt budgets, explicit scoring rubrics, and a calibration set for the judge model. Then publish baseline refusal, attack configuration, and stopping logic together. Reproducibility depends on method disclosure, because ASR without methodology is not comparable across studies.
👉 Read our full editorial: Attack success rate is not comparable without a shared method
