Failure rate is the share of phishing simulation recipients who take the unsafe action the test is designed to trigger. That may mean clicking a link, opening an attachment, or submitting credentials. On its own, it shows exposure to a lure, but not whether users also recognised and reported the message.
What Failure Rate Measures in Phishing Simulations
Failure rate is a behavioural outcome metric, not a full effectiveness score. It captures the proportion of recipients who took the lure’s intended unsafe action, so it is best read as a snapshot of susceptibility to that specific test design.
The same campaign can produce a high failure rate even when the broader programme is improving, because the metric reflects only the target action, not whether recipients recognised the message, escalated it, or learned from prior exercises.
What the Metric Does and Does Not Tell You
Used well, failure rate helps teams compare simulation runs, audience segments, and lure types. It can show which messages or channels still create the most exposure, but it should not be treated as a standalone measure of user awareness or security maturity.
The main blind spot is scope. A simple click rate may understate real risk if a user also entered credentials, while a credential-submission rate can overstate weakness if the exercise used an unusually convincing pretext. Context, audience, and scenario difficulty all matter.
How Teams Interpret Failure Rate Responsibly
Practitioners usually interpret failure rate alongside reporting rate, repeat-failure patterns, and the realism of the simulation. The most useful question is not only “who failed,” but also whether the organisation is improving its ability to recognise, resist, and report phishing attempts over time.
Because the measure is behaviour-specific, it is most valuable when trend lines are compared across similar campaigns rather than across unrelated exercises. A consistent metric design makes the result more defensible than a single headline percentage.
Common Pitfalls in Using the Metric
Failure rate is often over-read as a proxy for human error or programme failure. That is too narrow. A poor score can reflect weak training, but it can also reflect message realism, poor simulation targeting, seasonal workload, or a test that is intentionally harder than normal traffic.
Another common mistake is to optimise only for a lower failure rate. If people become better at spotting tests but still do not report suspicious messages, the organisation may improve the number while leaving a real response gap.
Risk and Threat Considerations
High failure rates indicate that a phishing lure can still move recipients from exposure to unsafe action, which increases the chance of credential theft, malware delivery, or account compromise. The metric is especially important when simulation results mirror real-world attack paths rather than just training artefacts.
Failure mechanism: A recipient clicks, opens, or submits data because the lure successfully exploits attention, trust, urgency, or habit, creating an access or malware foothold.
Impact: The resulting exposure can extend beyond the individual recipient to email account compromise, lateral movement, fraud, or a larger social engineering campaign.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and CIS Controls v8 set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.AT-01 — Security Awareness and Skills are Comprised of the Organization's Business Environment, Roles, and Responsibilities | Failure rate is interpreted through awareness and role-based susceptibility. |
| DE.CM-09 — Information and Events from Internal and External Sources are Correlated | Failure rate trends become more useful when correlated with reporting and incident signals. | |
| Recommendation — Use awareness results to refine role-specific phishing education and simulation design. Correlate simulation outcomes with report and incident data to spot recurring exposure patterns. | ||
| CIS Controls v8 | CIS-14 — Security Awareness and Skills Training | Phishing failure rate is a training outcome that helps validate awareness effectiveness. |
| CIS-8 — Audit Log Management | Tracking simulation outcomes and responses supports measurement of awareness and detection. | |
| Recommendation — Use simulation results to target training where unsafe responses remain high. Retain simulation and reporting data so you can measure behavioural change over time. | ||
| ISO/IEC 27001:2022 | A.6.3 — Information security awareness, education and training | Failure rate is a practical indicator for awareness and training effectiveness. |
| Recommendation — Review phishing simulation results to validate awareness and training control effectiveness. | ||
Related resources from NHI Mgmt Group
- What breaks when teams do not test auth failure paths like 422, 500, or rate limiting?
- How should security and engineering teams calculate change failure rate so the metric reflects real delivery risk?
- Why does change failure rate give a better view of delivery risk than deployment failure rate?
- What are the signs that change failure rate is being calculated inaccurately?
