Failure rate measures how often users take the bad action in a simulation, such as clicking a link, opening an attachment, or submitting credentials. Reporting rate measures how often users do the good action and alert the organisation. The two metrics answer different questions, and together they show whether awareness training is reducing exposure and improving response.
How Failure Rate and Reporting Rate Measure Different Behaviours
Failure rate is a measure of susceptibility: it tells you how often people take the simulated bait and create exposure. reporting rate is a measure of detection and response behaviour: it tells you how often people recognise the test and escalate it to the organisation. A programme can have a low failure rate but still be weak if suspicious messages are not reported quickly.
The practical distinction matters because the two numbers answer different operational questions. Failure rate is about how much simulated risk gets through the human layer, while reporting rate is about how quickly the organisation gets signal back. In mature programmes, both metrics should be tracked together rather than treated as substitutes for one another.
Why the Two Metrics Need to Be Read Together
Reporting rate is especially important when the goal is to reduce dwell time and improve incident intake. If users do not report suspicious messages, security teams lose an early warning channel even when click rates look acceptable. That can leave phishing, impersonation, and credential-harvest attempts undiscovered until a later control catches them.
Failure rate, by contrast, is the better indicator of how persuasive the lure was and how much direct exposure the simulation created. It is useful for comparing campaigns, user groups, and training changes, but it should not be interpreted as a full measure of awareness maturity on its own. A team that reports well but occasionally fails a harder test may be stronger than one that rarely fails but never reports anything.
What Good Measurement Looks Like in Practice
Use consistent definitions before you compare results over time. Decide whether “failure” includes only clicks or also attachment opens, form submissions, token reuse, or other risky actions, and define what counts as a valid report. If the rules change between campaigns, trend lines become misleading and the programme can appear better or worse than it really is.
It is also worth separating individual performance from programme performance. A high reporting rate can reflect strong user behaviour, but it can also reflect a simple, well-designed simulation or a recently announced awareness push. Likewise, a spike in failure rate may indicate a more believable lure, a weaker audience segment, or a simulation that was too close to a real business process to be fair.
Risk and Threat Considerations
Phishing simulations are not only a training metric, they are a proxy for how exposed the organisation is to social engineering, credential theft, and delayed detection. Poor reporting creates a blind spot because suspicious messages can sit in inboxes without reaching the people who can contain them quickly.
Failure mechanism: Users take the simulated bad action, or they ignore the message entirely, which leaves the organisation without useful behavioural signal and can mask where training or reporting channels are weak.
Impact: Low reporting plus high failure can increase the chance that a real phish progresses to account compromise, lateral abuse, or broader incident response workload before it is recognised.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5, CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AT-2 — Awareness Training | Phishing simulations measure whether awareness training changes user behavior. |
| Recommendation — Use awareness training outcomes to reduce phishing susceptibility and improve reporting habits. | ||
| CIS Controls v8 | CIS-14 — Security Awareness and Skills Training | Phishing simulation metrics are commonly used to assess awareness and user response. |
| Recommendation — Measure simulated-phish results to target awareness training and reporting improvement. | ||
| NIST CSF 2.0 | PR.AT-01 — All Users Are Informed and Trained | The metrics assess whether users are trained to recognize and respond to phishing. |
| DE.CM-09 — Personnel are Trained to Recognize and Report Incidents | Reporting rate directly reflects whether personnel can spot and escalate suspicious messages. | |
| Recommendation — Track user training effectiveness with phishing simulation failure and reporting outcomes. Validate that users can recognize and report suspicious emails quickly. | ||
Practitioner Guidance
What to prioritise: Treat reporting rate as the faster operational indicator when you want earlier warning, and treat failure rate as the stronger indicator of susceptibility. If one improves while the other stalls, do not assume the programme is healthy yet.
What to verify: Confirm that the simulation rules, report mechanism, and scoring logic are stable across campaigns. The most common mistake is comparing a click-based failure rate from one exercise with a broader failure definition from another and drawing the wrong conclusion.
What good looks like: Users should both avoid the bait and know how to report it quickly, with enough consistency that suspicious messages reach security before they spread. The best result is not zero failures alone, but a combination of low failure and rapid, reliable reporting.
Practitioner takeaway: Use failure rate to understand exposure and reporting rate to understand response readiness; the gap between them is often more informative than either metric by itself.
Related resources from NHI Mgmt Group
- What is the difference between reporting rate and reporting accuracy in phishing defense?
- What is the difference between attack surface management and NHI governance?
- What is the difference between reviewing human access and reviewing NHIs?
- What is the difference between role-based access and API key governance for NHI security?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 25, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
