Look for movement in behavior-based metrics, not just phishing click rates. A useful programme shows that risky users are becoming less risky, that top issues are being addressed, and that the organisation is trending toward stronger vigilance over time. If the only numbers are completion rates, the programme may be busy but not effective.
What to Measure Beyond Completion Rates
A security awareness programme is improving risk when the metrics change the way people behave, not just whether they show up. The useful signals are fewer repeated risky actions, better reporting of suspicious activity, and measurable movement in the population that previously accounted for most incidents. Completion can support delivery, but it is not evidence of reduced exposure on its own.
To tell whether the programme is actually working, focus on behaviour-based measures that are tied to real control outcomes. That includes phishing simulation trends, help desk and reporting behaviour, policy violations, repeat offender rates, and the time it takes people to respond correctly to risky prompts or warnings. Where those measures improve together, the programme is starting to affect risk rather than just awareness.
- Ultimate Guide to NHIs — What are Non-Human Identities is useful as a benchmark for behaviour-based governance thinking, especially where identity-related risk must be measured through actual lifecycle and access outcomes rather than attendance-style metrics.
How to Read Real Improvement Signals
The strongest sign is trend, not a single campaign result. If high-risk groups are improving faster than the average user, the programme is reducing concentration of risk where it matters most. If top recurring issues are falling quarter by quarter, that suggests the training, nudges, and reporting paths are addressing actual failure modes instead of producing generic compliance activity.
Good programmes also improve vigilance in ways that are visible outside the training platform. For example, users may report more suspicious messages earlier, make fewer avoidable policy exceptions, or show better judgment when faced with unusual requests. Those outcomes are more meaningful than a one-time drop in click rates because they indicate that the organisation is becoming harder to trick and faster to respond.
One useful indicator is whether the same weak behaviours reappear after reinforcement. If they do, the programme may be creating short-lived test effects rather than durable habit change. If they do not, and the organisation can show that the high-risk behaviours are shrinking over time, the programme is contributing to resilience rather than simply generating awareness content.
Risk and Threat Considerations
The main risk is mistaking participation for protection. A programme can look successful on dashboards while the underlying exposure remains unchanged, especially if it rewards course completion, generic quiz scores, or isolated phishing results that do not connect to real-world behaviour.
Failure mechanism: weak measurement design can hide repeat risk, because the same users may keep making the same mistakes, or staff may learn to pass simulations without changing how they handle suspicious requests, reporting, or policy exceptions.
Impact: the organisation keeps investing in a programme that is busy but not effective, and attackers still benefit from the same human failure modes, delayed reporting, and inconsistent escalation behaviour.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.AT — Awareness and Training | Awareness must improve user behavior and response, not just completion. |
| Recommendation — Measure training against behavior change, reporting quality, and repeat-risk reduction. | ||
| CIS Controls v8 | 14 — Security Awareness and Skills Training | This control family focuses on training outcomes and reinforcement of secure behavior. |
| Recommendation — Track awareness with outcome metrics that show reduced risky actions over time. | ||
Practitioner Guidance
What to verify: Check whether your reporting trend, repeat-issue trend, and high-risk-user trend all move in the right direction together. If only completion rises, treat that as delivery evidence, not risk reduction evidence.
What good looks like: The people who previously generated the most security friction should show fewer repeat errors, faster escalation of suspicious events, and fewer policy exceptions over time. If the same small group remains static, the programme probably needs targeted intervention rather than more general awareness content.
Decision rule: If a metric does not map to a security outcome or a behaviour change, do not use it as proof of improvement. Prioritise measures that show whether people are becoming easier to defend, faster to alert, and less likely to repeat the same risky action.
Practitioner takeaway: A real awareness programme changes observable behaviour in the population that creates risk, and it does so persistently enough that the improvement holds up outside the training exercise.
Related resources from NHI Mgmt Group
- What are the signs that a security awareness programme is failing to reduce cyber risk?
- How should security teams build a third-party risk programme that actually reduces identity risk?
- How can organisations tell whether their data security programme is actually improving?
- How should security teams build a patch compliance programme that actually reduces risk?
