The clearest signs are overreliance on course completion, lack of access context, weak linkage to security events, and no evidence that risky behaviour is falling over time. If the platform cannot show which behaviours are changing, the programme is probably measuring participation rather than reducing exposure.
How to tell when a human-risk programme is only counting training, not changing behaviour
A shallow programme tends to measure attendance, completion, or quiz scores while leaving the real control problem untouched: whether people behave differently in the situations that matter. The deeper test is whether the programme can show fewer risky actions, better decisions under pressure, and measurable change in the behaviours tied to incidents, access, or data exposure.
One sign is that the programme is detached from the operational context where risk actually appears. If everyone gets the same content regardless of role, system access, or exposure, then the programme may be efficient to administer but weak as a control. It should reflect where risk concentrates, not just where communication is easiest.
Another sign is that the programme cannot connect learning activity to observable security outcomes. Completion data alone does not show whether phishing resistance improved, whether sensitive-data handling changed, or whether risky shortcuts declined. A useful programme creates a traceable line from intervention to changed behaviour and then to lower exposure.
A third sign is that it treats human risk as static. If the same topics are recycled without evidence that the underlying behaviour has shifted, the programme is probably running on habit rather than feedback. Mature programmes use incident trends, policy exceptions, and behaviour telemetry to decide what to reinforce, retire, or redesign.
What shallow human-risk programmes usually get wrong
Shallow programmes often over-index on awareness because it is easy to report and easy to scale. That creates a false sense of control: the organisation can show that people were exposed to content, but not that the right people changed the right behaviours in the right scenarios.
They also miss the difference between generic education and targeted risk reduction. A broad campaign can raise baseline understanding, but it does not necessarily reduce the actions that create exposure. If the programme cannot distinguish between broad familiarity and actual behaviour change, it is probably managing optics rather than risk.
Shallow programmes usually underuse evidence from NIST Cybersecurity Framework 2.0 style governance because they do not translate human-risk work into measurable outcomes, and they often fail to align with control evidence that security teams can inspect. That gap matters because human-risk work should support detection, prevention, and response, not sit beside them as a separate communications exercise.
Where the organisation wants a stronger behaviour-control model, the programme should be anchored to operating conditions such as access level, data sensitivity, and incident patterns. For that reason, the same underlying logic seen in NIST AI Risk Management Framework governance, namely measurable risk treatment and feedback loops, is useful as a design mindset even when the subject is human risk rather than AI.
A programme is also shallow when it cannot explain why behaviour changed. If the only success metric is a dashboard of completions, there is no way to tell whether the intervention worked because of the content, the manager conversation, the policy change, or simple repeat exposure. Without that attribution, improvement claims are weak.
What practitioners should measure instead of participation
Measure the behaviours that create exposure, then tie them to the business or security events they influence. That may include click-through on suspicious content, reporting latency, policy exceptions, unsafe data handling, excessive privilege requests, or repeated control bypasses. The point is not to measure everything, but to measure the few behaviours that are clearly linked to risk.
Use trend evidence, not snapshots. A single campaign can produce a temporary dip in one metric, but shallow programmes often fail to sustain change once attention fades. Useful measurement shows direction over time, segment by segment, so you can see whether high-risk groups are improving or merely being reminded.
Where relevant, pair behavioural metrics with event data from MITRE ATT&CK Enterprise Matrix so you can judge whether human behaviour is actually reducing attack opportunity. That connection is especially helpful when the programme claims to reduce credential theft, phishing success, or social engineering impact.
If the programme covers access-sensitive tasks, verify whether it changes decisions in context, not just recall after the fact. A person who can answer a quiz question is not necessarily safer when prompted in a live workflow. Good measurement therefore looks for reduced risky action at the moment of choice.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.RM-01 — Risk Management Strategy | Human-risk programmes should show measurable risk reduction, not just participation. |
| Recommendation — Tie human-risk metrics to risk outcomes and review them in the governance cycle. | ||
| MITRE ATT&CK | T1566 — Phishing | Behaviour change is often judged by exposure to phishing and social-engineering outcomes. |
| Recommendation — Map training goals to phishing-related techniques and monitor attack-impact trends. | ||
| NIST AI RMF | MEASURE — Measure and Evaluate | The question is about whether interventions are producing measurable change over time. |
| Recommendation — Define behaviour metrics and evaluate whether interventions actually reduce exposure. | ||
Practitioner Guidance
What to verify: Ask whether each major intervention has a linked behaviour, a defined audience, and a measurable outcome. If the programme cannot name the behaviour it is trying to change, it is probably too abstract to be effective.
Decision rule: If the evidence stops at completion or awareness scores, treat the programme as immature and redesign it around exposure-relevant behaviours, not course volume. If you can show behavioural change plus fewer incidents or exceptions, the programme has moved beyond superficial reporting.
What practitioners underestimate: The hardest part is not delivering content, it is proving that the organisation did something differently because of it. Human-risk work becomes credible when it can withstand the question, “What changed in the environment, and how do we know?”
Practitioner takeaway: A human-risk programme is shallow when it can report attendance but not demonstrate behaviour change, because participation is a communications metric while risk reduction is an operational one.
Related resources from NHI Mgmt Group
- What are the signs that a third-party risk management programme is too shallow to detect real exposure?
- Why do non-human identities create more audit risk than human accounts?
- Why do non-human identities create audit risk in modern environments?
- Why do non-human identities create compliance risk even when policies exist?