Security awareness metrics often hide human risk reduction gaps

At a glance

What this is: Hoxhunt's analysis says security awareness training can look healthy on completion dashboards while real human risk stays flat because the program measures participation instead of exposure.

Why it matters: This matters because IAM and security leaders need behaviour-level evidence to justify human-risk controls, target interventions, and explain why training alone does not lower exposure.

Context

Security awareness training is supposed to reduce human risk, but completion-led programmes often confuse activity with outcomes. When teams measure modules finished, quizzes passed, and simulations sent on schedule, they can miss whether people actually detect, report, and resist attacks in live conditions. The identity connection is indirect but real: human behaviour remains one of the easiest entry points for credential theft, phishing, and account compromise.

The governance gap is measurement design. A programme built around participation tells leaders that training happened, not that exposure declined, and that gap becomes more dangerous when risk concentrates in high-value roles such as finance, IT, executives, and frequent email users. Hoxhunt's argument is that human-risk management has to move from annual awareness reporting to continuous behavioural evidence.

Key questions

Q: What breaks when security awareness programmes rely on completion rates?

A: Completion rates break because they measure attendance, not behaviour change or exposure. A user can finish training and still make the same risky decisions, especially if they have privileged access or are being targeted by active threats. That is why benchmarking has to connect behaviour to identity and threat context before it is treated as a meaningful risk signal.

Q: Why do low phishing failure rates sometimes create false confidence?

A: Low failure rates can hide the much larger group that never meaningfully engaged with the simulation. If users ignore tests, those results may not show learning or resilience, only weak sampling. The risk is especially high when programmes report averages without separating non-response, repeat failure, and role-based exposure. A low number is only useful when it reflects the full population.

Q: How should security teams measure whether human risk management is actually reducing risk?

A: Use outcome metrics, not just participation data. Track behaviour such as phishing reporting, policy exception rates, risky link clicks, and secure workflow adoption by persona or business unit. Then compare those signals against identity and access outcomes so the programme shows whether human behaviour is changing in ways that reduce real exposure.

Q: Why does security awareness belong in identity governance?

A: Because user behaviour is part of access risk. People with valid access still create exposure when they mishandle credentials, ignore policy, or fall for social engineering. Awareness supports identity governance by reducing the likelihood that legitimate accounts become the attack path, especially during onboarding, role changes, and elevated access periods.

Technical breakdown

Why completion metrics overstate security awareness impact

Completion metrics are easy to collect, but they are weak evidence of reduced risk. A user can finish training, pass quizzes, and still fall for a real phishing lure under pressure because the operational context is different from the training context. That creates a metric translation problem: the control measures exposure to content, while the threat exploits judgement in a live mailbox. Continuous behaviour-change programmes are trying to close that gap by measuring what people actually do when confronted with suspicious messages, not just whether they attended the lesson.

Practical implication: Measure behaviour under realistic conditions, not just training attendance.

Why low phishing failure rates can hide untested users

A low click rate often looks reassuring, but it can conceal a large population that neither fails nor reports. In practice, many users simply ignore simulations, which means they were not meaningfully tested at all. That is a governance problem because an apparently improving average can hide the exact people attackers target: high-volume communicators, time-pressured staff, and new or infrequent users. The result is a false reduction narrative built on incomplete sampling of the workforce.

Practical implication: Track reporting, non-response, and repeat exposure, not just click rates.

How exposure metrics reveal concentrated human risk

Exposure metrics focus on how behaviour changes across the workforce and where risk clusters. Instead of asking whether the programme ran, they ask which groups report quickly, who repeatedly clicks, and where failure persists despite training. This is closer to an identity-security control view because it maps human behaviour to account risk, phishing susceptibility, and escalation likelihood. Once teams can see concentration, they can intervene where the blast radius is highest rather than averaging the whole workforce into a comforting but misleading number.

Practical implication: Segment results by role and risk level before deciding whether the programme is working.

Threat narrative

Attacker objective: The attacker objective is to exploit human behaviour to obtain a foothold, credentials, or trust that can be converted into broader compromise.

1. Entry begins when phishing or similar social engineering reaches users who have been trained but not behaviourally hardened. Credential access follows when a recipient clicks, approves, or ignores a malicious prompt instead of reporting it. Escalation occurs when repeat exposure is concentrated in high-value users whose mistakes can lead to account compromise or lateral movement. Impact is reduced visibility of real human risk, which leaves the organisation exposed even when training dashboards look healthy.

Breaches seen in the wild

• Mailchimp breach 2022: Attackers socially engineered Mailchimp staff, used a support tool to export 102 customer lists and exposed customer API keys for phishing.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Activity-led awareness is a measurement trap, not a risk control. When programmes optimise for completions, quiz scores, and campaign throughput, they reward visibility rather than resilience. Human risk only falls when the control proves that employees detect and report threats in realistic conditions. The practitioner conclusion is to stop treating participation as the outcome and start treating behaviour change as the control objective.

Human-risk management is now an identity governance problem as much as a training problem. Phishing, reporting latency, and repeat failure concentrate around the accounts that matter most to the business. That makes the programme part of access-risk reduction, not just education. Security leaders should therefore align awareness metrics with identity risk, because the same user behaviour that drives compromise also drives account exposure.

False confidence emerges when averages are used as the control plane. A strong overall click rate can coexist with repeat clickers, silent non-responders, and high-risk roles that never get enough scrutiny. This creates governance debt: the dashboard improves while the attack surface remains uneven. The practical conclusion is to manage human risk as a distribution problem, not a single score.

Continuous measurement beats annual certification because attackers operate continuously. Annual or quarterly training captures knowledge at a point in time, while adversaries exploit inbox pressure, fatigue, and habitual responses every day. The right model is to measure detection, reporting, and response speed as living signals. Practitioners should use those signals to decide where human-risk controls need intensification, not just where compliance boxes were ticked.

Behaviour-based programmes define a more credible human-risk benchmark. The useful question is no longer whether a programme exists, but whether it changes the probability of failure in the accounts that matter most. That benchmark belongs alongside identity and access governance, because users remain part of the attack path and the control has to prove it can change outcomes. The conclusion for practitioners is straightforward: measure the human layer with the same seriousness you apply to privileged access.

What this signals

Human-risk programmes fail when they are managed as content delivery. The next maturity step is to govern user behaviour as an operational risk signal, not a training completion record. That means shifting executive reporting toward detection speed, reporting speed, and repeat exposure in the accounts that matter most.

Behavioural concentration is the concept teams should operationalise next. Risk is rarely evenly spread across a workforce, so programmes that average everyone together are blind to the pockets that attackers target. Practitioners should segment human-risk results by role and privilege so controls can follow exposure rather than vanity metrics.

For practitioners

• Shift from completion to exposure metrics Replace completion-only reporting with detection rate, reporting rate, time-to-report, repeat failure, and non-response tracking. Use those measures to decide whether the programme is reducing actual risk or only improving participation.
• Segment the workforce by risk concentration Break results out by role, privilege, email volume, and business criticality so that executive assistants, finance teams, and other high-exposure groups do not disappear into averages.
• Use continuous simulations with adaptive difficulty Run regular behaviour checks that adjust difficulty by user performance and reward reporting behaviour immediately so the programme measures real-world detection rather than pattern recognition.
• Treat non-reporters as an active risk signal Investigate users who neither click nor report, because silent non-response can hide the largest blind spot in awareness programmes and often signals unmeasured exposure.
• Tie awareness results to identity and access review Feed human-risk results into access governance for high-value users so persistent failure patterns can inform tighter review, coaching, and control escalation.

Key takeaways

• Security awareness training can produce a polished dashboard without materially lowering human risk.
• The main measurement failure is confusing participation metrics with exposure metrics, which hides silent users and concentrated risk.
• Practitioners should track detection, reporting, speed, and repeat failure at the role level if they want evidence that risk is actually falling.

Key terms

• Exposure Metric: A measure of how people behave when they are actually presented with risk, rather than how many items they completed. It focuses on reporting, detection, response speed, and repeat failure, which makes it far more useful for judging whether awareness activity is changing outcomes.
• Behavior Change Campaign: A behavior change campaign is a structured effort to shift everyday user actions toward safer habits. In cybersecurity, it focuses on a few practical behaviors, such as updating software, backing up data, and using stronger authentication, because people are more likely to adopt clear actions than broad instructions.
• Human Risk Concentration: The uneven clustering of risk in specific users, roles, or teams rather than across the workforce as a whole. It matters because attackers usually target the highest-value or most time-pressured users, so averages can hide the real exposure profile.
• Reporting Latency: Reporting latency is the time between discovering a security issue and being able to notify the right authority with confidence. In regulated environments, long latency often reflects governance gaps in ownership, inventory accuracy, and escalation, not just slow analysis.

Deepen your knowledge

The NHI Foundation Level course covers NHI governance, human identity, secrets management, and agentic AI identity as part of the industry's only accredited NHI security programme. It helps security, IAM, and GRC practitioners connect identity controls to the wider risk signals their programmes depend on.