Human risk dashboards fail when activity replaces behaviour change

At a glance

What this is: NHI Mgmt Group's analysis, drawing on MetaCompliance's blog post: Human cyber risk dashboards overstate progress when they track activity more reliably than behaviour change.

Why it matters: Security leaders need metrics that connect awareness activity to reduced exposure, because completion rates and click rates alone cannot show whether risk is falling.

By the numbers:

• 74% of CISOs say their current human cyber risk reporting produces dashboards, but not enough of a clear understanding to make better decisions.
• 89% can’t confidently link security awareness activity to reductions in incidents or near misses.
• 70% of CISOs say their current security awareness reporting relies on activity metrics such as training completions or phishing click rates.
• 77% of the CISOs surveyed say they’re expected to prove ROI more rigorously for human cyber risk initiatives than for technical controls.

Context

A human risk dashboard is only useful when it connects observed behaviour to a defensible change in exposure. Counting completions, clicks and reports can support programme administration, but it does not by itself explain whether people are safer or simply more measured.

The governance gap is measurement quality: many programmes still optimise for visible activity because it is easy to collect and present, while the harder task is proving whether those signals translate into lower risk. That tension is especially important where awareness data feeds board reporting, intervention design and workforce-level prioritisation.

Key questions

Q: How do I measure human cyber risk beyond training completions?

A: Use a mix of behavioural and activity signals, then look for change over time rather than a single score. Completion rates tell you whether people attended training, but they do not show whether behaviour changed. Combine reporting behaviour, simulation outcomes, intervention response and group-level trends to understand where exposure is actually falling.

Q: Why do click rates give a misleading picture of phishing risk?

A: Click rates compress very different outcomes into one number, so they hide privilege, context, and whether the user reported the message. A low click rate can still coexist with weak reporting habits or dangerous failures by highly privileged users. Better metrics combine clicks, reporting, credential submission, and role sensitivity.

Q: How should security teams report awareness outcomes to the board?

A: Report awareness outcomes in terms the board can govern: completion rates, phishing susceptibility, reporting rates, repeat-failure trends, and how interventions changed behaviour. Separate participation data from control effectiveness, and tie results to measurable actions such as MFA enrolment, simulated phishing, or targeted coaching. Include residual risk, confidence limits, and any groups that still need remediation.

Q: What is the difference between activity metrics and behaviour metrics?

A: Activity metrics count what people did, such as training attendance, email opens, or click rates. Behaviour metrics show whether that activity changed real security behaviour, for example fewer phishing clicks, faster reporting, stronger MFA use, or reduced policy violations. Activity measures volume, while behaviour measures the security outcome.

Technical breakdown

Why activity metrics are a weak proxy for human risk

Activity metrics record whether a task happened, not whether risk changed. Training completion, simulation opens and click rates are useful telemetry, but each is a point-in-time signal that loses context when treated as a performance score. A high completion rate can coexist with poor judgment under pressure, while a single click can reflect role, timing, fatigue or the realism of the lure rather than a stable risk pattern. The technical problem is one of construct validity: the metric measures participation more reliably than susceptibility, resilience or behaviour change. See the Human Risk Management Programme Guide.

Practical implication: Use activity metrics as inputs to a broader risk model, not as the headline measure of programme success.

How behavioural context changes phishing metrics

Phishing simulation results become materially more useful when they are joined to behavioural context. Difficulty level, audience relevance, reporting after the click, prior interventions and subsequent performance all alter what a click rate means. Without that context, a dashboard can flatten different behaviours into the same percentage and hide whether the issue is awareness, workload, role-specific susceptibility or repeated non-response. Human risk measurement needs sequence and comparison, not just isolated events, because people change over time and different groups face different threat patterns. See the Phishing Simulation Programme Guide.

Practical implication: Treat phishing outcomes as time-series evidence and segment them by role, campaign type and follow-up response.

Why board reporting needs risk concentration, not raw counts

Boards and senior leaders need a view of where exposure is concentrated and whether interventions are shifting that exposure. Raw counts of completions or reports tell them activity volumes, but not which groups remain most at risk or which controls are improving outcomes. The reporting challenge is therefore analytical, not just visual: security teams must translate operational telemetry into decision-grade insight that supports prioritisation. That usually means comparing groups, tracking change after interventions and identifying where risk persists despite programme activity. See the Board Cyber Risk Reporting Guide.

Practical implication: Build board reporting around concentration, movement and intervention effect rather than programme volume alone.

Threat narrative

Attacker objective: The attacker wants to turn a moment of human trust into durable access, fraud, or lateral movement into higher-value systems.

1. Entry begins with a convincing phishing message or other social engineering lure that reaches an employee in a normal workflow. The lure is designed to trigger a user action, such as opening content, clicking a link or submitting data.
2. Credential access follows if the lure captures credentials, session tokens or other account data, or if the user action creates a foothold for follow-on abuse. The attacker then uses that trust event as the pivot from contact to compromise.
3. Escalation occurs when the stolen access is reused against higher-value systems, privileged functions or downstream business processes. The impact is not the click itself but the operational and financial harm that follows from abused trust.

Breaches seen in the wild

• Mailchimp breach 2022: Attackers socially engineered Mailchimp staff, used a support tool to export 102 customer lists and exposed customer API keys for phishing.
• Solana web3.js npm compromise 2024: A phished maintainer's npm access let attackers publish two @solana/web3.js versions that stole private keys from bots and apps.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Human risk measurement fails when programmes mistake observability for control: dashboards can be full of activity data and still provide no decision-grade view of exposure. The problem is not that the signals are useless, but that they are often interpreted as evidence of reduced risk when they are really evidence of completed tasks. Mature reporting has to separate participation from behaviour change, otherwise the dashboard becomes a compliance artefact rather than a security instrument.

Phishing click rates create a false precision effect: they are easy to quantify, which makes them attractive for reporting, but they do not reliably map to risk concentration. Context such as role, lure difficulty, reporting behaviour and prior intervention determines whether a click is a one-off event or part of a repeatable pattern. Practitioners should treat click rates as one feature in a broader risk model, not as a verdict on employee resilience.

Risk dashboards should answer where intervention is needed, not how busy the programme was: the useful question is which groups, behaviours and patterns remain exposed after awareness activity. That shifts human cyber risk reporting from a retrospective scorecard to a prioritisation tool, which is the only form of reporting that changes security decisions.

Behavioural telemetry is becoming the named concept behind credible human risk governance: the article points to a shift from counting activity to correlating signals over time. That matters because security leaders need evidence of movement, not just attendance. Governance that cannot distinguish between programme output and risk reduction will continue to struggle in board reporting and resource allocation.

Human cyber risk reporting has to be evidence of change, not evidence of administration: the strongest programmes will show how exposure shifts after targeted intervention. That gives security leaders a defensible basis for escalation, funding and control design, instead of forcing them to infer risk from participation alone.

What this signals

Behavioural telemetry: human risk programmes are moving toward correlated evidence of change rather than isolated activity counts. That means the measurement model has to join participation, response and follow-up into one view if it is going to support prioritisation and board-level decisions.

Dashboards that cannot distinguish programme output from exposure reduction will keep producing comforting numbers with limited operational value. The next step for practitioners is to make reporting decision-led, so the metrics point to where intervention is needed and whether it worked.

For practitioners

• Build a behaviour-linked dashboard Combine training completion, simulation behaviour, reporting rates and follow-up outcomes so the dashboard shows whether risk is moving, not just whether activity occurred.
• Segment reporting by risk group Break metrics out by role, business unit, geography and prior behaviour so repeated patterns are visible and interventions can be targeted where exposure concentrates.
• Track post-intervention change Measure whether a group’s behaviour improves after training or a simulation campaign, then compare that trend with its previous baseline before treating the intervention as effective.
• Use simulation context in analysis Record campaign difficulty, relevance and reporting behaviour alongside click data so a single percentage is not mistaken for a stable risk score.

Key takeaways

• Human cyber risk dashboards lose value when they optimise for visible activity instead of showing whether behaviour changed.
• The strongest evidence in this topic is the gap between programme output and decision-grade risk reduction, not any single awareness metric.
• Security teams need reporting that identifies concentration, movement and intervention effect so they can act on exposure rather than just report on participation.

Related NHI Mgmt Group resources: Human Risk Management Programme Guide · Phishing Simulation Programme Guide · Board Cyber Risk Reporting Guide · Security Awareness and Human Risk Management Guide

Key terms

• Human Risk Dashboard: A human risk dashboard is a management view that combines workforce behavior signals and security indicators into one place. It helps leaders see where risk is concentrated, how it changes over time, and which groups may need attention. The value is not reporting alone, but supporting targeted action and clearer accountability.
• Phishing Simulation: Phishing simulation is a controlled exercise that sends realistic but harmless lure messages to users to measure how they respond to credential theft attempts. In IAM and security operations, it tests awareness, reports risk patterns, and helps tune controls, training, and incident response for human and NHI attack paths.
• Behavior Change: Behavior change is any measurable shift in how a person, system, or AI agent acts over time. In identity security, it usually refers to deviations from normal access, authentication, or workload activity that may signal compromise, misuse, policy drift, or the need for adaptive controls and investigation.
• Security Awareness Reporting: The process of turning a user’s suspicion of a scam, phishing message, or suspicious event into a response signal for security teams. Effective reporting is operational, not educational, because it can speed containment, investigation, and notification when identity abuse is underway.

Deepen your knowledge

The NHI Foundation Level course covers NHI governance, human identity, and secrets management through the industry's only accredited NHI security programme. It helps practitioners connect identity governance to the controls and reporting their programmes depend on.