TL;DR: Blocking risky user actions is a poor success metric on its own, according to Panaseer, because the more useful signal is whether blocks decline while the organisation still maintains a full block rate. That shifts measurement from raw volume to education, trend context and programme improvement.
At a glance
What this is: NHI Mgmt Group's analysis, drawing on Panaseer's blog post: Block-rate metrics are more useful as a measure of user education and programme improvement than as a simple tally of preventive actions.
Why it matters: Security and IAM teams need metrics that show behaviour change, control effectiveness and business context, not just tool output that looks impressive in isolation.
Context
Security teams often mistake intervention volume for programme maturity. A high count of blocked clicks, downloads or emails may show that controls are active, but it does not tell you whether users are learning, whether risky behaviour is falling, or whether the control is merely compensating for weak awareness.
The governance gap is measurement design: many programmes track the event without tracking the trend, the business unit, or the behavioural outcome. That matters for security metrics programmes because the same raw number can mean very different things depending on whether it is rising because threats are increasing or falling because education is working.
Key questions
Q: How do I measure whether security awareness is reducing risky user behaviour?
A: Measure change in risky actions, not course completion. Track phishing-report rate, credential-reuse events, policy violations, and privileged access mistakes before and after training, then compare by user group and time period. Use a stable baseline, consistent definitions, and control for exposure such as phishing volume or role changes.
Q: Why do block counts not prove that a security programme is effective?
A: Block counts only show how many attempts were stopped, not whether users changed behaviour or the programme reduced risk. A rise in blocks can mean more attacks, a stricter filter, or better detection, so compare the metric over time alongside exposure, report rates, and user error rates.
Q: How does reporting a phish help defenders more than simply not clicking?
A: Reporting a phish gives defenders indicators such as sender details, URLs, attachment hashes, and mailbox telemetry, which improves triage and containment. Not clicking only avoids one outcome for one user, but reporting can trigger blocking, hunting, and wider user warnings before other accounts are affected.
Q: What is the best way to use continuous controls monitoring for user behaviour metrics?
A: Use continuous controls monitoring to collect the same user behaviour metric repeatedly, automatically, and against a defined control objective. Feed access logs, email events, and policy exceptions into a baseline so deviations, not single counts, drive review. Pair it with thresholds, alerting, and manual validation for false positives.
Technical breakdown
Why block counts are a weak standalone signal
A block count measures control activity, not control effectiveness. If a firewall, secure email gateway or endpoint policy is stopping unsafe clicks, the number tells you that prevention is occurring, but not whether the underlying behaviour has changed. Without a baseline, a trend line or a tolerance threshold, the metric can be misread as success even when users continue to generate the same risky intent. The right interpretation depends on whether the programme wants to reduce exposure, reduce repeat behaviour, or simply demonstrate enforcement. See the Security Metrics and KPIs Programme Guide.
Practical implication: Measure block volumes alongside trend direction, not as a single-period headline.
Why phishing reporting is a stronger maturity indicator
A reported phish is more valuable than a mere non-click because it shows the user recognised the threat and chose the right response. Binary click metrics collapse too much information into good or bad, while report rates reveal whether the workforce is turning awareness into action. That distinction matters because security education aims to change decisions, not just suppress interaction. When reporting rises and click rates fall, the organisation is seeing a more mature human control surface.
Practical implication: Track report rate as a positive behaviour metric, not just click suppression.
How continuous controls monitoring gives security metrics context
Continuous controls monitoring turns isolated events into governed measurement by aggregating data from multiple tools, comparing it to an SLA or tolerance range, and showing movement over time. This makes the metric operationally meaningful because leaders can see where blocks occur, which populations are driving them, and whether a control change causes the expected spike before the curve settles. The result is a feedback loop between policy, training and observed behaviour.
Practical implication: Use continuous measurement to separate a one-off spike from a sustained improvement.
NHI Mgmt Group analysis
Block volume is an output, not an outcome. The fact that a control stops more events does not prove the organisation is safer if the same behaviour continues underneath it. Mature measurement has to distinguish enforcement activity from behaviour change, otherwise security teams end up optimising for visible friction rather than reduced risk. The practitioner conclusion is to treat block counts as supporting evidence, not the primary success measure.
Behavioural controls need a trend-based baseline: A single reporting period cannot show whether awareness is improving or whether the environment simply became noisier. The meaningful question is whether the same control produces fewer interventions over time without reducing protection. That is why measurement programmes need historical comparison, segment-level views and clear thresholds before they can support executive decisions. The practitioner conclusion is to measure change, not snapshots.
Human awareness is part of the control plane: In this domain, users are not just subjects of policy; they are part of how the control works. Reporting, safe decision-making and refusal to click are all signals that the organisation is shifting risk left into behaviour. That makes the human layer a governance issue, not just a training issue. The practitioner conclusion is to manage awareness as an operational control with evidence, ownership and targets.
Continuous monitoring closes the gap between policy intent and observed behaviour. Tool output alone rarely shows whether a policy change actually altered risk, but time-series measurement can reveal when education, tuning or stricter blocking begins to work. That makes continuous controls monitoring useful for justifying investment and for spotting where the programme still depends on repeated intervention. The practitioner conclusion is to pair every control change with a measurable outcome.
What this signals
Behaviour change is the real control objective: security teams should expect the number of blocks to fall only when the underlying behaviour improves. That makes the programme dependent on measurement design that captures trend, segment and outcome rather than celebrating tool activity.
The next maturity step is to treat user-awareness data like any other control signal: define a baseline, watch for drift and separate genuine improvement from one-off spikes. That approach helps leaders defend investment decisions without confusing friction with resilience.
For practitioners
- Set block-rate baselines by behaviour type Separate risky clicks, downloads and sent emails into distinct measures, then track each against a pre-change baseline so improvement means fewer interventions over time.
- Promote reporting as the preferred positive action Measure and reward users who report suspicious messages, because report behaviour shows awareness and gives a better maturity signal than click suppression alone.
- Compare metrics to the previous period Show every security behaviour measure against yesterday, last week, last month or last quarter so leaders can see whether the curve is improving, flat or deteriorating.
- Segment results by department and role Break outcomes down by business unit and seniority to find where awareness training, policy tuning or control tightening still needs targeted improvement.
- Tie control changes to measurable effects Whenever you tighten a filter or launch training, watch for the expected short-term spike and the later decline so you can prove the intervention changed behaviour.
Key takeaways
- Block-rate metrics are only meaningful when they are read as behavioural indicators, not as a simple tally of prevented actions.
- A falling number of blocks can be a better sign than a rising one if the organisation still maintains complete protection and users are learning.
- Continuous monitoring, baselines and segmentation are what turn security education into evidence that leaders can defend.
Related NHI Mgmt Group resources: Security Metrics and KPIs Programme Guide · Email Security Guide · Information Security Policy Template
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0, CIS Controls v8 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OV-03 — Oversight of Results | Trend-based measurement and stakeholder reporting are central to this metrics discussion. |
| ID.AM-01 — Physical devices and systems are inventoried | The dashboard approach depends on knowing where events occur across the organisation. | |
| PR.AT-01 — Personnel are provided awareness and training | The core argument is that blocking metrics can indicate whether awareness has improved. | |
| Recommendation — Tie security behaviour metrics to oversight reviews so leaders can judge whether controls improve outcomes over time. Map block events to business locations and user groups so metrics reflect the environment being governed. Use awareness results to measure whether training reduces risky user behaviour. | ||
| CIS Controls v8 | CIS-8 — Audit Log Management | Continuous monitoring relies on telemetry and time-series evidence from multiple tools. |
| Recommendation — Centralise event data so you can compare behaviour measures against baselines and thresholds. | ||
| NIST SP 800-53 Rev 5 | AU-6 — Audit Record Review, Analysis, and Reporting | The article is about interpreting security events as useful management evidence. |
| Recommendation — Review behaviour metrics regularly and report them in trend form rather than as isolated counts. | ||
Key terms
- Block Rate: Block rate is the percentage of transactions or login attempts a fraud system prevents or rejects. It is a practical control metric, but it only has value when paired with false positive and customer friction analysis. A rising block rate can mean stronger detection, more attack pressure, or overly aggressive tuning.
- Trend Baseline: A trend baseline is the normal pattern of a metric over time. It is built from repeated measurements so you can compare current values against expected variation and spot meaningful change in security, operations, or user behaviour.
- Continuous Controls Monitoring: Continuous controls monitoring is the ongoing evaluation of transactions, access, and configuration changes against policy rules. It replaces occasional sample testing with near-real-time detection, which gives security, audit, and finance teams faster evidence and a better chance to correct drift before it becomes a finding.
- Security Awareness Training: Security Awareness Training is the practice of teaching people how to recognize and respond to security risks in daily work. It covers phishing, password hygiene, data handling, social engineering, device safety, and reporting procedures. Effective training changes behavior, supports policy compliance, and reduces human error as an attack path.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and secrets management. It helps practitioners connect identity controls to measurable security outcomes across their programme.
Published by the NHIMG editorial team on November 9, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org