What happens when cyber risk metrics cannot be defended?

Why Defensibility Matters More Than the Metric Itself

Cyber risk metrics only help when the audience can trace them back to the underlying asset inventory, control evidence, and calculation method. If that chain is weak, the metric becomes a presentation artifact rather than a decision input. The immediate symptom is not just disagreement over a number, but loss of trust in the reporting process that produced it.

That is why defensibility is a requirement, not a nice-to-have. A metric that cannot be explained in plain language, reproduced from source data, or tied to a defined scope will often be challenged before anyone debates whether the number is “good” or “bad”.

When the metric is used for board reporting, the problem widens because the metric is now carrying governance weight. NHIMG’s Board Cyber Risk Reporting Guide is useful here because it frames reporting around appetite, tolerances, and decision quality rather than isolated figures.

What Breaks When the Metric Cannot Be Explained

The first break is methodological. If the scope is unclear, the metric may mix incompatible assets, controls, or business units, which makes trend comparisons unreliable. If the formula is opaque, the same inputs can be interpreted differently by risk, audit, and control teams, so the number stops behaving like a control signal.

The second break is organisational. Once auditors or executives cannot see how the metric was built, they usually ask for source evidence, revised assumptions, or a narrower report. That creates rework and often exposes that the metric was never anchored to a repeatable process in the first place.

For practitioners building leading indicators, NHIMG’s Key Risk Indicators for Cyber Risk Guide is relevant because it focuses on deriving metrics from tolerance, thresholds, and repeatable logic rather than intuition.

How Defensibility Fails in Practice

Defensibility usually fails in one of three ways: the underlying data is incomplete, the scope does not match the business question, or the method cannot be repeated by someone else using the same evidence set. Any one of those failures can make a metric look precise while still being unreliable.

A common pattern is using a metric that sounds mature, but cannot survive a request to show the population covered, the control evidence sampled, and the formula used to roll the inputs together. Another pattern is metric drift, where the calculation changes over time but the label stays the same, making trend lines misleading.

Where the report is meant to inform leadership decisions, the issue is amplified. Board Cyber Report Template helps because it forces the reporting pack to separate posture, incidents, and decisions needed, which reduces the chance that an unsupported metric is mistaken for a decision-grade signal.

Risk and Threat Considerations

When cyber risk metrics cannot be defended, the immediate risk is governance failure: decision-makers may act on a number that does not accurately reflect exposure, while control owners may reject the result because they cannot verify the inputs or scope. Over time, that weakens the reporting function itself and can create blind spots in prioritisation, assurance, and escalation.

Failure mechanism: Incomplete inventory, weak evidence linkage, or an unrepeatable formula creates a metric that cannot be reconciled during audit, challenge, or incident review.

Impact: The metric loses credibility, reporting is reworked, and management decisions may be based on a signal that is not trustworthy enough to carry governance weight.

Practitioner Guidance

What to verify: Before publishing a cyber risk metric, verify that every input can be traced to a named source, the scope statement matches the asset population, and the formula can be rerun by someone else without special knowledge.

Decision rule: If you cannot defend the data lineage, do not frame the output as a risk metric for executives or auditors; treat it as an internal draft until the evidence chain is complete.

What good looks like: A defensible metric has a stable definition, a documented calculation, a clearly bounded population, and a reviewer can reproduce the result without negotiation about hidden assumptions.

Practitioner takeaway: The standard is not whether the number is interesting, but whether an informed challenger can reproduce it, test it, and agree that it measures the intended risk.

Related resources from NHI Mgmt Group

• Why do completion metrics fail as a measure of human cyber risk?
• When should organisations prioritise cyber risk scoring over broad security metrics?
• What breaks when organisations cannot visualise risk exposure and blast radius during a cyber incident?
• What happens when CISOs and boards do not have a shared view of cyber risk?