Risk scores are replacing black-box cyber underwriting

At a glance

What this is: Cowbell says cyber risk scoring should function as a transparent, peer-relative measure rather than a static grade or black-box judgment.

Why it matters: That matters to IAM and security practitioners because trust decisions increasingly depend on explainable signals, context, and evidence rather than a single score that hides its inputs.

Context

Cyber risk scoring becomes a governance problem when the number is treated as a verdict instead of an evidence-backed signal. In insurance, security, and identity programmes, opaque scoring weakens trust because practitioners cannot see how the inputs, peer set, or assumptions shaped the outcome.

The article is really about explainability, not just AI. When a score is used to influence access, pricing, or control decisions, teams need to understand whether the model is measuring the right population and whether the score changes when context changes.

Key questions

Q: How should security teams use risk scores in identity governance?

A: Security teams should treat risk scores as decision inputs, not dashboard metrics. The practical move is to attach them to policy conditions so approvals, access requests, and revocations can respond when identity state changes. That turns risk into an enforcement signal and reduces the delay between detection and control action.

Q: When does a black-box score become a governance problem?

A: It becomes a governance problem when people must act on it but cannot see how it was produced. If a score affects access, pricing, remediation, or risk acceptance, then explainability and challengeability are part of the control, not optional extras.

Q: What do security teams get wrong about high CVSS scores?

A: They often treat CVSS as a complete ranking signal. In practice, a lower-scored issue can be more dangerous if it is reachable, chained to other weaknesses, or linked to exposed credentials. Teams should look at real attack paths, not just the score attached to an individual finding.

Q: Should organisations combine technical posture and human-risk indicators in one view?

A: Yes, when the score is used for decisions that affect exposure or prioritisation. Technical controls alone rarely capture the ways people create risk, so combining the two can produce a more realistic picture. The key is to keep the model transparent enough that users can understand the trade-off.

Technical breakdown

How relative risk scoring differs from a static rating

A relative risk score compares an organisation against a peer group rather than pretending there is one universal pass mark. That matters because the same security posture can look acceptable in one sector and weak in another, depending on threat exposure, digital footprint, and control maturity. The governance issue is not the arithmetic itself but the choice of reference population and the assumptions embedded in it. Without that context, stakeholders read the score as a fixed grade and make bad decisions from it. Practical implication: define the peer group, the inputs, and the conditions under which a score should be reinterpreted.

Practical implication: define the peer group, the inputs, and the conditions under which a score should be reinterpreted.

Why confidence metrics matter in model-driven underwriting

Confidence metrics tell users how certain the system is about a score, which is different from the score itself. In cyber underwriting, that distinction matters because incomplete telemetry, changing threat conditions, and uneven data quality can all distort the answer while still producing a neat number. Explainability without confidence can still mislead if practitioners do not know whether the model is operating on strong evidence or thin signals. Practical implication: require confidence disclosure whenever a score can affect pricing, risk acceptance, or remediation prioritisation.

Practical implication: require confidence disclosure whenever a score can affect pricing, risk acceptance, or remediation prioritisation.

Why human error still dominates the control gap

The article correctly rejects the idea that technical control strength alone determines risk. Human actions such as clicking a phishing link can bypass strong perimeter controls and turn a high-score environment into an exposed one. That is an identity and governance issue because user behaviour, access patterns, and awareness maturity shape the real attack surface as much as software posture does. Practical implication: treat human-risk signals as part of the control model, not as an external exception to it. See the Security Awareness and Human Risk Management Guide.

Practical implication: treat human-risk signals as part of the control model, not as an external exception to it.

NHI Mgmt Group analysis

Explainability is now a trust control, not a nice-to-have reporting feature. When a score influences underwriting, remediation, or security investment, stakeholders need to see the basis for the decision, not just the conclusion. Black-box outputs create governance debt because they cannot be challenged, calibrated, or explained to the people who must act on them. Practitioners should treat transparency as part of the control environment, not a communications layer.

Relative scoring is only useful if the reference frame is defensible. A score compared against the wrong peer group can be more misleading than no score at all. That is especially important when sector, exposure, or operational footprint changes the threat model. Practitioners should validate how the comparison set is built before they let a score drive prioritisation.

Human risk belongs inside the control model. The article is right that a strong technical posture can still fail through phishing or other user-driven events. That is where identity governance intersects with cyber risk: access, awareness, and user behaviour are part of the exposure surface. Teams should measure human factors alongside technical ones rather than treating them as a separate concern.

Glass-box scoring is a named example of a broader governance pattern: evidence-led risk decisions. The important shift is from opaque judgment to visible, reviewable inputs and confidence levels. That same pattern applies wherever AI mediates access, triage, or trust decisions, including identity and agentic systems. Practitioners should demand decision transparency wherever a model affects security outcomes.

The insurance use case shows that trust increasingly depends on machine-generated evidence that humans can interrogate. That is relevant beyond underwriting because any AI-assisted security workflow will be judged on explainability, not just accuracy. Organisations that cannot explain why a model said “high risk” will struggle to defend the decision when challenged. Practitioners should require auditable reasoning, not just a score.

What this signals

Evidence-led scoring is becoming a governance expectation, not just a product design choice. As more security and insurance decisions depend on machine-generated assessments, the organisation must be able to explain not only the answer but the basis for it. That raises the bar for any workflow that uses AI to shape trust, including identity and access decisions.

Trust gaps widen when the score is detached from operational reality. If the score does not change when the context changes, the organisation is managing a number rather than a risk. Practitioners should watch for scoring models that ignore sector shifts, changing exposure, or human behaviour because those are the places where the trust model breaks first.

For practitioners

• Define the peer set behind every score Document which organisations, sectors, and exposure profiles the model compares against so users know what the score actually means.
• Require confidence disclosure for decision use Do not let a score influence pricing, acceptance, or prioritisation unless the model also surfaces how certain it is about the result.
• Fold human-risk signals into governance reviews Include user behaviour, phishing susceptibility, and awareness indicators when evaluating whether a score reflects real-world exposure. See the Security Awareness and Human Risk Management Guide.
• Test whether the score changes with context Reclassify or rescope a sample case to check whether the model responds appropriately when the operating environment changes.
• Ask for auditable inputs before accepting the output Challenge any model that cannot show the evidence, assumptions, and data sources that produced a materially important risk score.

Key takeaways

• Opaque risk scores create governance problems when practitioners cannot see the peer frame, assumptions, or evidence behind the number.
• The article’s central point is that confidence and context matter as much as the score itself when decisions are being made from it.
• Security teams should treat human behaviour and model transparency as part of the same trust system, not as separate issues.

Related NHI Mgmt Group resources: Identity Security Posture Management (ISPM) Guide · Security Awareness and Human Risk Management Guide · AI Risk Assessment Guide

Key terms

• Risk Score: A risk score is a point-in-time measure of what is currently exposed or misconfigured in an identity environment. In NHI programmes it is useful for remediation, but it does not tell you whether the architecture is becoming safer over time or just clearing today’s backlog.
• Confidence Metric: A confidence metric indicates how certain a model is about its own output. In governance terms, it helps practitioners distinguish a well-supported assessment from one built on thin, incomplete, or noisy data, which is critical when the result affects pricing, access, or remediation priorities.
• Black-box model: A black-box model is an AI system whose internal reasoning cannot be fully observed or explained from the outside. In security operations, that limits auditability and makes it harder to prove whether a decision followed policy, used sound inputs, or should have been overridden.
• Glass Box Approach: A glass box approach exposes enough of the model’s inputs, logic, and certainty to let humans inspect and challenge the result. It does not remove judgment, but it makes the basis for judgment visible, which is essential when AI helps shape trust decisions.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and identity lifecycle fundamentals. It helps security and identity practitioners connect access governance to broader trust decisions across the programme.