Precision and provenance are the metrics that make vulnerability data usable

At a glance

What this is: This is an analysis of why vulnerability counts mislead executives and why precision and provenance are better operational metrics.

Why it matters: It matters to security and identity practitioners because unreliable asset and dependency data undermines governance decisions, exposes hidden supply chain risk, and makes remediation effort look more effective than it is.

By the numbers:

• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read Kusari's analysis of vulnerability precision and provenance metrics

Context

Vulnerability governance breaks down when teams optimise for visible counts instead of trustworthy inputs. The article argues that the board-friendly metric only works if asset classification, ownership, and provenance are accurate, which is often the part programs cannot actually prove.

In practice, scanner output is only as useful as the underlying build and dependency graph. That creates a direct governance problem for application security, supply chain security, and identity-adjacent controls such as secret handling and workload provenance, where false precision can be more damaging than an honest gap.

The article’s starting position is typical of mature security programmes, because many teams already know the right reporting metric but cannot reliably produce the data needed to support it.

Key questions

Q: What breaks when vulnerability findings are treated as board-grade risk without precision testing?

A: The metric collapses into filtered noise, because false positives and misplaced findings consume remediation capacity without reducing actual exposure. Boards then see a number that looks authoritative but does not reliably reflect deployed risk. The fix is to validate precision against real artifacts and track how much of the queue changed nothing.

Q: Why does provenance matter more than total vulnerability counts in supply chain security?

A: Because counts do not tell you whether the vulnerable component should have been there, how it entered the build, or whether anything in production can reach it. Provenance answers those questions and turns vulnerability management into a boundary control problem rather than a ranking exercise.

Q: How do you know if vulnerability remediation is actually working?

A: Look for reduced mean time to remediate, fewer reopened findings, and verified closure rather than ticket closure alone. If retesting shows the same issue recurring, the process is not controlling root cause. Effective remediation changes the environment, not just the report status.

Q: What is the difference between precision and recurrence in vulnerability reporting?

A: Precision measures how many acted-on findings were actually real, while recurrence measures whether the same class of exposure is still appearing over time. Precision tells you whether the instrument is trustworthy, and recurrence tells you whether the system is improving. Both are needed for executive reporting.

Technical breakdown

Why vulnerability precision fails in build and dependency scanning

Most vulnerability scanners are matching engines. They compare manifests or component inventories against advisory feeds and return matches, but that does not prove the vulnerable code exists in the built artifact or is reachable at runtime. This creates a precision problem: findings can be true statements about a manifest while being false statements about the software actually deployed. Once false positives enter the queue at scale, executive metrics become filtered noise rather than decision-grade risk data.

Practical implication: measure scanner precision against deployed artifacts, not just reported findings.

Why provenance matters more than total vulnerability counts

Provenance asks where a component came from, how it entered the build, and whether the deployed artifact still contains it. That matters because most open-source exposure sits in transitive dependencies and other code paths developers do not directly control. SBOM generation alone is not enough if teams cannot consume that data to answer placement, ownership, and change questions. Provenance turns vulnerability management from a counting exercise into a boundary and supply chain control problem.

Practical implication: verify dependency origin and artifact composition before treating a finding as actionable.

Why recurrence is a better outcome metric than ticket volume

Ticket closure measures workload, not improvement. Recurrence measures whether the same vulnerability class keeps reappearing after remediation efforts, which tells you whether the underlying system is changing. If a class of exposure persists, the program may be busy without becoming safer. This is the difference between activity metrics and control outcomes, and it is what makes preventative security visible to leadership.

Practical implication: track exposure recurrence by class over time, not just remediation throughput.

NHI Mgmt Group analysis

Precision is now a governance control, not just a tooling quality issue. When vulnerability data is half noise, executive reporting becomes a capacity sink rather than a risk signal. That means precision belongs in the same governance conversation as asset inventory and ownership, because the metric drives where scarce engineering time is spent. Practitioners should treat precision as a measurable control objective, not a vendor claim.

Provenance is the missing boundary control in modern supply chain security. The article correctly shifts attention from whether a component is vulnerable to whether it should have been present, how it was introduced, and whether it is actually reachable. That is a boundary question, not a scanner question, and it aligns with the way supply chain risk should be governed across build systems, artifacts, and release paths. Teams should elevate provenance to a first-class control domain.

Asset classification failure is what makes board metrics collapse in practice. Even a technically sound vulnerability metric fails if the organisation cannot distinguish internet-facing, business-critical, and low-value assets with confidence. This is the same governance weakness that affects IAM and NHI programmes: you cannot enforce meaningful controls over identities, workloads, or dependencies you cannot place. Practitioners need reliable classification before they can trust any board-facing metric.

False-positive waste should be expressed in capacity terms, not abstract risk language. The strongest executive argument in the article is that poor precision consumes the scarcest security resource, which is engineering attention. That reframes remediation efficiency as operational resilience, because every inaccurate finding diverts effort from actual exposure reduction. Security leaders should report wasted capacity as a governance defect, not just a tooling annoyance.

What this signals

Provenance debt will become a board-level issue faster than most vulnerability teams expect. As software supply chains grow more transitive, the real question is no longer how many issues are open but whether the organisation can prove what entered the build and why. That has direct overlap with NHI governance, because secret exposure and workload identity drift create the same problem: you cannot govern what you cannot place.

Security programmes that still optimise for closure volume will keep over-reporting progress. The more useful signal is whether precision improves while recurrence falls, which is the only combination that turns vulnerability management into structural risk reduction rather than queue management.

For practitioners

• Measure scanner precision on a single team queue Track every finding the team acted on during one sprint, then verify whether each one mapped to a real deployed exposure. Use that sample to calculate precision and to identify which tools or pipelines are producing the most waste.
• Prove artifact provenance before counting exposure Select a sample of findings and trace each one back to the exact dependency, transitive parent, and deployed artifact. If you cannot show where the component came from and whether it is reachable, do not treat the finding as board-grade risk data.
• Replace ticket volume with recurrence tracking Choose one vulnerability class and chart its incidence over the last two to six months rather than counting closed tickets. Use the trend to show whether remediation is reducing the underlying exposure pattern or only clearing the queue.
• Validate asset classification for executive metrics Test whether the assets behind your most important metric are correctly labelled for internet exposure, business criticality, and ownership. If classification is weak, the metric is not ready for leadership consumption.

Key takeaways

• The article shows that vulnerability counts are a weak executive metric when asset data, build provenance, and scanner precision are unreliable.
• The most useful measures are precision and recurrence, because they expose wasted capacity and reveal whether exposure classes are actually disappearing.
• For practitioners, the next step is to prove what is real, where it came from, and whether the same failure keeps returning.

Key terms

• Scanner Precision: Scanner precision is the share of acted-on findings that turn out to represent real exposure in the deployed environment. In application security, low precision wastes engineering effort, inflates board reporting, and masks the difference between a theoretical match and an exploitable issue.
• Provenance: Provenance is the traceable history of where a software artifact came from, who approved it, and what controls were applied along the way. In container security, provenance supports trust decisions because it links delivery steps to accountable identities and review points.
• Exposure Recurrence: Exposure recurrence is the repeated appearance of the same vulnerability class over time despite remediation activity. It is a stronger outcome signal than ticket closure because it shows whether the underlying control problem has changed, rather than whether the queue has been cleared.
• Asset Classification: Asset classification is the process of identifying which systems matter most by exposure, criticality, ownership, and business function. In practice, weak classification causes even good metrics to fail, because security teams cannot rank risk accurately without trustworthy context.

What's in the full article

Kusari's full analysis covers the operational detail this post intentionally leaves for the source:

• How the precision metric was calculated across real codebases and deployed artifacts
• The false-positive comparison methodology behind the scanner evaluation
• The dependency provenance questions teams should ask at build and release time
• Why recurrence tracking is more useful than raw vulnerability counts for executive reporting

👉 Kusari's full post covers scanner false positives, dependency provenance, and the metric trade-offs behind executive reporting

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management, and workload identity. It helps practitioners connect identity controls to the broader governance problems that make risk metrics trustworthy.