TL;DR: Raw vulnerability counts have become easy to inflate with cheap AI-assisted detection and noisy rules, so numbers like “we surfaced 1,296 vulnerabilities” now say more about generation speed than actual risk, according to Nullify. Verified risk removed, not volume found or closed, is the only defensible measure of control.
At a glance
What this is: This analysis argues that vulnerability counts have become a vanity metric because both attackers and defenders can now generate findings cheaply and at scale.
Why it matters: It matters because security leaders, IAM teams, and application security owners need metrics that reflect verified risk reduction, not noisy output that can distort budgeting, prioritisation, and board confidence.
Context
Security programmes often collapse into reporting what is easy to count instead of what is actually controlled. In vulnerability management, raw finding totals can rise or fall for reasons that have little to do with exposure, especially when noise, duplicate findings, and automated generation distort the dataset.
The deeper governance problem is metric integrity. If a leadership dashboard cannot distinguish verified risk from inflated signal, then it cannot reliably support budget decisions, staffing, or escalation thresholds. That is true across application security, cloud operations, and any programme that depends on trustworthy measurement.
Key questions
A: Measure effectiveness by verified risk reduction, not by the number of findings produced or closed. The useful metrics are reachability, true-positive rate, confirmed removals after re-scan, and how much developer friction the programme creates. Raw counts can still help with volume trends, but they should never be the primary proof that security improved.
Q: What breaks when vulnerability dashboards are driven by raw finding totals?
A: They break prioritisation. Noise, duplicates, and unreachable findings can make a programme look busy while the real risk stays unchanged, which leads to bad paging, bad budgeting, and eventual distrust from engineers. Once that trust is lost, real issues become harder to get fixed because every alert looks equally suspect.
Q: What breaks when reachability analysis is missing from vulnerability triage?
A: Without reachability analysis, teams rely on scan volume instead of evidence. That causes two failures: over-reporting low-risk findings and missing the moment a vulnerability becomes a reportable incident. For CRA readiness, reachability is what separates a long vulnerability list from a short list of obligations.
Q: What is the difference between fixing vulnerabilities and suppressing noisy findings?
A: Fixing removes the weakness from the system and should survive re-scan. Suppressing, allowlisting, or deduplicating changes how a finding is handled, but it does not prove the underlying issue was eliminated. Both can be valid outcomes in a programme, but they should be reported separately so leaders do not confuse noise reduction with remediation.
Technical breakdown
Why raw vulnerability counts drift away from risk
Raw vulnerability counts aggregate everything a scanner or model emits, including duplicates, false positives, and findings that never map to exploitable conditions. Once detection becomes cheap, the count stops representing exposure and starts representing throughput. The operational problem is not only noise volume, but the fact that different tools and tuning choices can change the number without changing the underlying risk surface. In practice, a count without validation tells you how much output you generated, not how much danger you removed. See the CWE Top 25 Guide.
Practical implication: Treat raw counts as a diagnostic input only, and require validation before using them for programme reporting.
How reachability analysis changes severity triage
Reachability analysis asks whether a reported weakness is actually on a code path the application can execute. That makes it a control on triage, not on detection: the finding still exists, but its operational priority changes when the vulnerable code is unreachable from real entrypoints. This is especially important in dependency scanning, where one package version can inherit many CVEs but only a subset may matter to the application. Done well, reachability reduces paging noise without pretending unreached issues never existed. See the Zero-Day and Exploited Vulnerabilities Guide.
Practical implication: Use reachability to demote unreachable findings, but preserve conservative handling for anything the analysis cannot prove out.
What verified risk removal looks like in practice
Verified risk removal is a closed loop, not a dashboard label. A finding should only count as removed when the organisation can show the original issue is no longer present, the remediation did not introduce a new weakness, and the result survives re-scan. That standard matters because allowlisting, deduplication, and false-positive handling are legitimate outcomes, but they are not the same as repair. In governance terms, the question shifts from how many items were closed to what evidence proves the risk state changed.
Practical implication: Define closure states separately for fixed, suppressed, deduplicated, and false-positive findings so metrics stay interpretable.
Threat narrative
Attacker objective: The objective is to overwhelm defender decision-making so real risk is hidden inside an inflated volume of low-value findings.
- Entry occurs through cheap AI-assisted vulnerability generation or noisy scanner output that floods teams with plausible-looking findings.
- Escalation follows when defenders use raw count as a decision metric, because the inflated number obscures which findings are actually reachable or exploitable.
- Impact is metric corruption: prioritisation, budget justification, and board reporting all drift away from verified risk reduction.
Breaches seen in the wild
- United Nations breach 2021: Sakura Samurai used exposed Git credentials to reach 100,000+ UNEP staff records, then reported the flaw through the UN disclosure programme.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Raw vulnerability counts have become governance theatre. A number that rises because detection got cheaper is not a measure of control, and it should not be used to defend budget or maturity. The real question is whether the programme can show that exposed risk went down, not whether the scanner or model produced more findings. Leaders should treat count inflation as a metric design failure, not a tooling success.
Verified risk removal is the only defensible security outcome. Once a finding can be generated, duplicated, or hallucinated at trivial cost, closure metrics must prove that the weakness was eliminated or safely ruled out. That is a stronger governance standard than “closed,” because it preserves the distinction between remediation and suppression. Practitioners should align reporting to evidence of state change, not activity volume.
Metric integrity is now part of security architecture. In programmes that include application security, cloud posture, and NHI governance, bad measurement can hide weak control boundaries just as effectively as a bad policy can. board cyber risk reporting becomes more important when metrics are contested, because leadership needs a dashboard that distinguishes verified exposure from noisy output. The right question is no longer how many findings exist, but which ones still matter.
Noise management is now a core operational control. False positives, duplicates, and unreachable findings are not administrative clutter. They shape whether engineers trust the programme enough to fix the issues that actually matter. Teams that do not separate signal from noise will eventually optimise for reporting convenience, which is how a vulnerability programme becomes a volume exercise instead of a risk programme.
Read next: Board Cyber Risk Reporting Guide
What this signals
Verification is replacing volume as the governance standard. Security programmes that still optimise for finding counts will keep confusing activity with control. The more practical model is to ask whether each reported issue was reachable, confirmed, or genuinely removed, because that is what changes operational risk.
Reachability is becoming the missing layer between detection and action. In application and dependency scanning, a finding that cannot be executed from a real entrypoint should not receive the same treatment as one that can. That distinction will matter more as AI-assisted detection makes raw output cheaper and more abundant.
Metric design now shapes control behaviour. If the dashboard rewards quantity, the programme will produce quantity. If it rewards verified removal, teams will tune for precision, confirmation, and reduced noise, which is the only path that scales without burning developer trust.
For practitioners
- Define closure states separately Separate fixed, suppressed, allowlisted, deduplicated, and false-positive outcomes in reporting so “resolved” never disguises what actually happened.
- Gate board metrics on verified risk Report counts only after validation, and pair them with evidence of reachable risk removed rather than raw scanner output. See the Board Cyber Risk Reporting Guide.
- Tune detections against noisy rule sets Review the highest-noise rules first and strip out patterns that generate volume without changing true-positive coverage.
- Use reachability for triage, not denial Demote findings only when you can prove the application can hit the vulnerable path, and keep unknown cases conservatively visible.
- Measure developer friction alongside risk Track how much effort it takes to remove real findings, because a programme that burns trust will eventually lose remediation velocity.
Key takeaways
- Raw vulnerability counts no longer tell leaders whether a security programme is actually reducing exposure.
- The article shows that tuning and reachability analysis can collapse noisy output dramatically while preserving the real issues.
- Programmes should report verified removal, separate noise-handling outcomes, and keep board metrics tied to evidence rather than volume.
Related NHI Mgmt Group resources: Board Cyber Risk Reporting Guide · Zero-Day and Exploited Vulnerabilities Guide · CWE Top 25 Guide · Attack Surface Management Guide
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST CSF 2.0, CIS Controls v8 and OWASP ASVS set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | TA0006 — Credential Access | Cheap finding generation and noisy outputs can hide real exploit paths that still depend on credential access. |
| Recommendation — Map noisy detection output back to likely exploit tactics and prioritise findings that align with real attacker movement. | ||
| NIST CSF 2.0 | GV.OV-01 — Oversight of Risk Management Strategy | This article is about whether the programme's metrics actually support governance decisions. |
| Recommendation — Tie vulnerability reporting to governance metrics that prove risk reduction, not scanner volume. | ||
| CIS Controls v8 | CIS-7 — Continuous Vulnerability Management | The piece challenges how teams measure and manage vulnerability findings at scale. |
| Recommendation — Use continuous vulnerability management to validate, prioritise, and verify fixes instead of counting raw findings. | ||
| OWASP ASVS | V16 — Security Logging and Error Handling | The article's concern with trustworthy evidence and false positives aligns with validation and reporting integrity. |
| Recommendation — Instrument security reporting so findings, suppressions, and confirmations remain auditable and separable. | ||
Key terms
- Reachability analysis: Reachability analysis checks whether a vulnerability can actually be exploited in the application’s real code paths and dependency graph. It helps teams distinguish theoretical findings from issues that an attacker can reach, which makes prioritisation far more accurate for both AppSec and identity risk management.
- False Positive: A false positive is a scanner result that looks like a secret but is not actually sensitive. In secret governance, false positives matter because they consume analyst time, weaken trust in alerts, and can delay response to the findings that truly change exposure and access risk.
- Real Risk Reduction: Real risk reduction means lowering actual exposure rather than only improving policy language or audit documentation. For AI and data security, it depends on measurable controls such as discovery, classification, access restriction, monitoring, and remediation that reduce the chance of sensitive data misuse or leakage.
- Findings Noise: Findings noise is the volume of low-value, duplicated, or misleading alerts that obscures real security issues. It is a governance problem as much as a tooling problem, because noisy data pushes teams toward activity metrics instead of risk-based decisions.
Deepen your knowledge
NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore nhimg.org for resources that connect identity governance to the broader security disciplines your programme depends on.
Published by the NHIMG editorial team on October 22, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org