Vulnerability Deduplication and Correlation Guide: Turning Many Scanner Findings into One Fix

When three scanners report the same flaw on the same asset, a vulnerability programme that counts findings instead of fixes will overstate its backlog, page the wrong owners and miss its service levels. Deduplication collapses repeated reports of one condition into a single record, and correlation links records that share a root cause, an asset or a fix. This guide covers the mechanics practitioners have to design: the identifiers that make matching possible (CVE, CWE, CPE, purl, OSV aliases, SARIF fingerprints), the asset matching that sits underneath them, hash field selection, merge rules and the measurements that show the pipeline is working. It is for vulnerability management leads, platform engineers and AppSec teams. Prioritisation itself is covered in the Risk-Based Vulnerability Management Guide; this page is about getting a clean, countable finding set before any scoring happens.

Key takeaways

• Deduplicate on identity, correlate on fix. Deduplication asks "is this the same condition on the same thing?" Correlation asks "will one change close all of these?" Keep the two questions, and their outputs, separate.
• Asset identity comes first. No finding key survives a bad asset key. Reconcile hostnames, IP addresses, cloud instance IDs, image digests and repository paths into one asset record before matching findings.
• Use the most specific stable key the source offers. A scanner's own unique ID or a SARIF fingerprint beats a hash of title and severity; a purl beats a free-text package name; a CVE beats a plugin name.
• Never delete, always link. Mark duplicates inactive and point them at an original so evidence from every tool stays auditable and closes when the original closes.
• Measure the pipeline. Track duplicate ratio, reopen rate, unmatched asset rate and the share of findings with a normalised identifier, or silent matching errors will undo the work.

Why the same flaw appears many times

Duplicate findings are not a scanner defect; they are a predictable result of layered tooling. The ASPM vs Unified Vulnerability Management guide covers the cross-domain case; the sources below are what a normalisation layer handles.

The last row causes the most damage because it looks like progress: a rebuilt host gets a new identifier, its old findings close as "not seen", and the same findings open again under the new identity, so mean time to remediate looks short while exposure never changed.

The identifiers that make matching possible

Normalisation means mapping every source finding onto a small set of shared identifiers. Each answers a different question, and none is sufficient alone.

Vulnerability identity: CVE and aliases

Treat the vulnerability as its own entity with a canonical ID and a list of aliases. The OSV schema gives a precise test: two records are aliases if they affect any given component the same way, which in practice means one patch fixes both and no others. Aliases are symmetric and transitive, so if a GHSA aliases a CVE and the CVE aliases a distribution advisory, all three collapse into one vulnerability identity. Downstream distribution advisories that bundle several upstream CVEs belong in upstream, not aliases; merging them would wrongly close several CVEs when one is fixed. Records that are close but fail the alias test go in related, which is symmetric but not transitive.

Pick a canonical ID by precedence (for example CVE first, then GHSA, then the scanner's own ID) and keep the others as searchable aliases. Distribution backports complicate version matching; see the Backported Security Fixes Guide before treating a version mismatch as a true positive.

Component identity: purl and CPE

The purl specification exists to identify a software package independently of its ecosystem or distribution channel, and it became Ecma standard ECMA-427 in December 2025 with an ISO track in progress. A purl has a type (npm, pypi, maven, deb, rpm, oci and others), an optional namespace, a name, a version, qualifiers and a subpath. Normalise component findings to purl, and compare on type, namespace and name first, version second. The OSV schema recommends a purl without the version component for the affected package, which suits matching: identity on the package, applicability on the version range.

CPE 2.3 names, defined in NISTIR 7695, describe a product by part (application, operating system, hardware), vendor, product, version, update, edition, language and further attributes. NVD applicability data is expressed in CPE, so network and agent scanners that work from NVD often emit CPEs. Vendor and product strings differ between sources, so keep a maintained mapping table from CPE vendor:product pairs to purls or internal software catalogue IDs rather than string matching.

Weakness identity: CWE

CWE classifies the weakness type. It is valuable for correlation (all injection findings in one service) and for root-cause programmes, but two different SQL injection findings share a CWE without being duplicates. NVD maps CVEs to a simplified set defined in CWE View 1003, "Weaknesses for Simplified Mapping of Published Vulnerabilities"; MITRE marks the view ID itself as prohibited for mapping, so map to the individual CWE entries it contains. Some SAST tools report CWE as a set, so store it as a list.

Code result identity: SARIF fingerprints

For static analysis, SARIF 2.1.0 defines a fingerprint as a stable value a result management system can use to uniquely identify a result over time, even if a relevant artifact is modified. The relevant properties are result.fingerprints (section 3.27.16), result.partialFingerprints (3.27.17), result.guid and result.correlationGuid (3.27.3 and 3.27.4) and result.baselineState (3.27.24), which records whether a result is new, unchanged, updated or absent relative to a baseline. Fingerprint names are versioned (a /v2 suffix supersedes the unversioned name); never compare across versions.

GitHub's code scanning documentation shows how fragile this is in practice: it uses partialFingerprints to decide whether two results are the same, the rule ID must stay constant across analyses, and a changed file path for the same result opens a new alert and closes the old one. Uploading SARIF through the API without fingerprint data can produce duplicate alerts. If your tool does not emit fingerprints, compute them before ingestion rather than relying on line numbers.

Asset matching underneath everything

A finding key is roughly (asset, vulnerability or rule, location). If the asset part is unstable, every other step fails. Build an asset reconciliation layer that resolves each source's asset reference to one canonical asset ID before findings are matched. See the Asset Inventory and CMDB Security Guide for inventory governance; the matching rules below are specific to findings.

• Rank sources: decide which source wins for each attribute (cloud API for instance ID, agent for installed software, CMDB for owner and criticality) and record that order explicitly.
• Keep history: store previous identifiers against the canonical asset so a renamed host's old findings migrate rather than close.
• Quarantine unmatched assets: findings on assets that cannot be reconciled go to an "unmatched" queue with an owner, not straight into the backlog under a guessed identity.

Designing the deduplication key and merge rules

Open-source aggregation tools show the design space well. OWASP DefectDojo documents four selectable algorithms per scanner type: Unique ID From Tool, Hash Code (a hash of a configured field set), Unique ID From Tool or Hash Code (tool ID with hash fallback) and Legacy. Hash fields are set per scanner (HASHCODE_FIELDS_PER_SCANNER), a global HASH_CODE_FIELDS_ALWAYS setting adds fields to every hash (defaulting to service), and endpoint matching can be switched on or off with DEDUPE_ALGO_ENDPOINT_FIELDS. Its documentation also notes that reimport and same-tool deduplication can use different hash fields for the same scanner, which can turn a changed line number into a new finding that is then marked duplicate. The same choices apply on any platform.

Merge rules

• Original selection: the earliest-created finding is the original and is never demoted by a newer match (DefectDojo uses this rule; it keeps first-seen dates and service level clocks honest).
• Status: duplicates are set inactive and linked to the original; closing the original closes its duplicates. Never hard-delete evidence.
• Severity: take the highest source severity for display, but compute priority from your own model, not from a vote between tools.
• Last seen: update the original's last-seen date from any source; a finding closes only when every source that reported it stops reporting it, or a verified fix is recorded.
• Exceptions: risk acceptances and VEX "not affected" statements attach to the canonical finding so they suppress every duplicate. See the VEX Guide.
• Scope: start with deduplication inside one asset or product; widen to global matching only for component or vulnerability views, and only deliberately. DefectDojo's commercial edition, for example, ships cross-asset matching switched off by default behind feature flags.

Correlation: from findings to fixes

Deduplicated findings still overstate work if one change closes many of them. Correlation adds a grouping layer above findings without merging them, so per-asset evidence and per-asset closure stay intact.

• Fix groups: group by (component purl, fixed version) or by (product, patch or KB). One ticket per fix group per owning team.
• Image lineage: group container findings by base image digest; route base layer fixes to the image owner and application layer fixes to the application team. See Container Image Scanning vs SCA.
• Code to cloud: link a repository finding to images built from it (via build provenance or SBOM) and to workloads running those images, so runtime exposure informs priority. See the SBOM Guide.
• Weakness clusters: group SAST findings by CWE and component to spot a missing shared control (for example a query builder that every service bypasses).
• Exploitation context: join canonical CVE IDs to the CISA KEV catalogue, whose data feed carries fields such as cveID, vendorProject, product, dateAdded, dueDate and knownRansomwareCampaignUse, and to EPSS. Clean CVE normalisation is what makes those joins reliable. See the EPSS Guide.

Where identity fits

Every aggregation pipeline runs on non-human identities: scanner service accounts with authenticated access to hosts, API tokens that pull results from each tool, and the integration account that writes tickets. Authenticated scanning credentials (the subject of NIST SP 800-53 RA-5(5) Privileged Access) are high value, and an aggregation platform holds tokens for every security tool in the estate. Store them in a vault, scope them read-only where the tool allows, rotate them and inventory them as owned machine identities. A host whose scanner credential has failed looks clean, so track authenticated scan success per asset. See the Service Account Security Guide and the Secrets Management Guide.

Implementing a normalisation pipeline

1. Inventory sources: list every tool that produces findings, its output format (native API, SARIF, CycloneDX, CSV), the identifiers it emits (CVE, CPE, purl, CWE, fingerprints) and its asset reference.
2. Build asset reconciliation first: define the canonical asset key per asset type and the source precedence; measure the unmatched rate before touching findings.
3. Normalise identifiers: map advisories to a canonical vulnerability ID with aliases, packages to purl, products to a catalogue ID, and store CWE as a list.
4. Configure keys per finding type: start with the table above; prefer the tool's unique ID within a tool and a field hash across tools.
5. Shadow mode: run the new matching alongside the current process for two to four scan cycles. Compare counts, sample 50 to 100 merged pairs and 50 to 100 unmerged near matches by hand, and record false merge and missed merge rates.
6. Cut over by source: enable matching one scanner at a time, starting with the noisiest overlap (often agent and network scanner on servers).
7. Add correlation groups: fix groups and image lineage only after the deduplicated set is stable; route tickets per group.
8. Roll back safely: because duplicates are linked rather than deleted, rollback means unlinking and reactivating. Keep the original raw findings for at least one full remediation cycle, and version your hash field configuration so a change can be reverted and replayed.

Decision guidance

• Aggregation platform or scanner console: a single scanner family on a small estate can rely on its own console's deduplication. Once two or more tools overlap on the same assets, a separate aggregation layer (open-source, UVM or ASPM) is needed, because no scanner deduplicates another vendor's output.
• Strict or loose matching: a false merge hides a real finding; a missed merge wastes effort. For internet-facing and crown jewel assets, prefer strict keys and accept some duplicates. For bulk internal hygiene, looser keys with set matching on CVE lists are acceptable.
• Merge or group: merge only when the condition, asset and location are the same. If closure requires separate changes on separate assets, group instead.
• Small organisations: one scanner, one asset source of truth and a documented key per finding type is enough. Large organisations need a named owner for matching rules, change control on hash fields and a quarterly accuracy sample.

Detection and measurement

Alert on changes to hash fields and matching scope, and log every merge and unmerge with its rule.

Framework mapping

Common failure patterns

• Hashing on volatile fields: including line numbers, plugin output or scan timestamps in the key creates a new finding every scan and resets remediation clocks.
• IP address as the asset key: in cloud and DHCP estates, findings jump between unrelated machines and false closures follow.
• Treating upstream as alias: merging a distribution advisory with each bundled CVE closes several vulnerabilities when one is fixed.
• Deleting duplicates: removes the evidence that a second tool saw the flaw and breaks audit trails; link and deactivate instead.
• Global matching switched on by default: cross-product matching merges the same CVE on unrelated assets owned by different teams, so one team's fix closes another's exposure.
• No owner for matching rules: scanner updates change plugin IDs or output formats and keys silently break; duplicate ratio is the early warning.

Practitioner checklist

• List every finding source with its identifiers, format and asset reference.
• Define a canonical asset key per asset type and a source precedence order.
• Normalise vulnerabilities to a canonical ID with aliases, following the OSV alias, upstream and related definitions.
• Normalise components to purl and keep a CPE to catalogue mapping table.
• Require SARIF fingerprints (or compute them) for every static analysis source.
• Document the key fields per finding type and version the configuration.
• Link and deactivate duplicates; never delete raw findings.
• Run new matching in shadow mode and sample merges before cut-over.
• Build fix groups and image lineage on top of the deduplicated set.
• Report duplicate ratio, reopen rate, unmatched assets and false merge rate monthly.

Standards and references

• OASIS: Static Analysis Results Interchange Format (SARIF) Version 2.1.0
• OpenSSF: Open Source Vulnerability (OSV) Schema
• Package-URL: purl Introduction and ECMA-427
• NIST: NISTIR 7695 Common Platform Enumeration Naming Specification Version 2.3
• MITRE: CWE View 1003, Weaknesses for Simplified Mapping of Published Vulnerabilities
• NIST: SP 800-40 Rev. 4 Guide to Enterprise Patch Management Planning
• OWASP DefectDojo: About Deduplication
• GitHub Docs: SARIF Support for Code Scanning

Related NHI Mgmt Group resources: Risk-Based Vulnerability Management Guide, ASPM vs Unified Vulnerability Management, Exposure Management and CTEM Guide, VEX Guide.