Most organisations can describe their software supply chain controls, but few can show a quarterly number for how much of what they ship has an SBOM, a signature and verifiable build provenance. This health check is a recurring scorecard for the software supply chain programme: ten metrics covering SBOM coverage, artefact signing, provenance, SLSA build levels, dependency hygiene and intake, registry proxying and exploitability status (VEX). Each metric has a formula, a data source type, thresholds, an owner and a cadence, plus a findings log and a remediation map. It is run by the product security or platform engineering team and signed off by the head of application security. The controls themselves are explained in the Software Supply Chain Security Guide; this page measures whether they are actually in place across your products.
What this health check covers
Scope is software your organisation builds and releases, internally or to customers: source repositories, build pipelines, artefacts, container images and the open source dependencies they pull in. It does not assess suppliers who sell you software (that belongs in the TPRM Programme Health Check), and it does not audit the CI server as a product (see the CI/CD server health check). Use the Software Supply Chain Maturity Model to describe where you are going; use this page to measure where you are each quarter.
It also differs from a per-project score. The OpenSSF Scorecard Guide explains automated checks run against a single repository. This health check rolls results up across your whole release portfolio, weights them by what reaches production, and turns them into owned findings. If you searched for software supply chain KPIs, SBOM coverage metrics or a supply chain security audit checklist, this is the working version.
Key takeaways
- Measure what ships, not what exists: every coverage metric uses production releases or deployed artefacts as the denominator, so abandoned repositories do not flatter the score.
- Signing without verification is half a control: count signatures and provenance that are checked at deploy or admission time, not files that merely sit beside an artefact.
- Dependencies carry most of the risk you did not write: pinned, maintained and free of known exploited vulnerabilities are three separate metrics because they fail for different reasons.
- Build isolation is a level, not a yes or no: report the SLSA Build level distribution so progress from L1 to L3 is visible.
- Thresholds are starting points: the bands are NHI Mgmt Group recommendations; baseline your own and record why you change them.
How to use this template
Copy the metrics table and findings log into your own tracker. Define the release inventory first (which products, which repositories, which artefacts reach production), because every metric depends on that denominator. Collect each figure from the data source types described, score it against the bands, and log every amber or red result as a finding with an owner and a due date. Keep each quarter's workbook: a falling trend in one product line usually points to a pipeline template or a team, not to individual repositories.
Cadence and sign-off
| Review | Frequency | Owner | Approver | Evidence kept |
|---|---|---|---|---|
| Known exploited dependency sweep (SC-07) | Weekly | Product security | Head of application security | Dependency scan export matched to the known exploited vulnerabilities catalogue, ticket list |
| Dependency intake and VEX review (SC-08, SC-10) | Monthly | Application security | Head of application security | Intake records for new dependencies, VEX status list, open items |
| Full metric run (SC-01 to SC-10) | Quarterly | Product security lead | CISO | Metric workbook, release inventory snapshot, findings log, sign-off record |
| SLSA level re-assessment (SC-04) | Every six months, and after any build platform change | Platform engineering | Head of application security | Build platform assessment against the SLSA Build track, provenance samples |
| Independent re-performance | Yearly | Internal audit or second line | Audit committee | Re-performed samples from five releases, variance notes |
Metrics
The denominator for SC-01 to SC-04 is the set of production releases in the review period: every versioned artefact (package, binary, container image, firmware) promoted to production or shipped to customers. NHI Mgmt Group recommended thresholds; adjust to your risk appetite.
| ID | Metric | Formula (numerator / denominator) | Data source type | Green | Amber | Red |
|---|---|---|---|---|---|---|
| SC-01 | SBOM coverage | Production releases with a machine-readable SBOM generated at build time and stored with the release / production releases | Release records; SBOM repository or artefact metadata | ≥ 95% | 80% to below 95% | < 80% |
| SC-02 | Signed artefact share | Production artefacts whose signature was verified by a deploy gate or admission policy before running / production artefacts deployed | Deployment logs; admission controller decision logs; signature verification logs | ≥ 95% | 75% to below 95% | < 75% |
| SC-03 | Provenance attestation share | Production releases with signed build provenance naming the source commit and build definition / production releases | Attestation store; release records | ≥ 90% | 60% to below 90% | < 60% |
| SC-04 | SLSA level distribution | Production releases by assessed SLSA Build level (L0, L1, L2, L3), each level / production releases; headline = share at L2 or above | Build platform assessment; provenance samples | ≥ 80% at L2 or above, critical products at L3 | 50% to below 80% at L2 or above | < 50% at L2 or above, or any critical product at L0 |
| SC-05 | Pinned dependency share | Build-time dependencies (base images, pipeline actions or plugins, downloaded tools) referenced by immutable digest or hash / build-time dependency references in release pipelines | Pipeline definition and container build file scan | ≥ 95% | 80% to below 95% | < 80% |
| SC-06 | Unmaintained dependency count | Count of direct dependencies in production releases that are archived, past end of life, or have had no release or commit in 24 months; report per 100 direct dependencies | SBOM data joined to package registry and project metadata | ≤ 1 per 100, each with an exception | Above 1 to 3 per 100 | > 3 per 100, or any without an exception |
| SC-07 | Known exploited dependency count | Count of deployed artefacts containing a component with a vulnerability listed in the CISA Known Exploited Vulnerabilities catalogue, unremediated past your remediation deadline | SBOM or dependency scan joined to the KEV catalogue; deployment inventory | 0 | 1 to 2, each within an approved exception | ≥ 3, or any without an exception |
| SC-08 | New dependency intake review | New direct dependencies added to production code in the period that passed the intake check (maintainer activity, licence, known malicious package screen) before first merge / new direct dependencies added in the period | Dependency manifest change history; intake review records | ≥ 95% | 75% to below 95% | < 75% |
| SC-09 | Registry proxy coverage | Dependency downloads in release builds served through the organisation's approved proxy or mirror / dependency downloads in release builds | Proxy or mirror access logs; build network egress logs | ≥ 98% | 90% to below 98% | < 90% |
| SC-10 | Exploitability status coverage | Critical and high vulnerabilities in components of shipped releases with an exploitability status (affected, not affected with justification, fixed, under investigation) recorded or published as VEX within 30 days of detection / critical and high vulnerabilities in components of shipped releases detected in the period | SBOM vulnerability matching results; VEX documents or status records | ≥ 90% | 60% to below 90% | < 60% |
Notes on the formulas
- SC-01 counts build-time SBOMs: an SBOM produced by scanning the finished image later is useful, but it misses components removed during build and cannot be tied to the exact release. The NTIA minimum elements (July 2021) group requirements into data fields, automation support, and practices and processes; check that each SBOM carries the baseline fields. The SBOM operations guide covers storage and refresh.
- SC-02 counts verification, not signatures: a signature file that no gate checks gives no protection. Count an artefact only if the deploy or admission step rejected unsigned or wrongly signed artefacts in enforce mode. Background in the Sigstore Guide and Code Signing Security Guide.
- SC-03 and SC-04 use the SLSA Build track: in SLSA v1.2 (approved), Build L1 means provenance exists, L2 means a hosted build platform generates and signs it, and L3 means hardened builds where runs cannot influence one another and signing secrets are kept away from user-defined steps. Assess the build platform once and inherit the level for releases built on it. See the SLSA and Build Provenance Guide and in-toto Attestations Guide.
- SC-05 covers the build, not the application lockfile: application dependencies are governed by lockfiles and update automation; this metric targets mutable references in pipelines and container builds, such as a base image by tag or a pipeline action by branch. The OpenSSF Scorecard Pinned-Dependencies check uses the same idea (pinned to a hash rather than a mutable version).
- SC-06 needs a fixed staleness rule: choose one definition (archived, declared end of life, or no release in 24 months) and keep it stable so quarter-on-quarter trends compare.
- SC-07 uses deployed artefacts: a vulnerable component in an image that is no longer deployed is housekeeping, not exposure. Use the remediation deadlines from your vulnerability policy; see Remediation SLAs and Vulnerability Exceptions Guide.
- SC-10 tells customers what matters: an SBOM without an exploitability status pushes every downstream team to triage the same findings. Count a vulnerability only when its status is recorded with a justification. Pipeline tokens, runners and other build platform controls are measured in the CI/CD Health Check, not here.
How to collect the data
- Freeze the release inventory: export every production release in the period from release records or the deployment system, with product, version, artefact digest and build job identifier. This is the denominator for most metrics.
- Join SBOMs and attestations: for each artefact digest, look up the SBOM and the signed provenance in the attestation store or artefact metadata (SC-01, SC-03). Match on digest, never on version tag.
- Pull verification evidence: export deploy gate and admission controller decision logs for the period, including denials, and confirm the policy was in enforce mode (SC-02).
- Scan pipeline definitions: parse pipeline definition files and container build files in release repositories for references by tag or branch rather than digest (SC-05); compare manifest changes in the period with intake review records (SC-08).
- Enrich dependency data: join SBOM components to package registry metadata for last release date and archive status (SC-06), and to the CISA KEV catalogue for known exploited vulnerabilities (SC-07).
- Read build and network logs: join SBOM vulnerability matches to VEX documents or status records (SC-10); proxy access logs and build egress logs show downloads that bypassed the approved proxy (SC-09).
- Sample and trace: pick five releases at random and trace each from source commit to running workload, checking every metric value by hand. Record any variance in the workbook.
Red flags that need action now
- A deployed artefact containing a known exploited vulnerability with no exception and no ticket (SC-07). Handle it under the Supply Chain Compromise Response Playbook if exploitation is suspected.
- Signature or provenance verification set to audit-only on a production cluster or deploy path (SC-02).
- A critical product built at SLSA Build L0, for example on a developer workstation (SC-04).
- Release pipelines pulling base images or pipeline actions by mutable tag or branch (SC-05).
- New dependencies merged with no intake check, especially newly published packages with names close to popular ones (SC-08).
- Release builds downloading packages directly from public registries, bypassing the proxy (SC-09). See Malicious Package and Dependency Confusion Guide.
- Critical vulnerabilities in shipped components with no exploitability status weeks after customers have asked (SC-10).
- No SBOM for a product shipped to customers who have asked for one (SC-01).
Findings log
| Finding ID | Finding | Metric | Severity | Owner | Due date | Status |
|---|---|---|---|---|---|---|
| HC-SSC-01 | [Number] deployed artefacts contain a KEV-listed component past deadline | SC-07 | [High / Medium / Low] | [Role] | [Date] | [Open / In progress / Closed / Risk accepted] |
| HC-SSC-02 | Signature verification in audit mode on [environment] | SC-02 | [Severity] | [Role] | [Date] | [Status] |
| HC-SSC-03 | [Product] built at SLSA Build L[n] | SC-04 | [Severity] | [Role] | [Date] | [Status] |
| HC-SSC-04 | [Number] critical vulnerabilities with no VEX status | SC-10 | [Severity] | [Role] | [Date] | [Status] |
| HC-SSC-05 | [Finding] | [Metric] | [Severity] | [Role] | [Date] | [Status] |
Raise one finding per root cause: twenty repositories using the same unpinned pipeline template are one finding against the template owner. Risk accepted findings carry an expiry date and the approver's name, and are re-opened at the next quarterly run.
Remediation map
| Finding type | Metrics | Where the fix is explained |
|---|---|---|
| Missing or late SBOMs | SC-01 | SBOM Guide; SBOM operations |
| Unsigned or unverified artefacts | SC-02 | Sigstore Guide; Code Signing Security Guide |
| Missing provenance, low SLSA level | SC-03, SC-04 | SLSA and Build Provenance Guide |
| Unpinned or unmaintained dependencies | SC-05, SC-06 | Dependency Update Automation Guide; OpenSSF Scorecard Guide |
| Known exploited components | SC-07 | CISA KEV Catalogue Guide; VEX Guide |
| Dependencies added without intake review | SC-08 | Malicious Package and Dependency Confusion Guide |
| Proxy bypass | SC-09 | Malicious Package and Dependency Confusion Guide |
| Missing exploitability status | SC-10 | VEX Guide |
Related guides
- Software Supply Chain Security Guide: the controls this health check measures.
- Software Supply Chain Maturity Model: map the results to a maturity level.
- NIST Secure Software Development Framework (SSDF, SP 800-218) Guide: the practice framework behind several metrics.
- Supply Chain Compromise Response Playbook: when a finding becomes an incident.
- CI/CD server health check: the matching product health check for the build server.
References
Framework anchors for the metrics: NIST SP 800-53 Rev. 5 SR-3 (Supply Chain Controls and Processes), SR-4 (Provenance), SR-11 (Component Authenticity), CM-14 (Signed Components), SI-7(15) (Code Authentication), SA-15 (Development Process, Standards, and Tools), SA-22 (Unsupported System Components) and RA-5 (Vulnerability Monitoring and Scanning); NIST CSF 2.0 GV.SC-07 and GV.SC-09.
- OpenSSF: SLSA v1.2 Build Track Basics
- NIST: SP 800-218 Secure Software Development Framework Version 1.1
- NIST: SP 800-161 Rev. 1 Cybersecurity Supply Chain Risk Management Practices
- NIST: SP 800-53 Rev. 5
- NTIA: The Minimum Elements for a Software Bill of Materials
- CISA: Known Exploited Vulnerabilities Catalog
- OpenSSF: Scorecard Checks
Related NHI Mgmt Group resources: Software Supply Chain Security Guide, Software Supply Chain Maturity Model, SLSA and Build Provenance Guide, SBOM Guide.