Software Supply Chain Security Buyer's Guide: Criteria, PoC

Many software supply chain security purchases end with an SBOM generator, a dependency scanner and a dashboard, none of which stops a malicious package being installed or an unsigned image being deployed. The category spans software composition analysis (SCA), SBOM and VEX management, malicious package detection, repository and pipeline posture, provenance and signature verification, artifact registry protection and admission policy. Products range from single-purpose tools to platforms that claim all of it. This vendor-neutral buyer's guide is for AppSec, platform engineering and procurement teams writing requirements or running an RFP. It covers what to define first, the criteria to score, questions that expose weak answers, red flags and a proof of concept.

Key takeaways

Buy for the decisions you need to enforce. Blocking a malicious package at install, refusing an unverified image at deploy and answering a customer's SBOM request are different capabilities; list which ones you need.
Verification matters more than generation. SBOMs and provenance only reduce risk when something checks them. SLSA describes Build L1 provenance as "trivial to bypass or forge"; signed provenance verified against expected values is what changes outcomes.
Malicious is not the same as vulnerable. A package can have no CVE and still steal credentials on install. Ask how the platform detects and blocks malware, not only how it lists vulnerabilities.
Regulatory demand is real but moving. The EU Cyber Resilience Act requires SBOMs for products with digital elements from 11 December 2027, while US federal attestation became optional in January 2026.
The platform holds keys to your source and pipelines. It needs broad access to source control, CI and registries. Treat those integrations as high-privilege non-human identities.

Define requirements

Ecosystems: list package managers (npm, PyPI, Maven, NuGet, Go modules, crates), container base images, operating system packages and any vendored or copied code. Coverage of lockfiles and transitive dependencies per ecosystem is the first filter.
Toolchain: source control platforms, CI systems (including self-hosted runners), artifact registries and deployment targets such as Kubernetes clusters, where admission policy would be enforced.
Producer or consumer: if you ship software, you need SBOM, VEX and provenance output per release; if you mainly buy software, you need to ingest suppliers' SBOMs and VEX and match them to new vulnerabilities. Most organisations are both.
AI components: if you consume models, datasets or agent frameworks, ask whether they are inventoried too. See the AI Supply Chain and AI-BOM Guide.
Existing controls: you may already have SCA in an application security testing suite or secrets scanning in your SCM. Decide whether you need replacement, consolidation or only the missing layers. See the Open Source Security Guide.

Size changes the answer. A small team on one SCM and one cloud often gets most value from native SCM features plus a package proxy and image signing. A large organisation with many ecosystems and regulated products needs central policy, SBOM storage across thousands of releases, supplier SBOM ingestion and reporting by product line.

Driver	What it asks for	Status (as of October 2026)
EU Cyber Resilience Act, Regulation (EU) 2024/2847	Annex I Part II(1): identify and document components "including by drawing up a software bill of materials in a commonly used and machine-readable format covering at the very least the top-level dependencies"; SBOM in technical documentation, provided to market surveillance authorities on reasoned request	In force 10 December 2024; reporting of actively exploited vulnerabilities from 11 September 2026; main obligations from 11 December 2027
US federal software procurement	OMB M-26-05 (23 January 2026) rescinded M-22-18 and M-23-16; use of the CISA secure software attestation Common Form is now optional, and "agencies may continue to require SBOMs or other artifacts", according to Wiley's analysis	Risk-based, agency by agency; check each contract
Customer and sector requirements	SBOM and VEX on request, SSDF-aligned development evidence (NIST SP 800-218)	Contractual; varies by customer

Evaluation criteria

Weight these to your requirements. An illustrative split is 25% dependency and malicious package protection, 20% pipeline and repository posture, 20% provenance and admission enforcement, 15% SBOM, VEX and reporting, 20% platform security, integration and cost.

Criterion	What good looks like	Weak signs
SBOM generation and management	SPDX and CycloneDX output generated at build time for every release, stored and versioned; ingestion of supplier SBOMs; search for "where do we use component X?"	One-off SBOMs from source scans only; no storage or query
VEX	Produce and consume VEX with standard statuses (`not_affected`, `affected`, `fixed`, `under_investigation`) and machine-readable justifications, as in the OpenVEX specification	Free-text "not exploitable" notes that cannot be exported
Dependency vulnerabilities (SCA)	Transitive coverage from lockfiles, reachability analysis, fix version guidance, multiple advisory sources	Manifest-only scanning; one advisory feed
Malicious package detection	Blocking before install through a registry proxy or package firewall; behavioural analysis of install scripts; known-malicious feeds such as the OpenSSF Malicious Packages repository (OSV format); typosquat and new-package checks	Detection only after the package is already in a build
Repository and pipeline posture	Checks for branch protection and review, workflow changes owned in CODEOWNERS, least-privilege CI tokens, actions pinned to a commit SHA, risky triggers such as `pull_request_target`, and self-hosted runner exposure, in line with GitHub's Actions hardening guidance	Posture limited to one SCM; findings with no owner or fix path
Provenance and signing verification	Verifies SLSA provenance and Sigstore signatures against an expected builder, repository and signer identity; reports the SLSA Build level actually achieved	"Signed" means only that a signature exists, with no identity check
Admission and deployment policy	Enforces signature and attestation policy at deploy, with audit mode, exceptions that expire, and digest pinning	Policy only in a dashboard, never at a gate
Secrets in code and CI	Scans commits, history, CI logs and images; validity checks; push-time blocking	History never scanned; CI logs ignored
Artifact registry security	Proxy and curation of upstream registries, immutable tags, retention, publishing through OIDC rather than long-lived tokens	Direct pulls from public registries with no record
Open source risk scoring	Uses OpenSSF Scorecard checks individually (for example Branch-Protection, Code-Review, Dangerous-Workflow, Token-Permissions, Signed-Releases, Maintained) alongside maintainer and release activity	A single opaque risk score
Reporting and exit	SBOM and VEX export per product and release, evidence packs for CRA technical documentation, full API, data export at contract end	PDF-only reports; SBOM history locked in

Two caveats are worth putting in writing. Scorecard's maintainers say "The checks themselves are heuristics; there are false positives and false negatives", and that the aggregate score "tells you nothing about what individual behaviors a repository is or is not doing", so a platform should show the checks, not just the number. And for provenance, SLSA Build L2 requires a hosted build platform that signs provenance, while L3 adds hardened builds that keep signing secrets away from user-defined build steps; ask vendors which of these they verify, and see the SLSA and Build Provenance Guide. Verification follows the Sigstore model: the signature is valid, the certificate identity matches the expected identity, the chain reaches Sigstore's root and the entry is in the Rekor transparency log.

Where identity fits

To assess posture the platform needs read access to every repository, workflow file, branch protection setting and registry, and some products ask for write access to open fix pull requests or change settings. That makes it a high-value target holding high-privilege tokens. Require an SCM app integration with fine-grained, read-only permissions by default, short-lived installation tokens and repository selection; keep any write capability in a separate, narrowly scoped integration; and give each connector a named owner. Apply the same standard to the pipelines it protects: GitHub recommends OIDC so workflows can "stop storing these credentials as long-lived secrets", and npm says "we strongly recommend restricting traditional token-based publishing access" once trusted publishing is in place. The tj-actions/changed-files compromise used a stolen bot token, the Shai-Hulud npm worm spread through stolen publishing tokens, and the LiteLLM PyPI breach followed a poisoned security scanner leaking the project's publishing token: security tooling in CI is part of the attack surface. See the CI/CD Pipeline Identity Security Guide.

Questions to ask vendors

Which ecosystems and lockfile formats do you support for transitive dependencies, and how do you handle vendored code and container layers?
How do you detect a malicious package with no CVE, and can you block it before install rather than alert afterwards? How quickly do new detections reach customers?
Which SBOM formats and versions do you generate and ingest, and can we query every product that contains a given component version?
Can we author and import VEX statements with standard statuses and justifications, and do they suppress findings across all affected releases?
Show provenance verification failing for an artifact built by the wrong workflow, and a signature failing for the wrong signer identity.
Which policies can be enforced at Kubernetes admission or deployment, how is audit mode run, and how are exceptions approved and expired?
Which permissions does each SCM, CI and registry integration need, which are write permissions, and can they use short-lived app tokens or OIDC instead of personal tokens?
How would you help us produce SBOM and vulnerability handling evidence for CRA technical documentation?
What does the licence count (developers, repositories, components, products or scans), and what happens to cost as we add products and releases?

Red flags

"SBOM management" that is only a download button, with no storage, versioning or query.
Malicious package detection limited to matching names already published in advisories.
Integrations that require organisation owner or administrator rights, or long-lived personal tokens.
Signature checks that do not pin the expected signer identity or builder.
Posture scores without the underlying checks, or checks you cannot tune.
No export of SBOM, VEX and findings history, or claims of "CRA compliance" as a product feature.

Proof of concept

Scope it: two or three products covering your main ecosystems, one CI system, one registry and one Kubernetes cluster. Agree weights and pass criteria before testing.
Connect with least privilege: install each platform through a dedicated read-only app identity limited to trial repositories, record every permission requested, and never use an administrator's personal token.
Test malicious package blocking: in a disposable sandbox with install scripts disabled, request a package recorded in the OpenSSF Malicious Packages repository and a typosquatted name, and record whether each is blocked, alerted or missed.
Test SBOM and VEX: generate SBOMs for two releases, import one supplier SBOM, mark a finding not_affected with a justification, and query which releases contain a chosen component.
Test provenance and admission: deploy a correctly signed image, an unsigned image and one signed by an unexpected identity, with policy in audit mode first, then enforce. Note that some admission controllers cache verification results (Kyverno's default is 60 minutes), so time your tests.
Test posture findings: seed a workflow with an unpinned action, a write-all token and a pull_request_target trigger, and measure what is flagged and how the fix is explained.
Test exit and roll back: export SBOMs, VEX and findings, switch admission policy back to audit, revoke every integration token and confirm the platform loses access.

Keep measuring after go-live: share of releases with a stored SBOM and verified provenance, malicious packages blocked before install, percentage of deployments passing signature policy, workflows with least-privilege tokens and pinned actions, and age of open exceptions.

How NHI Mgmt Group can help

We provide independent requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.

Software Supply Chain Security Buyer’s Guide