Signing and provenance prove who built a binary and how, but not that the binary contains only what the source says; a compromised build system can sign a tampered artefact perfectly. Reproducible builds close that gap: if anyone can rebuild the same source in the same declared environment and get bit-for-bit identical output, then independent rebuilders can check every release, and a binary that differs from its source stands out. Debian, NixOS and the Go toolchain have shown this is achievable at scale. This guide explains what reproducibility means, the sources of nondeterminism and how to remove them, how to run independent verification, where it fits alongside SLSA, signing and SBOMs, and how to decide how far to go. It is for build, release and supply chain security engineers, and it goes deeper than the reproducibility section of the SLSA and Build Provenance Guide.
Key takeaways
- Reproducible means anyone can recreate identical artefacts. The reproducible-builds.org definition requires bit-for-bit identical copies from the same source, build environment and build instructions, verified by comparing cryptographic hashes.
- Reproducibility only pays off with independent rebuilds. The security value comes from a second, separately operated builder comparing hashes, which can detect a compromised build platform that provenance alone would not.
- Most nondeterminism has a handful of causes. Timestamps, file ordering, build paths, locales and timezones, randomness and volatile network inputs account for most differences, and each has a known fix.
- It is achievable at scale. Debian's test infrastructure reported about 95% of packages in its forky and unstable suites reproducible on amd64 and arm64 in October 2026, and Go 1.21.0 was the first perfectly reproducible Go toolchain.
- It complements, not replaces, other controls. Reproducibility proves the binary matches the source; it says nothing about whether the source itself is malicious, as the XZ Utils backdoor showed.
What reproducible builds are
Reproducible Builds, the cross-project initiative behind reproducible-builds.org, defines a build as reproducible if, given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artefacts. Each term matters:
- Source code: usually a version control checkout at a specific revision, or a source archive.
- Build environment: the attributes that affect the output, such as dependencies and their versions, build configuration flags and environment variables like the locale. The project notes that it is preferable to reduce this set.
- Artefacts: the primary outputs, such as executables, distribution packages or filesystem images. Build logs and similar ancillary outputs are usually excluded.
- Relevant attributes: authors or distributors decide which parts of the environment and which artefacts are in scope, and must document them so others can rebuild.
- Verification: bit-by-bit comparison, usually by comparing cryptographic hashes.
| Property | What it guarantees | What it does not |
|---|---|---|
| Signed artefact | The holder of a key published this file | That the file matches the reviewed source |
| Build provenance (SLSA) | Which builder, source revision and parameters produced the artefact, as attested by the builder | That the builder was not compromised |
| Hermetic build | All inputs were declared and pinned; no network access during the build | That two builds give identical output |
| Reproducible build | Identical output from identical inputs, checkable by anyone | That the source is benign |
| Independent rebuild | A separately operated party obtained the same hash | Protection if all rebuilders share the same compromised toolchain |
| Bootstrappable toolchain | The compiler itself can be rebuilt from a minimal, auditable seed | Coverage of firmware and hardware |
SLSA's history shows how the ideas relate. The retired SLSA v0.1 Level 4 required two-person review of all changes and a hermetic, reproducible build process, with reproducibility described as not strictly required but valuable for auditability. The current SLSA versions (v1.2 is current) do not include reproducibility in the Build track levels, which is why it is best treated as an additional control for critical artefacts.
Sources of nondeterminism and how to remove them
The reproducible-builds.org documentation groups the work under managing variance, defining the build environment and verification. The table maps the common causes to fixes.
| Source of variance | Typical symptom | Fix |
|---|---|---|
| Timestamps | Build date embedded in binaries, archives, documentation or version strings | Honour SOURCE_DATE_EPOCH; clamp file timestamps (GNU tar --clamp-mtime) |
| Timezones and locales | Dates formatted differently, sort order changes with locale | Fix TZ=UTC and a known locale in the build environment, or make tools locale-independent |
| Archive metadata | File owners, permissions and modification times differ inside tar, zip or jar files | Normalise owner, group, mode and mtime when creating archives |
| Input and output ordering | Files listed in filesystem order, hash map iteration order | Sort inputs and outputs explicitly; avoid relying on directory listing order |
| Randomness | Random seeds, temporary names or UUIDs embedded in output | Derive any needed values deterministically from the source |
| Build path | Absolute source paths embedded in debug info or error messages | Build in a fixed path or use path-trimming flags (Go -trimpath; compiler prefix-map options) |
| Volatile inputs | Dependencies fetched from the network change or disappear | Pin by digest, vendor or mirror inputs, build without network access |
| Version information | Hostnames, user names or VCS state in version strings | Generate version strings only from the tagged source |
| Host toolchain leakage | Output depends on the host C compiler or dynamic linker | Pin the toolchain in a container or package manager; Go removed this dependency by building the toolchain with cgo disabled |
| Embedded signatures | Signed binaries cannot be rebuilt identically by others | Use detached signatures, or the patterns reproducible-builds.org describes: pasting signatures as a build input, ignoring signature bytes, or stripping signatures before comparison |
SOURCE_DATE_EPOCH
The SOURCE_DATE_EPOCH specification is the single most useful convention. The variable holds a UNIX timestamp in seconds as an ASCII integer. Build tools must use it instead of the current time for embedded timestamps, must not unset it for child processes, must clamp any build-specific timestamps so they are not later than it, and should fail on a malformed value. The value must depend only on the source: distributions use the latest changelog entry (for example debian/changelog), and developers use the date of the latest commit.
Verifying builds independently
Reproducibility is a property; verification is the control. Without a second party rebuilding and comparing, a reproducible build adds no security.
- Record the build environment: publish what is needed to rebuild: exact source revision, toolchain and dependency versions by digest, flags and environment variables. Debian's buildinfo files and lockfile-based ecosystems are examples of this practice.
- Rebuild in a different administrative domain: the rebuilder should use different infrastructure, credentials and ideally a different operating system. The Go team's verification of Ubuntu's Go 1.21.0 packages using only non-Ubuntu tools on a Mac illustrates the point: a compromised packaging tool would not go unnoticed.
- Compare and publish: Go release builds run on both Linux and Windows hosts and must match bit for bit or the release stops. The
gorebuildtool rebuilds releases and compares them with published archives, and results are posted nightly at go.dev/rebuild. Signed installers such as macOS and Windows packages use relaxed checks that compare contents rather than signatures. - Diagnose differences: diffoscope shows exactly where two artefacts differ, recursing into archives and binaries. NixOS documents rebuilding with
--checkor--rebuildand analysing differences with diffoscope. - Do not trust a single double build: NixOS warns that building twice on the same machine cannot prove reproducibility, because a match may be luck. Vary the environment deliberately between builds.
- Act on mismatches: decide in advance what happens when a rebuild does not match: block promotion of the artefact, alert the release owner and investigate as a potential build compromise.
Ecosystem status
| Project | What is tested | Reported status (as fetched October 2026) |
|---|---|---|
| Debian | Packages in bookworm, trixie, forky, unstable and experimental on amd64 and arm64, rebuilt on jenkins.debian.net | forky 95.8% (amd64) and 95.0% (arm64); unstable 94.5% and 94.4%; experimental about 69% |
| NixOS | Minimal and graphical installation ISOs, both runtime and build-time dependencies | Reports published as a general health check; no single percentage on the status page |
| Go toolchain | Every release archive, rebuilt nightly by gorebuild | Perfectly reproducible since Go 1.21.0 (August 2023) |
Read these figures carefully. A distribution-level percentage counts packages, not risk: the remaining few per cent can include large, security-relevant packages, and "reproducible in the test environment" depends on which variations the test applies. For the packages you actually deploy, check the per-package result rather than the headline number.
Trusting the toolchain
Independent rebuilders usually share the same compiler and base system, so a compromised toolchain could produce matching malicious output everywhere. The Go project's observation is useful here: a self-hosting toolchain can only be perfectly reproducible when the source is its sole input, which is what makes cross-platform rebuild checks meaningful. For the highest-assurance artefacts, use rebuilders on different operating systems and hardware architectures, and prefer toolchains that can themselves be rebuilt and verified.
Choosing how far to go
| Artefact type | Recommended target | Reason |
|---|---|---|
| Widely distributed software, OS packages, security agents, firmware, developer toolchains | Fully reproducible with at least one independent rebuilder | A build compromise affects many downstream users and is otherwise very hard to detect |
| Internal services deployed as containers | Hermetic, digest-pinned builds with SLSA provenance; reproducibility for base images and shared libraries | Cost of full reproducibility is high relative to blast radius; provenance gives most of the value |
| Open source libraries you maintain | Reproducible release artefacts built from tagged source, with the source tarball generated from the repository | Lets distributions and consumers verify releases cheaply |
| Third-party binaries you consume | Prefer suppliers that publish reproducibility and rebuild results; rebuild critical ones yourself | Turns supplier claims into something you can check |
Small teams should start with hermetic builds, pinned dependencies and SOURCE_DATE_EPOCH, which remove most variance cheaply, and rebuild only their most important release artefact in a second environment. Large organisations should run a rebuilder service owned by a different team from the main CI platform, with separate credentials, so a single compromise cannot both tamper and verify.
Implementing reproducible builds
- Pick a pilot artefact: one release binary or container base image with real downstream impact.
- Build twice with variation: run two builds on different machines with different time, timezone, locale, user and build path, and compare hashes. Use diffoscope to list every difference.
- Remove variance by category: set
SOURCE_DATE_EPOCH, normalise archives, sort inputs, trim paths, pin the toolchain and dependencies by digest, and disable network access during the build. - Record the environment: publish the toolchain versions, digests and flags with the artefact, alongside SLSA provenance.
- Stand up an independent rebuilder: a separate account or project, different runners, separate credentials and no write access to the release repository.
- Gate on the result: start in report-only mode, then block promotion of artefacts whose rebuild does not match. Keep a documented override for emergencies, with approval and logging.
- Regression-test reproducibility: run the double build in CI on every change so new nondeterminism is caught when it is introduced, not at release.
- Extend: move on to dependencies you build yourself and to base images, then to the toolchain.
Where identity fits: the rebuilder is a machine identity whose trust must be independent of the primary builder. Give it its own workload identity, read-only access to source and published artefacts, and a separate signing key or keyless identity for its attestations, so a stolen CI token cannot forge both the release and its verification. See the Sigstore Guide and the Code Signing Security Guide.
Detection and measurement
- Mismatch alerts: any rebuild that differs from a published artefact is a security event until explained.
- Drift in reproducibility: track when previously reproducible artefacts stop being reproducible, which often signals an unpinned dependency or toolchain change.
- Source-to-release comparison: for consumed open source, compare release tarballs with the tagged repository contents, and flag generated build scripts that exist only in the tarball.
Metrics:
- Percentage of release artefacts that are reproducible (target 100% for critical tiers).
- Percentage of release artefacts independently rebuilt and matched before promotion.
- Number of unexplained mismatches and mean time to resolve them.
- Percentage of builds that are hermetic (no network access, all inputs pinned by digest).
- Percentage of critical third-party components with published, verifiable rebuild results.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev. 5 | SA-10 Developer Configuration Management; SA-10(6) Trusted Distribution; SR-4 Provenance; SR-11 Component Authenticity; SI-7 Software, Firmware, and Information Integrity | Controlled builds, verified distribution, provenance and integrity checking of released software |
| NIST CSF 2.0 | PR.PS-06 Secure software development practices; GV.SC-05 supply chain requirements in contracts | Build integrity in the SDLC; requiring reproducibility or rebuild evidence from suppliers |
| SLSA | Build track (v1.2); retired v0.1 Level 4 (hermetic, reproducible) | Provenance levels that reproducibility complements |
| MITRE ATT&CK | T1195.002 Compromise Software Supply Chain | Tampering with software during build or distribution |
Common failure patterns
- Reproducible but never verified: builds are deterministic, but no independent party rebuilds them, so a compromised builder would still go unnoticed.
- Rebuilder shares the primary's trust: the verification job runs on the same CI platform with the same credentials, so one compromise defeats both.
- Verifying the wrong source: rebuilding from a release tarball rather than the repository. In the XZ Utils backdoor, a modified
build-to-host.m4present in the release tarballs extracted a payload from binary test files during the build, so the code reviewed in git was not the code shipped. Generate release tarballs from the tagged repository and compare them. - Assuming reproducibility means safety: malicious code committed to the source reproduces perfectly. Pair reproducibility with source review, maintainer controls and dependency analysis.
- Unpinned inputs: builds that fetch "latest" dependencies or base images pass today and fail to reproduce next week, eroding trust in the rebuild signal until mismatches are ignored.
- Signatures blocking comparison: embedding signatures in artefacts without a strategy for comparing them, so rebuilders cannot match and the check is quietly dropped.
Practitioner checklist
- Identify the release artefacts where a build compromise would have the widest impact and target them first.
- Honour
SOURCE_DATE_EPOCH, fix timezone and locale, normalise archives, sort inputs and trim build paths. - Pin toolchains, dependencies and base images by digest and build without network access.
- Publish the build environment and SLSA provenance with every release.
- Run an independent rebuilder with separate infrastructure, identity and credentials.
- Block promotion on rebuild mismatch after a report-only period, with an approved emergency override.
- Run a varied double build in CI to catch new nondeterminism early.
- Generate release source archives from the tagged repository and compare them for consumed open source.
- Ask critical suppliers for reproducibility and rebuild evidence.
Standards and references
- Reproducible Builds: Definitions
- Reproducible Builds: SOURCE_DATE_EPOCH specification
- Reproducible Builds: Documentation
- Debian: Reproducible builds test results
- NixOS: Reproducible builds status
- The Go Blog: Perfectly Reproducible, Verified Go Toolchains
- OpenSSF SLSA: Security levels (v0.1)
- NIST: SP 800-53 Rev. 5
Related NHI Mgmt Group resources: SLSA and Build Provenance Guide · SBOM Guide · Sigstore Guide · Linux Package Signing and Repository Trust Guide