Container Image Security Guide: Base Images, Scanning and Signing

A container image is a bundle of someone else's operating system, libraries and build decisions that you run with your credentials, and most teams cannot say exactly what is inside the images they deploy or who built them. Image vulnerabilities, secrets baked into layers, poisoned public images and mutable tags all reach production through the same path: build, registry, deploy. This guide goes deeper than the Container Security Guide on that path: how OCI images are put together, choosing and pinning base images, distroless and hardened images, scanning and analysis, signing and provenance, registry controls and enforcement at admission in Kubernetes. It is written for platform engineers, DevSecOps teams and security architects responsible for container supply chains.

Key takeaways

• Deploy by digest, not tag: Kubernetes documentation states that "Tags can be moved to point to different images, but digests are fixed"; a digest is the only reference that guarantees you run what you scanned and signed.
• The base image is most of your attack surface: choose minimal or distroless bases from a small curated catalogue, pin them by digest and rebuild on a schedule so fixes actually arrive.
• Layers never forget: deleting a file in a later layer only hides it, so secrets copied into any layer remain retrievable by anyone who can pull the image.
• Scanning without enforcement is reporting: scan at build, keep rescanning in the registry, and use admission control to block images that are unsigned, unprovenanced or from unapproved registries.
• Registry push rights are supply chain keys: ATT&CK T1525 Implant Internal Image describes adversaries planting backdoored images in your own registry; restrict push to pipeline identities and make release tags immutable.

Anatomy of an OCI image

Docker and most other tools build images to the Open Container Initiative (OCI) image specification, so the same structure applies whichever builder or registry you use, and it explains most of the controls below.

Deletions are recorded with whiteout files: the layer specification describes a whiteout as "an empty file with a special filename that signifies a path should be deleted", and it applies only to lower layers. The earlier layer blob is unchanged, so a credential added in one RUN step and removed in the next is still in the image. That is why container image analysis must unpack every layer, not just the final filesystem.

Image and registry risks

NIST SP 800-190 groups image risks into five categories and registry risks into three. They remain the clearest checklist for container image security:

Two ATT&CK v20 techniques describe how attackers use these weaknesses. Container image poisoning inside your estate is T1525 Implant Internal Image (Persistence): a backdoored image pushed to an internal registry and then deployed. T1204.003 Malicious Image (Execution) relies on someone running a backdoored public or third party container image, sometimes named to look legitimate; ATT&CK lists TeamTNT as using malicious Docker images. Both list M1045 Code Signing and M1047 Audit as mitigations.

Choosing and pinning base images

The base image is the largest single input to every image you build, and its vulnerabilities become yours. Decide centrally which bases teams may use rather than letting each Dockerfile choose.

• Distroless images: the open source distroless project publishes images that "do not contain package managers, shells or any other programs", which it says "improves the signal to noise of scanners". Its smallest static image is about 2 MiB, against about 5 MiB for Alpine and 124 MiB for Debian. Use the nonroot variants in production and keep the debug variants, which add a BusyBox shell, out of it. The images are keyless-signed with cosign, so verify them before building on them.
• Curated base image catalogue: SP 800-190 asks organisations to "maintain a set of trusted images and registries". In practice this is a small set of hardened base images (one or two per language runtime), rebuilt by a platform team on a fixed cadence and on critical fixes, published to an internal registry with signatures and SBOMs.
• Pinned base image: reference the base by digest in the FROM line, with the tag kept for readability, and let dependency automation propose digest updates as pull requests. Pinning without automated bumps simply freezes vulnerabilities in place.
• Record the lineage: OCI annotations such as org.opencontainers.image.base.name, org.opencontainers.image.source and org.opencontainers.image.revision let scanners and responders attribute a finding to the base or to the application layer.

Building a hardened container image

The Container Security Guide covers Dockerfile basics (non-root USER, multi-stage builds, build secret mounts). The failures that still reach production are mostly about what enters the build context:

• Control the build context: maintain a .dockerignore that excludes .git, .env files, credentials, caches and compiled artefacts. In the PyPI admin token leak, an uncommitted token survived in a compiled .pyc file because the ignore file did not exclude __pycache__, and the image stayed public for about 15 months.
• Never let the build path escape the repository: in the Smithery.ai hosting breach, an unvalidated build path made the build machine's home directory the build context, exposing a registry token that also controlled the hosting platform.
• Build from clean checkouts in CI, never from developer workspaces, so local files and tokens cannot slip in.
• Ship only the runtime: copy the built artefact into a distroless or minimal final stage; no compilers, package managers, shells or test tooling.
• Set safe defaults in the config: a numeric non-root user, no secrets in ENV, and a filesystem layout that works with a read-only root filesystem at runtime.

Image scanning and container image analysis

Container image scanning tools mostly identify software from package manager metadata and known file signatures, then match versions against vulnerability data. Two consequences follow: binaries copied in without package metadata can be invisible, and every package in the base counts whether or not your application uses it. Container image analysis should therefore cover more than CVEs: secrets in every layer, configuration defects, malware and licences.

• Prioritise, do not just count: fixable findings in running images, on known exploited vulnerability lists, in internet-facing workloads come first. Attribute each finding to the base or application layer so the right team fixes it.
• Use exploitability statements: VEX (Vulnerability Exploitability eXchange) documents let a producer state that a component is present but not affected, which cuts noise without hiding the finding.
• Docker image scanning in the registry only helps if results reach the image owner and admission policy.

Generate an SBOM at build time and attach it to the image so the question "which running images contain this library?" takes minutes. See the SBOM Guide and the Vulnerability Management Guide.

Image signing, provenance and signature validation

SP 800-190 asks for images to be identified "discretely by cryptographic signature" and for signatures to be validated before execution. Container image signing proves who published a digest and container image provenance proves how it was built; together they establish container image integrity.

• Image signing with Sigstore cosign: keyless signing binds the signature to a short-lived certificate issued for the CI workload's OIDC identity, so there is no long-lived signing key to steal. Key-based signing with a key in a cloud KMS or HSM is the alternative where keyless is not acceptable.
• What verification checks: cosign verifies the signature, checks the certificate against the expected --certificate-identity and --certificate-oidc-issuer, validates "that this digest matches the container", and can confirm the entry in the transparency log. Pin both identity and issuer: verifying "any valid signature" accepts anything anyone signed.
• Image provenance: SLSA provenance (predicate type https://slsa.dev/provenance/v1) records the builder, build type, external parameters and resolved dependencies. The SLSA specification says "Consumers MUST accept only specific signer-builder pairs" and that external parameters "MUST be verified downstream", for example that the source repository and branch are the ones you expect. See the SLSA and Build Provenance Guide.
• Attestations travel with the image: SBOMs, scan results and provenance are attached as attestations referring to the image digest and checked with cosign verify-attestation.
• Container image signature validation at deploy time is the control that matters; signatures that nothing checks add no protection. See the Code Signing Security Guide.

Container registry security

• Authenticate every pull of proprietary images and use TLS only; SP 800-190 says "All access to registries that contain proprietary or sensitive images should require authentication."
• Separate push from pull: only pipeline identities push to release repositories, and SP 800-190 limits developers to pushing to repositories they are responsible for. Clusters get pull-only credentials scoped to the repositories they need.
• Remove image tag mutability: Docker Hub's immutable tags setting (in beta) makes tags that "cannot be overwritten or deleted", either for all tags or for tags matching a pattern; cloud registries offer equivalent settings. Leave mutable only tags that are meant to float, such as development builds.
• Prune stale images with retention rules, keeping only versions that are supported or needed for rollback.
• Proxy public images: pull public and third party container images through a pull-through cache or mirror that scans and records them, and block direct pulls from public hubs at the network or admission layer.
• Watch for shadow registries: the Docker Hub secrets leak found 10,456 public images exposing keys, several from personal or contractor accounts outside corporate monitoring. Require corporate namespaces and monitor public registries for your organisation's names.

Enforcing image policy in Kubernetes

• Reference images by digest: Kubernetes advises "You should avoid using the :latest tag when deploying containers in production". With a digest, the default imagePullPolicy is IfNotPresent and the node runs exactly the scanned content.
• Admission policy: ValidatingAdmissionPolicy evaluates CEL rules inside the API server with no external call, suitable for rules such as "images must come from approved registries and include a digest". Signature and attestation verification needs a validating admission webhook backed by a policy engine. The built-in ImagePolicyWebhook is disabled by default and its defaultAllow setting "determines behavior if the webhook backend fails"; set it to fail closed for production namespaces.
• Private image access on shared nodes: the AlwaysPullImages admission controller "modifies every new Pod to force the image pull policy to Always", so a pod cannot reuse a private image cached by another tenant. The kubelet feature KubeletEnsureSecretPulledImages addresses the same problem by re-validating pull credentials for cached images, with imagePullCredentialsVerificationPolicy set to NeverVerifyPreloadedImages or AlwaysVerify.
• Roll out in audit mode first: run policies as warn or audit, review violations by namespace, then enforce, keeping a documented break-glass exemption for incident fixes.

Registry credentials, imagePullSecrets (which pods can only reference in their own namespace), push tokens and signing identities are all non-human identities. Prefer workload identity federation over static registry passwords. See the Kubernetes NHI Security Guide.

Detection and measurement

• Registry audit logs: alert on pushes by any identity other than the pipeline, tag overwrites or deletions on release repositories, and new repositories created outside automation. ATT&CK detection DET0334 for T1525 centres on implanted images pushed to a registry and then deployed.
• Admission decisions: log denials and warnings, and investigate any deployment of a digest that has no signature or provenance.
• Runtime: ATT&CK analytic AN0691 for T1204.003 flags containers from untrusted images that soon spawn unexpected utilities or make unusual outbound connections, correlated by image digest.

Metrics that show the programme is working:

• Share of running workloads referenced by digest and verified at admission.
• Share of running images built on an approved base, and median age of base images in production.
• Critical and known exploited vulnerabilities in running images, and mean time from fix availability to redeploy.
• Images in registries with detected secrets (target zero), and time to revoke.
• Pushes to release repositories by non-pipeline identities (target zero).

Implementation roadmap

1. Inventory: list running images by digest, their registries, base images and owners. This usually reveals unmanaged public images.
2. Lock down registries: authenticated pull, pipeline-only push, immutable release tags, retention rules and a pull-through proxy for public images.
3. Publish curated bases for the main runtimes, signed with SBOMs, and migrate the highest-risk services first.
4. Add build controls: .dockerignore standards, secret scanning of built images, SBOM and provenance generation, keyless signing in CI.
5. Enforce at admission in audit mode for a few weeks (approved registries, digests, valid signatures), fix violations, then enforce per namespace. Rollback is switching the policy back to audit.
6. Automate rebuilds: digest bump pull requests for bases and scheduled rebuilds so fixes reach production without manual effort.

Decision guidance

• Distroless or minimal: choose distroless for compiled languages and well-supported runtimes; choose a minimal distribution where the application needs system packages or a shell for operations you cannot remove.
• Build or adopt curated bases: small teams should adopt well-maintained, signed open source bases and pin them by digest; large estates benefit from an internal catalogue with a platform team owning rebuilds.
• Keyless or key-based signing: keyless removes key management and ties signatures to CI identity; key-based suits air-gapped environments or where a public transparency log is not acceptable.
• Blocking thresholds: block on secrets, missing signatures and known exploited vulnerabilities from day one; treat other severities as SLA-driven tickets until noise is under control.

Framework mapping

Common failure patterns

• Deploying mutable tags: the image that runs is not the one that was scanned or approved, and rollback is unpredictable.
• Secrets "removed" in a later layer: the credential remains in the earlier layer, as the PyPI and Docker Hub cases show, and deleting the image does not help anyone who already pulled it. Revoke the credential.
• Pinned but never updated: bases pinned by digest years ago, with no automation to bump them, so base image vulnerabilities accumulate.
• Signing without verification: CI signs every image but admission policy is absent or checks only that some signature exists.
• Broad registry push rights: developer and CI tokens that can overwrite any repository, the precondition for T1525.
• Debug images in production: debug or full-distribution variants deployed "temporarily" and never replaced.

Practitioner checklist

• Maintain a curated catalogue of signed minimal or distroless base images, rebuilt on a fixed cadence.
• Pin bases and deployments by digest, with automation that proposes digest updates.
• Enforce .dockerignore standards and build only from clean CI checkouts.
• Scan every layer of built images for vulnerabilities, secrets and malware, and rescan registries daily.
• Generate and attach SBOM and SLSA provenance attestations at build.
• Sign images keylessly in CI and verify identity, issuer and digest at admission.
• Restrict registry push to pipeline identities, enable immutable release tags and prune stale images.
• Route public and third party images through a scanning proxy; block direct pulls.
• Enforce approved registries and digests with admission policy, after an audit-mode period.
• Alert on non-pipeline pushes, tag overwrites and unknown digests running in clusters.

Standards and references

• NIST: SP 800-190 Application Container Security Guide
• Open Container Initiative: Image Manifest Specification
• Open Container Initiative: Distribution Specification
• Kubernetes: Images
• Sigstore: Verifying Signatures with Cosign
• SLSA: Provenance
• MITRE ATT&CK: T1525 Implant Internal Image
• Docker: Immutable Tags

Related NHI Mgmt Group resources: Container Security Guide · Kubernetes Security Guide · Software Supply Chain Security Guide · SBOM Guide · Secrets Management Guide