Most container estates scan images somewhere, yet the same CVEs stay in production for months because nobody owns the step from finding to rebuilt, redeployed image. A container image is immutable, so you do not patch it: you rebuild it from an updated base and updated dependencies, push a new digest and roll the workload forward. That changes how vulnerability management has to work compared with servers. This guide covers the process end to end: where to scan (pipeline, registry and runtime), how to prioritise findings, how to run rebuild workflows for base images and application layers, and how to measure whether the loop is closing. It is written for platform, DevSecOps and vulnerability management teams who already know how to build an image and now need a process that scales across hundreds of repositories.
Key takeaways
- Rebuild, never patch in place. A fix only counts when a new image digest built from a fixed base or dependency is running, so measure time to redeploy, not time to ticket.
- Scan at three points with different jobs. The pipeline scan gates new builds, the registry scan catches new CVEs in images already built, and runtime inventory tells you which findings are actually running.
- Prioritise on exploitation and reachability, not counts. CISA KEV listing, EPSS and whether the package is present in the final runtime image matter more than raw CVSS severity.
- Split ownership by layer. The platform team owns base images and their rebuild cadence; application teams own the dependencies they add and must pick up new base digests promptly.
- Record decisions as VEX, not as silent ignores. A not_affected statement with a justification survives audits and scanner changes; a local ignore file does not.
Why images need their own vulnerability process
NIST SP 800-190, the Application Container Security Guide, frames containers as operating system virtualisation combined with application packaging, and it treats image vulnerabilities as a risk of the image itself rather than of the host. Three properties make the server playbook a poor fit:
- Immutability: running
apt upgradeinside a container changes nothing durable. The next restart, scale-out or node replacement starts the original vulnerable image again. Remediation means a new build. - Layered ownership: one image carries an operating system base (often from a public registry), runtime layers such as a JRE or Python interpreter, and the application's own packages. Each layer has a different owner and update source.
- Copies everywhere: the same vulnerable base can sit under hundreds of images, many tags per repository, and old digests that still run in a forgotten namespace. A finding count per image hugely overstates the work and hides where the single fix belongs.
The aim of the process is therefore to fix each vulnerability once, at the lowest layer that contains it, and then propagate the fix through automated rebuilds. For choosing and hardening the base itself, see the Container Image Security Guide and the Hardened Container Images buyer's guide; this page focuses on the operational vulnerability loop.
Where to scan: pipeline, registry and runtime
Each scanning point answers a different question. Running only one leaves a predictable gap.
| Scan point | Question it answers | What it misses on its own | Typical action on a finding |
|---|---|---|---|
| Pipeline (CI build) | Is this new image acceptable to publish? | CVEs disclosed after the build; images built elsewhere | Fail or warn the build against a policy threshold |
| Registry (scan on push plus continuous rescan) | Which stored images are affected by newly published CVEs? | Whether the image is running anywhere | Open a rebuild ticket or trigger an automated rebuild |
| Runtime inventory | Which vulnerable digests are actually deployed, where, and how exposed? | Images not yet deployed; build-time context | Prioritise by exposure; drive redeploy of the fixed digest |
| Admission control | Should this digest be allowed to start? | Workloads already running before the policy | Block or warn on images above threshold or unscanned |
Pipeline scanning
Scan the built image, not only the source tree, because the final image includes the OS packages from the base and anything copied in by a multi-stage build. Open source scanners such as Grype and Trivy (listed alphabetically) both scan images directly. Grype can also scan a Syft SBOM (grype sbom:./sbom.json), supports OpenVEX to filter and augment results, and uses EPSS, KEV and risk scoring for prioritisation. Trivy checks OS packages against the OS vendor's own advisories and language packages against ecosystem sources such as the GitHub Advisory Database, prefers vendor severity over NVD, and offers --ignore-unfixed to hide vulnerabilities with no available fix and --detection-priority (precise by default, or comprehensive). Whichever tool you use, generate the SBOM at build time and store it with the image so later rescans and incident lookups do not need to pull and unpack every image again. See SBOM operations.
Registry scanning
New CVEs are published every day against packages already in your images, so the registry needs continuous rescanning, and each managed registry defines "continuous" differently. Know the window, because images outside it silently stop getting new findings. As described in each provider's documentation in November 2026 (providers listed alphabetically):
| Registry service | Scanning feature | How rescanning is scoped | Setting to check |
|---|---|---|---|
| Amazon ECR | Enhanced scanning with Amazon Inspector | Scan filters set scan on push or continuous scanning per repository; Inspector's push date duration and re-scan duration control how long images stay eligible; expired images show SCAN_ELIGIBILITY_EXPIRED | Repositories matching no filter have scan frequency Off; when first enabled, images pushed in the last 14 days are recognised |
| Azure Container Registry | Defender for Containers vulnerability assessment (Microsoft Defender Vulnerability Management) | New pushes and imports scanned within a few hours; daily rescan for images pushed or pulled in the last 30 days or running in a monitored Kubernetes cluster | Turn on Registry access; pulling an inactive image reactivates scanning |
| Google Artifact Registry | Artifact Analysis automatic scanning | Scanned once per digest on push; continuous analysis while the image was pulled in the last 30 days; metadata stale for over 90 days is archived | Turn on scanning per repository (off by default for Artifact Registry repositories); re-pull to refresh |
The practical consequence is the same everywhere: an old digest still running in production but not recently pushed or pulled may fall outside the rescan window. Runtime inventory closes that gap. Protect the registry itself as well: see the Container Registry Security Guide.
Runtime inventory
Runtime data turns a list of vulnerable images into a list of vulnerable workloads. Collect the digest (not the tag) of every running container, the cluster and namespace, whether the workload is internet-facing, and its service account privileges. Managed services increasingly provide this mapping: Amazon ECR enhanced scanning reports when an image was last used on Amazon EKS or Amazon ECS and how many clusters use it, and Defender for Cloud maps registry-scanned images to running workloads. A finding in an image that has not run for six months is a clean-up task; the same finding on an internet-facing pod is an incident in waiting.
Prioritising findings
A typical general-purpose base image produces dozens of findings, most of them low severity, unfixed or in packages your application never loads. Treating every finding equally burns the team out and teaches developers to ignore the scanner. Use a small number of explicit factors:
| Factor | Source | How to use it |
|---|---|---|
| Known exploitation | CISA Known Exploited Vulnerabilities catalogue | Top priority regardless of CVSS. BOD 22-01 binds US federal civilian agencies only, but CISA strongly recommends every organisation prioritise KEV remediation |
| Exploit likelihood | EPSS score | Raise priority above an agreed probability threshold; review the threshold quarterly |
| Fix available | Distribution advisory or upstream release | No fix means mitigate, accept with expiry or switch base; it cannot be rebuilt away yet |
| Present at runtime | Final image SBOM, runtime inventory | Build-stage-only packages removed by a multi-stage build are not a runtime risk |
| Exposure | Ingress, network policy, service account privileges | Internet-facing or highly privileged workloads first |
| Severity | Vendor severity, then CVSS | Tie-breaker, not the primary sort; distribution severity often differs from NVD |
For the scoring models themselves, see the EPSS Guide, the CISA KEV Catalogue Guide and the Vulnerability Prioritisation Guide.
Recording decisions with VEX
When a finding is not exploitable in your image, record why in a machine-readable VEX document rather than a scanner-specific ignore list. OpenVEX (v0.2.0) defines four statuses: not_affected, affected, fixed and under_investigation. A not_affected statement carries one of five justifications: component_not_present, vulnerable_code_not_present, vulnerable_code_not_in_execute_path, vulnerable_code_cannot_be_controlled_by_adversary or inline_mitigations_already_exist. An affected statement needs an action_statement describing remediation or mitigation. Give every statement an owner and a review date, and keep VEX documents in version control next to the Dockerfile. See the VEX Guide.
Rebuild workflows
Rebuilds are the remediation. Design them so that a fixed base image reaches every dependent workload without a human opening each repository.
Base image rebuilds (platform team)
- Maintain a small set of approved bases: for example one minimal OS base, one per language runtime and one distroless or static variant. Fewer bases means fewer places to fix each CVE.
- Rebuild on a schedule and on trigger: rebuild every approved base at least weekly to pick up distribution updates, and immediately when a KEV-listed or critical fixed CVE affects it.
- Publish by digest with a moving tag: push the new base under an immutable version tag and update a channel tag such as
stable. Record the digest, SBOM and build provenance for each release. - Notify dependants automatically: dependency bots (for example Dependabot or Renovate) can propose pull requests that update
FROMlines pinned by digest. Each pull request runs the application's own tests.
Application image rebuilds (service teams)
- Pin bases by digest:
FROM registry.example/base/python@sha256:...makes builds reproducible and makes base updates visible as reviewed changes. Kubernetes documents digests as immutable while tags can be moved to point to different images. - Accept base update pull requests within the agreed window: merge, build, scan, deploy. The pipeline gate checks the new image did not introduce new critical findings.
- Update application dependencies separately: language package fixes come from the application's lock file, not the base. Track them as their own pull requests.
- Roll forward, keep rollback by digest: deploy the new digest through the normal progressive rollout. If it fails, roll back to the previous digest and treat the vulnerability as an open exception with an expiry, not as closed.
Deployment hygiene that makes rebuilds stick
- Reference digests in manifests: deploying by digest guarantees that what you scanned is what runs. If you deploy by tag, know the pull policy: Kubernetes sets
imagePullPolicytoAlwayswhen the tag is:latestor omitted and toIfNotPresentfor other tags or a digest, so a re-pointed version tag may never be pulled on nodes that already cache it. - Avoid mutable tags for third-party tooling: the Checkmarx KICS supply chain attack pushed malicious images under existing tags including
latest; pipelines that pinned the scanner image by digest did not pull them. Pin your scanners and build tools the same way as your bases. - Retire old digests: registry lifecycle rules should remove untagged and superseded images after a retention period, otherwise old vulnerable digests remain deployable. Note that deleting only a tag may not delete the underlying manifest.
Policy gates and exceptions
Set gates that developers can meet. A gate that blocks on every high severity finding in an unfixable OS package will be bypassed within a week.
| Condition | Pipeline | Admission | Remediation window (example) |
|---|---|---|---|
| KEV-listed CVE with a fix available | Block | Block new deploys | Rebuild and redeploy within 7 days of KEV listing |
| Critical, fix available, present at runtime | Block | Warn | 14 days |
| High, fix available | Warn | Allow | 30 days, normally via the next base rebuild |
| No fix available | Warn | Allow | Documented exception with expiry; reassess when a fix ships |
| Image not scanned or no SBOM | Block | Block | Immediate |
The windows above are illustrative starting points; set your own in the vulnerability management policy (see the Vulnerability Management Policy Template). Admission control should verify that the image was scanned and signed, not rescan it on every pod start; see the Container Runtime Security Guide and the Sigstore Guide.
Exceptions need an owner, a justification (ideally the VEX justification), compensating controls such as a network policy or read-only root filesystem, and an expiry date after which the gate applies again.
Where identity fits
The rebuild loop runs on non-human identities: CI service accounts that push to the registry, bot tokens that open base update pull requests, and node or workload credentials that pull images. A registry push credential with rights over the approved base repositories can poison every dependent image at the next rebuild, which is the attack described by MITRE ATT&CK T1525 Implant Internal Image. Give base image publishing its own short-lived, narrowly scoped identity (federated OIDC from the build system rather than a static token), keep pull credentials read-only, and never bake credentials into images. See the Kubernetes Secrets Security Guide and the Secrets in Container Images and Build Artefacts Guide.
How to implement
- Inventory first (weeks 1 to 2): list every registry and repository, and collect running digests from every cluster. Map each image to its base and owning team. Expect to find bases nobody approved.
- Turn on registry scanning everywhere (weeks 2 to 3): enable continuous scanning on all repositories, and note each service's rescan window so runtime inventory can cover what it misses.
- Add pipeline scans in warn mode (weeks 3 to 6): scan every built image, publish the SBOM, report findings without blocking. Use this period to tune thresholds and gather baseline metrics.
- Publish approved bases and the rebuild cadence (weeks 4 to 8): stand up the platform team's base pipeline, digest pinning and automated update pull requests. Migrate the highest-traffic services first.
- Enforce gates in rings (from week 8): start blocking KEV and unscanned images in pipelines for one team, then all teams. Add admission enforcement in audit mode, review what it would have blocked, then enforce per namespace.
- Close the loop: connect findings to tickets by owning team, track exceptions with expiry, and review metrics monthly.
Rollback: keep every gate behind a configuration flag so it can return to warn mode within minutes, and keep the previous image digest deployable. Never roll back by disabling scanning.
Decision guidance
- Managed registry scanning or a standalone scanner: managed scanning needs no pipeline changes and covers stored images, but each service has its own rescan window and data sources. A standalone scanner in CI gives consistent results across clouds and registries. Larger estates usually run both, and accept that counts will differ between them.
- General-purpose or minimal bases: minimal and distroless bases cut findings sharply but remove the shell and package manager that some teams use for debugging. Offer a debug variant for non-production rather than allowing full bases in production.
- Small organisations: one approved base per language, managed registry scanning, a weekly scheduled rebuild and a single KEV gate will cover most of the risk. Skip admission control until the pipeline gate is stable.
- Large organisations: a dedicated platform team for bases, digest-pinned bases with automated update pull requests, VEX shared across teams, and runtime-informed prioritisation are needed, because finding volume otherwise overwhelms triage.
Detection and measurement
Monitor scanner and registry events, not only dashboards: registry scan completion and new-finding events (for example Amazon Inspector emits events when a finding is created, updated or closed), admission denials, and deployments of digests that have no scan record. Useful metrics:
- Time to redeploy: median days from a fixed CVE being available in the approved base to the new digest running in production, per severity.
- Base freshness: percentage of running workloads built on the current approved base digest, or one release behind.
- KEV exposure: count of running containers with a KEV-listed CVE that has a fix, which should trend to zero.
- Scan coverage: percentage of running digests with a scan inside the last 7 days and a stored SBOM.
- Exception health: open exceptions, and the number past expiry.
- Unapproved bases: number of images built from bases outside the approved set.
For broader programme metrics, see Vulnerability Management Metrics.
Common failure patterns
- Patching running containers: fixes applied with a shell inside a pod disappear at the next restart, and the scan record still shows the old digest as vulnerable.
- Counting findings per image: the same base CVE reported across 300 images reads as 300 tasks. Group by base digest and package so the fix is made once.
- Trusting the registry rescan window: long-running images that are neither pushed nor pulled drop out of continuous scanning and look clean because nothing new is reported.
- No SBOM per image: when a library compromise such as the XZ Utils backdoor is disclosed, teams without stored SBOMs must pull and unpack every image to find affected builds, which takes days instead of hours.
- Silent ignores: local ignore files with no justification or expiry accumulate until real findings are suppressed too.
- Gates nobody can pass: blocking on unfixable findings leads to gate bypasses that then also let KEV findings through.
Framework mapping
| Framework | Reference | How this process meets it |
|---|---|---|
| NIST CSF 2.0 | ID.RA-01, PR.PS-02, DE.CM-09 | Image vulnerabilities identified and recorded; software maintained and replaced by rebuild; runtime environments monitored |
| NIST SP 800-53 Rev. 5 | RA-5, SI-2, CM-2, CM-7, CM-8 | Vulnerability monitoring and scanning; flaw remediation through rebuilds; approved base baselines; minimal images; image and digest inventory |
| CIS Controls v8.1 | Control 7 (Safeguards 7.1, 7.2, 7.5, 7.7) | Documented vulnerability and remediation processes; automated scanning; remediation of detected vulnerabilities |
| ISO/IEC 27001:2022 | Annex A 8.8, 8.9, 8.25 | Management of technical vulnerabilities; configuration management of approved bases; secure development life cycle |
| MITRE ATT&CK | T1525, T1610, T1195.002 | Implant Internal Image; Deploy Container; Compromise Software Supply Chain |
| MITRE D3FEND | D3-CIA | Container Image Analysis |
Practitioner checklist
- Inventory every registry, repository and running digest, and map each image to its base and owner.
- Turn on continuous registry scanning and document each service's rescan window.
- Scan every built image in CI and store its SBOM with the image.
- Publish a small set of approved bases, rebuilt at least weekly and on KEV triggers.
- Pin bases and third-party tool images by digest, with automated update pull requests.
- Prioritise on KEV, EPSS, fix availability, runtime presence and exposure before CVSS.
- Record non-exploitable findings as VEX with justification, owner and review date.
- Gate pipelines on KEV and unscanned images first, then tighten in rings.
- Give base image publishing a short-lived, narrowly scoped identity.
- Report time to redeploy, base freshness and KEV exposure monthly.
Standards and references
- NIST: SP 800-190 Application Container Security Guide
- CISA: Known Exploited Vulnerabilities and BOD 22-01
- OpenVEX: Specification
- Kubernetes: Images
- AWS: Amazon ECR enhanced scanning
- Microsoft: Vulnerability assessment for Azure container images
- Google Cloud: Artifact Analysis container scanning
- MITRE D3FEND: Container Image Analysis
Related NHI Mgmt Group resources: Container Image Security Guide · Container Security Guide · Risk-Based Vulnerability Management Guide · SBOM Guide · Vulnerability Management Guide