Registry pull credentials are the most copied, least owned secrets in a container estate: one docker login on a build host or one imagePullSecret in a namespace can quietly outlive the team, the project and the person who created it. This guide covers the lifecycle of those credentials where they are actually used: Kubernetes Secrets of type kubernetes.io/dockerconfigjson, the imagePullSecrets field, Docker and Podman credential files on hosts, credential helpers, the kubelet image credential provider and federated, short-lived access from clusters to cloud registries. It is written for platform engineers, cluster administrators and secrets owners. Registry-side controls (robot accounts, signing, scanning, immutability) are in the Container Registry Security Guide; this page picks up where the credential leaves the registry. Cloud platforms are listed alphabetically.
Key takeaways
- A pull secret is a credential, not configuration. The value in a
.dockerconfigjsonSecret or aconfig.jsonfile is base64, which the Kubernetes documentation states provides no confidentiality over plain text. - Who can create pods can use the secret. Kubernetes warns that a user who can create a Pod that uses a Secret can also see its value, so RBAC on pod creation matters as much as RBAC on Secrets.
- Prefer fetched over stored. The kubelet image credential provider (stable since v1.26) fetches registry credentials at pull time, and its service account token mode (beta since v1.34) lets each workload's own identity drive the pull.
- Cached images bypass credentials unless you say otherwise. Use the AlwaysPullImages admission controller or the
KubeletEnsureSecretPulledImagesbehaviour (beta and on by default since v1.35) so one tenant cannot run another tenant's private image from node cache. - Every pull credential needs an owner, a scope and an end date. Inventory Secrets, ServiceAccount attachments, node files and host logins together, because they merge at pull time.
Where pull credentials live
A single image pull can draw on several credential sources at once. The kubelet merges imagePullSecrets with any node-level .docker/config.json, and when several entries match an image it tries them in turn. That merging is convenient and is also why stale credentials survive: removing one source rarely stops pulls.
| Location | Format and mechanism | Who can use it | Typical lifetime | Main risk |
|---|---|---|---|---|
Pod spec.imagePullSecrets | References Secrets of type kubernetes.io/dockerconfigjson (or legacy kubernetes.io/dockercfg) in the Pod's namespace | Pods that reference it, plus anyone who can read Secrets or create pods in the namespace | Whatever the registry token allows, often no expiry | Copied between namespaces and clusters by hand |
ServiceAccount imagePullSecrets | Kubernetes adds the listed Secrets to every pod that uses the ServiceAccount | Every workload on that ServiceAccount, including the default account if attached there | As above | Silent spread to unrelated workloads |
Node config.json | Docker-format file read by the kubelet on the node | All pods scheduled on the node | Until someone edits the node image | Baked into machine images and never rotated |
| Kubelet credential provider | Exec plugin named in CredentialProviderConfig, called at pull time | Pulls matching matchImages patterns on that node | Cached for defaultCacheDuration or the duration the plugin returns | Over-broad node identity if the plugin uses node credentials |
Docker host ~/.docker/config.json | auths entries (base64) or pointers to credsStore / credHelpers | The user account, and anything running as it | Until docker logout | Plain-text-equivalent tokens on build hosts and laptops |
Podman, Buildah, Skopeo auth.json | Primary file ${XDG_RUNTIME_DIR}/containers/auth.json on Linux, falling back to ${XDG_CONFIG_HOME}/containers/auth.json, then Docker's config.json and .dockercfg | The user account | Runtime directory is per session; fallbacks persist | Fallback to an old Docker file nobody remembers |
Two details catch teams out. First, when a Docker credential store is configured, config.json holds a credsStore entry rather than an auth entry, so copying the file into a Kubernetes Secret produces a Secret with no usable credential; the Kubernetes documentation directs you to create the Secret from explicit values instead. Second, Kubernetes accepts glob and path-prefix keys in auths (for example *.registry.example, where * matches exactly one subdomain level), while Docker accepts only registry root addresses, so the same file can behave differently on a host and on a node.
Choosing a credential pattern
The Kubernetes images documentation lists five ways to give a cluster access to a private registry: imagePullSecrets on pods, node configuration, a kubelet credential provider plugin, pre-pulled images and vendor-specific extensions. Ranked by how well each one supports least privilege and expiry:
| Pattern | Credential stored in cluster? | Granularity | Choose when | Trade-off |
|---|---|---|---|---|
| Credential provider with service account token mode | No | Per workload identity | Different workloads on a node need different registry rights | Needs v1.33+ nodes, a plugin that supports token exchange and registry-side trust configuration |
| Credential provider using node identity, or cloud node role | No | Per node or node pool | Most cloud clusters pulling from their own cloud registry | Every pod on the node shares the node's pull rights; separate sensitive images into dedicated node pools |
Per-namespace imagePullSecrets with short-lived tokens refreshed by a controller | Yes, briefly | Per namespace | Third-party or self-hosted registries without federation | A refresher becomes a privileged component with write access to Secrets |
Per-namespace imagePullSecrets with static robot tokens | Yes | Per namespace | Small clusters, external registries, transitional use | Manual rotation; the token is reusable outside the cluster |
Node config.json or pre-pulled images | On disk | Whole node | Air-gapped or edge nodes with a fixed image set | No tenant separation; Kubernetes notes pre-pulling needs root access to every node |
- Static Pods: they cannot reference ServiceAccounts or Secrets, so the credential provider (with
requireServiceAccount: false) or node configuration is the only route for private images. - Small organisations: one cloud registry pulled by node identity, plus one scoped, expiring robot token per external registry, is a reasonable and auditable baseline.
- Large or multi-tenant clusters: move towards per-workload credentials through service account token mode, combined with AlwaysPullImages or credential verification for cached images.
The kubelet image credential provider in practice
The credential provider has been stable since Kubernetes v1.26 and replaces the in-tree cloud providers, which were removed in the same release. The kubelet is started with --image-credential-provider-config (the path to a CredentialProviderConfig) and --image-credential-provider-bin-dir (the plugin directory), and communicates with the plugin over stdio using a versioned API.
| Field | Purpose | Security note |
|---|---|---|
name | Must match the executable in the plugin directory | Protect the bin directory as you would kubelet binaries: whoever can replace the plugin controls pulls |
matchImages | Image patterns that trigger the plugin; globs only in the domain, one segment per * | Keep patterns tight so a plugin is not asked for credentials for unrelated registries; overlapping providers are combined, earlier entries win on key conflicts |
defaultCacheDuration | In-memory cache time when the plugin does not return one | Shorter values make revocation take effect sooner at the cost of more calls |
apiVersion | Request and response version, for example credentialprovider.kubelet.k8s.io/v1 | Mismatches fail closed; pin and test on upgrade |
tokenAttributes.serviceAccountTokenAudience | Audience of the pod-bound token sent to the plugin | Use a registry-specific audience; with ServiceAccountNodeAudienceRestriction the node needs the request-serviceaccounts-token-audience permission for it |
tokenAttributes.cacheType | Token caches per token, ServiceAccount per account | Token is the conservative choice when credentials are bound to the token lifetime |
tokenAttributes.requireServiceAccount and annotation keys | Whether pods without a ServiceAccount are served; which ServiceAccount annotations must or may be passed | Required annotations act as an opt-in gate: a workload without them gets no registry credential |
Service account token mode is beta since v1.34 and enabled by default through the KubeletServiceAccountTokenForCredentialProviders feature gate. If tokenAttributes is set while the gate is off, the kubelet refuses to start, so stage the change per node pool. The result is federated registry access: the plugin exchanges a short-lived, audience-bound token for the specific workload into registry credentials, and nothing long-lived sits in etcd. See the Workload Identity Federation Guide for the trust model behind the exchange.
Cloud registry access without stored secrets
Each major cloud documents a way for its managed Kubernetes nodes to pull from its own registry using a platform identity instead of a stored secret. The settings below are the customer-side controls to review.
- Amazon Web Services (Amazon ECR): ECR authorisation tokens are valid for 12 hours and carry the same permission scope as the IAM principal that requested them. Hosts and CI use
aws ecr get-login-passwordpiped todocker login --password-stdin, or the Amazon ECR Docker credential helper; clusters use a credential provider matched to*.dkr.ecr.*.amazonaws.com. Scope the requesting role to pull actions on the repositories it needs. - Google Cloud (Artifact Registry with GKE): nodes pull as the node service account. Grant it
roles/artifactregistry.readeron the repositories needed, and for cross-project pulls grant that role in the registry project. Google recommends removing the Editor role that some organisations grant automatically to the Compute Engine default service account, and access scopes cannot be changed on existing nodes, so plan scope changes as node pool replacements. - Microsoft Azure (Azure Container Registry with AKS):
az aks update --attach-acrassigns theAcrPullrole to the kubelet managed identity of the cluster's agent pools, and--detach-acrremoves it.az aks check-acrvalidates access. For registries using ABAC repository permissions, Microsoft documents assigning the Container Registry Repository Reader role instead.
Availability: each provider documents prerequisites (CLI versions, cluster versions, roles needed to create the assignment). Confirm them in the provider's current documentation before rollout. For federation from CI systems to these registries, see the CI/CD Pipeline Identity Security Guide.
Credentials on hosts and build machines
Build agents, bastions and developer laptops collect registry logins. Docker stores credentials in config.json in base64 unless a credential store is configured, and its documentation calls file storage less secure than a store.
- Use a credential store or per-registry helper: set
credsStore(for exampleosxkeychain,wincred,passorsecretservice) or map registries to helpers incredHelpers, such as a cloud provider's helper for its registry. Rundocker logoutbefore switching stores so old entries leave the file. - Keep secrets off the command line: use
--password-stdin; both Docker and Kubernetes warn that passwords typed as arguments can land in shell history and be visible to other users while the command runs. - Ephemeral CI runners: log in per job with a short-lived token from federation and destroy the runner. On persistent runners, clean the credential file at job end and run jobs as separate users.
- Podman and Buildah: the primary
auth.jsonsits in the runtime directory, but tools fall back to Docker's file. Remove stale Docker files when standardising on Podman. - Machine images: never bake a
config.jsoninto a node or VM image; use a credential provider or helper that fetches at runtime.
Lifecycle: create, scope, rotate, revoke
- Create from a scoped identity. Issue pull-only robot accounts or tokens per consumer (cluster, namespace or pipeline), named after that consumer, with an expiry. Registry-side options are covered in the Container Registry Security Guide.
- Deliver without humans copying values. Sync pull Secrets from a secrets manager into namespaces, for example with the External Secrets Operator, rather than
kubectl create secret docker-registryrun by hand. - Attach narrowly. Reference the Secret from the workloads that need it, or from a dedicated ServiceAccount; do not attach pull secrets to the
defaultServiceAccount in shared namespaces. - Rotate with overlap. Create the new token, update the source of truth, let sync replace the Secret, confirm new pulls succeed, then revoke the old token at the registry. With credential verification enabled, Kubernetes re-pulls from the registry the first time a rotated credential is used, so expect a short burst of registry traffic.
- Revoke at the registry, not in the cluster. Deleting a Kubernetes Secret does not invalidate the token; copies may exist in other clusters, node files and laptops.
- Retire with the workload. Tie pull credentials to the namespace or application lifecycle so decommissioning removes them.
Implementing it: a phased rollout
- Inventory (weeks 1 to 2): list every Secret of type
kubernetes.io/dockerconfigjsonandkubernetes.io/dockercfgacross clusters, every ServiceAccount withimagePullSecrets, node images containingconfig.json, and registry tokens with their last use. Decode only the registry hostnames and usernames, never log the tokens. - Restrict access (weeks 2 to 4): apply the Kubernetes guidance to restrict
listandwatchon Secrets to system components (grantinglistimplicitly exposes contents), and review who can create pods in namespaces holding pull secrets. Confirm encryption at rest for Secrets. See the Kubernetes RBAC Guide and How to check Kubernetes secrets encryption at rest. - Close the cache gap (pilot node pool first): on v1.35 and later, review
imagePullCredentialsVerificationPolicy; the defaultNeverVerifyPreloadedImagesskips images pulled outside the kubelet, whileAlwaysVerifychecks everything. On first enablement the kubelet treats images already on the node as pre-pulled, so drain or clean nodes if that matters. Where available, AlwaysPullImages is the alternative for multi-tenant clusters. - Move to fetched credentials (per node pool): configure the credential provider for cloud registries, then trial service account token mode on a non-production pool with
cacheType: Tokenand required annotations. Roll back by removingtokenAttributesand restarting the kubelet; keep the previous Secret in place until the pilot passes. - Retire static secrets: remove pull Secrets and ServiceAccount attachments that are no longer used, then revoke the matching registry tokens.
- Harden hosts and CI: enforce credential stores on build hosts, per-job federated logins and runner clean-up.
Detection and measurement
- Kubernetes audit logs: alert on
get,listorwatchof dockerconfigjson Secrets by humans or unexpected service accounts, and on bursts of Secret reads by a single identity, which Kubernetes suggests as an audit rule. See the Kubernetes Audit Log Detection Guide. - Registry logs: watch for pull tokens used from IP ranges outside your clusters and runners, pulls of repositories a token has never touched, and use after the intended retirement date.
- Kubelet events: track
ErrImagePulland authentication failures after rotations; a spike shows a consumer you missed in the inventory. - Secret scanning: scan repositories, Helm values and CI logs for
.dockerconfigjsonblobs andauthsentries. See the Git Secret Scanning Guide.
| Metric | Target direction |
|---|---|
| Share of image pulls served by credential providers or platform identity rather than stored Secrets | Rising towards 100% for cloud registries |
| Pull credentials with a named owner and expiry date | 100% |
| Median age of static registry tokens in use | Below your rotation policy (for example 90 days) |
| ServiceAccounts with pull secrets that are not used by any running workload | Zero |
| Node pools with credential verification or AlwaysPullImages in multi-tenant clusters | All |
| Registry tokens seen in secret-scanning findings per quarter | Falling |
Framework mapping
| Framework | Reference | How it applies to pull credentials |
|---|---|---|
| NIST CSF 2.0 | PR.AA-01, PR.AA-05 | Registry credentials for services are managed; pull rights follow least privilege and are reviewed |
| NIST CSF 2.0 | PR.DS-01, DE.CM-09 | Secrets at rest in etcd and on hosts are protected; runtime environments are monitored for misuse |
| NIST SP 800-53 Rev. 5 | IA-5, IA-9, AC-6 | Authenticator management (expiry, rotation, protection); service identification; least privilege for pull scope |
| NIST SP 800-53 Rev. 5 | SC-28, AU-2 | Encryption of Secrets at rest; logging of Secret access and registry pulls |
| ISO/IEC 27001:2022 Annex A | A.5.17, A.8.5 | Authentication information handling; secure authentication for systems |
| MITRE ATT&CK | T1552.001, T1552.007 | Credentials in files (config.json, auth.json); credentials obtained through the container API (reading Secrets) |
| MITRE ATT&CK | T1528, T1525 | Theft of application access tokens; a stolen token with push rights enables implanting internal images, so keep pull tokens pull-only |
Common failure patterns
- The cluster-wide copy: one pull Secret, created from an engineer's personal login, copied into every namespace by a script. When that person leaves, either every deployment breaks or the token is never revoked.
- Pull secret on the default ServiceAccount: attaching credentials there hands them to every pod in the namespace, including debug pods created by anyone with pod create rights.
- Push rights in a pull secret: reusing the CI push token for cluster pulls means a leaked Secret lets an attacker replace images rather than just read them.
- Deleting the Secret and calling it revoked: the token stays valid at the registry and in every other copy, including node images and laptops.
- Trusting the node cache: without credential verification or AlwaysPullImages, a pod in one namespace can run a private image another namespace pulled onto the same node.
- Copying a config.json that uses a credential store: the resulting Secret holds no credential, prompting someone to paste a long-lived password in its place.
Practitioner checklist
- Inventory all dockerconfigjson and dockercfg Secrets, ServiceAccount attachments, node files and host logins, with owner and expiry.
- Issue pull-only, per-consumer, expiring registry tokens; never reuse push tokens or personal logins.
- Restrict
list,watchandgeton Secrets and review pod create rights in namespaces holding pull secrets. - Enable encryption at rest for Secrets and audit Secret reads.
- Use platform identity or the kubelet credential provider for cloud registries; trial service account token mode for per-workload access.
- Set
imagePullCredentialsVerificationPolicydeliberately, or use AlwaysPullImages, on multi-tenant clusters. - Configure credential stores or helpers on hosts, use
--password-stdin, and keep credential files out of machine images. - Rotate with overlap and always revoke at the registry; tie credentials to workload retirement.
Standards and references
- Kubernetes: Images (private registries, credential verification)
- Kubernetes: Configure a kubelet image credential provider
- Kubernetes: Good practices for Kubernetes Secrets
- Docker: docker login and credential stores
- Containers project: containers-auth.json
- Amazon Web Services: Private registry authentication in Amazon ECR
- Google Cloud: Deploying to Google Kubernetes Engine from Artifact Registry
- Microsoft Learn: Authenticate with Azure Container Registry from AKS
Related NHI Mgmt Group resources: Container Registry Security Guide · Tekton Security · Kubernetes Secrets Security Guide · Kubernetes NHI Security Guide · Secrets Management Guide