Every secrets manager has the same weak point: the workload has to prove who it is before it can fetch anything, and whatever it uses for that first proof is itself a secret that has to come from somewhere. This is the secret zero, or bootstrap secret, problem. When the answer is a static token in an environment variable, a configuration file or a CI variable, the vault only moves the risk one step along. This guide covers the patterns that actually solve it: platform attestation, workload identity federation, attested node and workload identity with SPIFFE, and, where a bootstrap secret really is unavoidable, wrapped, single-use, narrowly bound delivery. It is for platform engineers, DevOps and security architects designing how applications, pipelines and agents receive credentials. The broader programme is in the Secrets Management Guide; this page goes deep on the first step.
Key takeaways
- Do not store secret zero, derive it. The strongest answer is a credential issued by the platform from facts it already knows about the workload, such as the instance, pod, job or process, so nothing long-lived is stored with the workload.
- Federation is the standard pattern. The IETF WIMSE practices draft describes workloads presenting a platform-issued credential to an identity provider in exchange for an access token, rather than holding static client secrets.
- Attestation quality decides security. A credential is only as trustworthy as the evidence used to issue it: a cloud-signed instance document or TPM-backed key is stronger than a pre-shared token.
- When a bootstrap secret is unavoidable, make it nearly useless. Single use, minutes-long TTL, bound to a network range or host, delivered by a trusted orchestrator and wrapped so interception is detectable.
- Environment variables are the wrong delivery channel. The WIMSE practices draft says they must not be used where the platform offers file or local API delivery, because they leak through logs, process inspection and error reports.
What secret zero is and why it persists
A workload starting up needs credentials for databases, APIs and cloud services. Putting those in a secrets manager is the right move, but the workload must authenticate to the manager first. Whatever proves its identity at that moment is secret zero. The problem persists because the easy answers all reintroduce a long-lived static credential:
- A vault token baked into the image or configuration: it lives as long as the image and is copied into every registry, cache and backup.
- A token in an environment variable set by the deployment system: it is visible to anything that can read the process environment, and to every tool that logs or dumps it.
- A CI variable holding cloud keys: every job step, and every third-party script that step runs, can read it. The Codecov Breach 2021 showed the consequence: a tampered uploader script sent the environment of customers' CI jobs to an attacker, exposing the credentials held there.
- A human pasting a token at deployment: the human becomes the trust anchor, and the token ends up in chat, tickets or shell history.
The underlying question is the one the IETF WIMSE practices draft frames as a chicken-and-egg issue: needing a credential to get a credential. The way out is to anchor the first credential in something the workload cannot copy and the platform can vouch for.
The solution patterns compared
| Pattern | What anchors trust | Strength | Limitations | Typical platforms |
|---|---|---|---|---|
| Cloud platform identity | The cloud provider's control plane knows which instance or function is calling and issues short-lived credentials | No stored secret; credentials rotate automatically | Bound to one cloud; metadata endpoints need SSRF protection | AWS instance roles via IMDS, Azure managed identities, Google service accounts |
| Workload identity federation | A platform-issued token (OIDC JWT) trusted by a different identity provider through configured trust conditions | Crosses clouds and SaaS without static keys | Trust conditions must be precise; bearer token unless key-bound | CI/CD OIDC, Kubernetes service account tokens to cloud IAM |
| Attested node and workload identity | Node attestation (cloud document, TPM, Kubernetes token) then workload attestation of the calling process | Platform-neutral, short-lived X.509 or JWT credentials, works on-premises | Requires running the attestation infrastructure | SPIFFE and SPIRE |
| Secrets manager platform auth | The vault validates a platform credential (cloud identity, Kubernetes token, OIDC) instead of a static token | Combines secretless bootstrap with existing vault policies | The vault's trust configuration becomes critical | Vault cloud, Kubernetes and JWT auth methods; cloud secret managers with IAM |
| Trusted orchestrator with wrapped secrets | A deployment system already trusted by the vault fetches a single-use credential and hands it to the workload | Works where no platform identity exists; interception is detectable | The orchestrator is a high-value target; still a secret in transit | Vault AppRole with response wrapping, configuration management tools |
| Pre-shared join token | A token generated by the identity system and given to a new node once | Simple for bare metal or edge | Weakest attestation; must be single use and short lived | SPIRE join token, device enrolment codes |
Platform attestation and federation in detail
The WIMSE practices draft (draft-ietf-wimse-workload-identity-practices-07, an Internet-Draft with intended status Informational, submitted to the IESG; checked 27 October 2026) describes how platforms issue workload credentials today. Treat draft details as subject to change; the patterns below are already in production use.
- Kubernetes: service account tokens are JWTs signed by the control plane, delivered to pods as projected files or requested through the TokenRequest API with custom audiences and lifetimes. The draft notes that the TokenRequest API is only a partial answer because the caller needs an initial projected token to use it. See the Kubernetes Secrets Security Guide.
- Cloud metadata services: an instance metadata endpoint issues credentials, often bearer tokens. On AWS, IMDSv2 requires a session token obtained with a PUT request, valid only on the instance that created it, with a default response hop limit of 1; setting the instance to require IMDSv2 makes token-less requests fail with 401. Azure managed identities give code Entra tokens without the code ever handling a credential, and Microsoft notes the credentials are not even accessible to the owner.
- CI/CD: GitHub Actions issues each job an OIDC token whose claims (sub, aud, repository, ref, environment, job_workflow_ref) the cloud provider checks against the trust conditions on a role, returning a short-lived access token valid for that job. GitHub states this removes the need to duplicate cloud credentials as long-lived secrets.
- Federation step: the draft describes the workload presenting its platform credential to an external identity provider in return for an access token, pointing to the OAuth assertion framework (RFC 7521) and the JWT assertion profile being revised as draft-ietf-oauth-rfc7523bis. See the Workload Identity Federation Guide.
What makes these secure is not the token format but the precision of the trust policy. A trust condition that accepts any repository in an organisation, any namespace in a cluster or any instance in an account turns federation into a shared credential.
Attestation with SPIFFE and SPIRE
SPIRE splits bootstrap into two attestations. Node attestation proves the machine: SPIRE documents attestors based on the AWS EC2 Instance Identity Document, Azure managed identities, GCE instance identity tokens, Kubernetes service account tokens, a TPM or HSM-held private key, an existing X.509 certificate, or a pre-shared join token that expires immediately after use. Workload attestation then identifies the calling process by querying the operating system or orchestrator (SPIRE includes Unix, Kubernetes and Docker attestors) and matching the result to registration entries before issuing an SVID. The WIMSE draft notes that the SPIFFE Workload API avoids the chicken-and-egg issue because it identifies workloads from environmental context. See the SPIFFE and SPIRE Guide and the Workload Attestation Guide.
When a bootstrap secret is unavoidable
Legacy hosts, air-gapped networks, appliances and some edge devices have no platform identity to attest. Here the goal is to make the bootstrap secret as close to worthless as possible. HashiCorp Vault's AppRole and response wrapping illustrate the controls, and the same principles apply to any system.
- Split the credential: AppRole separates a RoleID (an identifier, not secret) from a SecretID (the credential). Deliver them through different channels so neither alone is enough.
- Prefer pull mode: Vault documents that pull mode, where Vault generates the SecretID, is generally better than push mode, where another system supplies it, because push requires another system to know the full credential set.
- Constrain the SecretID: set secret_id_ttl to minutes, secret_id_num_uses to 1 for bootstrap, and secret_id_bound_cidrs to the network the workload will log in from.
- Wrap it: response wrapping stores the real response in a single-use token's cubbyhole and returns only a wrapping token with its own TTL. Vault lists three properties: cover (the secret never travels in clear), malfeasance detection (only one party can unwrap, so a failed unwrap signals interception) and lifetime limiting.
- Validate before unwrapping: Vault recommends looking up the wrapping token to confirm it has not already been unwrapped or expired, and checking that its creation path is exactly what you expect. Treat a missing or already-used token as a security event.
- Use a trusted orchestrator: the system that delivers the wrapped token (deployment pipeline, configuration management, scheduler) must already be trusted and should be the only party allowed to generate SecretIDs for that role. Vault's guidance is that the application should not know the SecretID directly, only receive a short-lived wrapping token.
- Exchange immediately: on start, the workload unwraps, logs in, obtains a short-lived token and discards the bootstrap material. Renewal then uses that token, not the bootstrap secret.
Delivery channels and storage of the first credential
| Channel | Risk | Guidance |
|---|---|---|
| Environment variable | Leaks through logs, process inspection, crash and error reporting, child processes | The WIMSE practices draft says it must not be used where file or local API delivery is available |
| File mount | Readable by other processes or containers; may persist in backups | Restrict with file permissions and mandatory access control, write atomically, prefer memory-backed mounts |
| Local API (metadata endpoint, Workload API socket, agent) | Server-side request forgery and confused-deputy calls can reach it | Require session tokens (IMDSv2), restrict hop limits, never let untrusted input build credential requests |
| Image or repository | Copied everywhere, permanent in history | Never; scan images and repositories for violations |
| Orchestrator injection at deploy | The orchestrator becomes a single point of compromise | Wrapped, single-use, TTL in minutes, orchestrator in a protected tier |
The credential that results from bootstrap should follow the draft's other recommendations: the smallest audience that works (ideally one), a lifetime no longer than the workload instance, and proof of possession (key binding) where the platform supports it, so a stolen bearer token cannot be replayed elsewhere.
Choosing an approach
- Workloads in a major cloud: use the cloud's native workload identity for everything inside that cloud and federation from it to anything outside. There is no reason to store a bootstrap secret.
- Kubernetes: use projected, audience-bound service account tokens to authenticate to the vault or cloud IAM. Disable automatic token mounting for pods that do not need it.
- CI/CD pipelines: replace stored cloud keys with OIDC federation and write trust conditions that pin repository, branch or environment, and workflow.
- Multi-cloud and on-premises fleets: SPIFFE and SPIRE give one identity model across environments, with TPM attestation for physical hosts.
- Legacy, air-gapped or appliance workloads: trusted orchestrator with wrapped, single-use, CIDR-bound secrets, and a plan to move to attestation when the platform allows.
- AI agents: agents should receive identity from the platform they run on, never a long-lived key in a prompt, configuration or tool definition. See the Credential Vaulting for AI Agents Guide.
Small teams get most of the benefit from two moves: cloud-native identity for compute and OIDC federation for CI. Large organisations need a standard per platform, a central record of trust policies, and an exceptions register for workloads still using bootstrap secrets.
Implementing the change
- Inventory bootstrap credentials: find every vault token, cloud key and API key used by workloads to authenticate to a secrets manager or cloud, starting with CI variables, deployment configuration and images. The Secret Sprawl Measurement Guide helps size the problem.
- Classify by platform: map each to the platform identity available (cloud, Kubernetes, CI OIDC, none).
- Configure trust in parallel: set up the platform auth method or federation trust alongside the existing secret, with narrow conditions.
- Pilot: move a non-production workload, confirm it obtains credentials through the new path, and watch audit logs for the old secret falling idle.
- Cut over in rings: move workloads by environment, keeping the old secret valid but monitored as a rollback for a defined window.
- Revoke: once the old secret shows no use for the agreed period, revoke it and remove it from every store.
- Guard the door: block new static bootstrap secrets with policy checks in pipelines and push protection in repositories.
Detection and measurement
- Vault and IdP audit logs: logins with static tokens or SecretIDs, failed unwraps, wrapping tokens looked up after use, and platform-auth logins from unexpected namespaces, repositories or accounts.
- Cloud logs: metadata credential use from outside the instance (for example, instance role credentials used from an external IP), which indicates theft through SSRF or host compromise (T1552.005).
- Federation logs: token exchanges whose subject claims fall outside the expected set.
- Scanning: secrets in images, environment dumps and CI logs.
Metrics:
- Share of workloads authenticating with platform identity or federation rather than a stored secret (target close to 100%).
- Number of static bootstrap secrets in the exceptions register, and their age.
- Median TTL and use count of remaining bootstrap SecretIDs (target minutes and one).
- Federation trust policies with wildcard or organisation-wide subject conditions (target zero).
- Instances not requiring IMDSv2 or equivalent metadata protection.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev. 5 | IA-5(7) No Embedded Unencrypted Static Authenticators; IA-9 Service Identification and Authentication; IA-3 Device Identification and Authentication; SC-12 Cryptographic Key Establishment and Management | No credentials in images or code; services and devices authenticated by attested identity |
| NIST CSF 2.0 | PR.AA-01; PR.AA-03 | Credentials for services and hardware managed; services authenticated |
| MITRE ATT&CK v20 | T1552.001 Credentials In Files; T1552.005 Cloud Instance Metadata API; T1552.007 Container API; T1078.004 Cloud Accounts | Where bootstrap credentials are stolen and how they are abused |
| OWASP Non-Human Identities Top 10 | NHI7 Long-Lived Secrets | Static bootstrap secrets are a long-lived secret by definition |
| IETF WIMSE | draft-ietf-wimse-workload-identity-practices; draft-ietf-wimse-arch | Practices for platform-issued workload credentials and federation (Internet-Drafts) |
| ISO/IEC 27001:2022 Annex A | A.5.17 Authentication information; A.8.24 Use of cryptography | Management of authentication information and keys |
Common failure patterns
- Vault adopted, token in an environment variable: the static vault token is now the most valuable secret in the estate, and it is the least protected.
- Federation with loose trust conditions: a cloud role trusting any job in a CI organisation lets any repository, including a fork or a compromised low-value project, assume it.
- Bootstrap secrets that are reusable and long-lived: a SecretID with no TTL and unlimited uses is just a password.
- Unprotected metadata endpoints: instances accepting IMDSv1 or containers with an unnecessary hop limit let SSRF or a compromised container obtain the host's role credentials.
- Join tokens treated as convenience: reused or long-lived pre-shared tokens undermine node attestation entirely.
- Third-party code in privileged CI steps: helper scripts run in the same step that holds cloud keys, as the Codecov incident showed.
Lessons from incidents
The Codecov case is a bootstrap problem at scale: customers' CI jobs held cloud keys, signing keys and tokens in environment variables, and a single altered script collected them all. Federation would have changed the outcome. A job that obtains a short-lived token only for the step that needs it, with a trust policy pinned to repository, branch and workflow, leaves a stolen environment dump with nothing reusable in it.
Practitioner checklist
- Inventory every static credential workloads use to reach a secrets manager or cloud, starting with CI variables and images.
- Use native platform identity for compute in each cloud and require IMDSv2 or the equivalent protection.
- Replace stored CI cloud keys with OIDC federation pinned to repository, branch or environment, and workflow.
- Authenticate Kubernetes workloads with audience-bound projected tokens and disable automatic token mounts where unused.
- Adopt SPIFFE and SPIRE or an equivalent where workloads span clouds and data centres, with TPM attestation for physical hosts.
- For unavoidable bootstrap secrets, enforce single use, minutes-long TTL, CIDR binding and response wrapping from a trusted orchestrator.
- Deliver credentials by file or local API, never by environment variable where an alternative exists.
- Alert on failed unwraps, static-token logins and metadata credentials used off-instance.
- Keep an exceptions register for remaining bootstrap secrets with owners and target dates.
Standards and references
- IETF: draft-ietf-wimse-workload-identity-practices, Workload Identity Practices (Internet-Draft)
- IETF: WIMSE working group documents
- SPIFFE: SPIRE Concepts
- HashiCorp: Vault response wrapping
- HashiCorp: Vault AppRole auth method
- AWS: Use the Instance Metadata Service (IMDSv2)
- GitHub Docs: OpenID Connect in GitHub Actions
- Microsoft Learn: Managed identities for Azure resources
Related NHI Mgmt Group resources: Secrets Management Guide · Workload Identity Federation Guide · SPIFFE and SPIRE Guide · OWASP NHI Top 10 NHI7 Long-Lived Secrets Guide · Dynamic Secrets and Ephemeral Credentials Guide