Secrets are the passwords of the machine world: API keys, database passwords, tokens, private keys and certificates that let software authenticate. Every organisation has thousands of them, and most secret-related breaches come not from sophisticated attacks but from secrets stored in the wrong place, shared too widely or left unchanged for years. Our Secret Sprawl Challenge describes the problem. This guide covers the practice: how to centralise secrets, deliver them safely to applications, rotate them, move towards dynamic and secretless patterns, and run secrets management as a programme.
Key takeaways
- Good secrets management rests on four principles: centralise, restrict, shorten and remove. Put secrets in a manager, limit who can read them, make them short-lived, and eliminate them where the platform can issue identity instead.
- A secrets manager solves storage, not sprawl. Pair it with scanning, ownership and rotation, or it becomes another place secrets hide.
- Dynamic secrets (generated per use, expiring automatically) and secretless patterns (workload identity) are the target state for most internal systems.
- Authenticating to the secrets manager is itself a secret problem, the "secret zero". Solve it with platform identity, not a master password in a config file.
What counts as a secret
- API keys and access tokens
- Database, service account and application passwords
- OAuth client secrets and refresh tokens
- Private keys for TLS, SSH, code signing and JWT signing
- Cloud access keys and service account key files
- Encryption keys and key-encryption keys
- Webhook signing secrets and connection strings
Where secrets end up without management
- Source code and repository history
- Configuration files and environment variables on servers
- CI/CD variables and build logs
- Container images and infrastructure-as-code state files
- Notebooks, wikis, tickets and chat
- Developer laptops, including AI coding tool and MCP configuration
Incidents such as the Docker Hub image leak, misconfigured Git servers and the tj-actions CI/CD attack show how each of these becomes an exposure route.
Principle 1: Centralise
- Choose a small number of approved secrets managers: typically a cloud-native service per cloud, and possibly a cross-platform vault. Avoid one per team.
- Migrate secrets from code, configuration and CI/CD variables into the manager, starting with production and privileged secrets.
- Record metadata with each secret: owner, consuming application, environment, creation date, rotation schedule.
Principle 2: Restrict
- Grant each application access only to the secrets it needs, using its own identity, not a shared token.
- Separate secrets by environment and by team, using paths, namespaces or separate vaults.
- Limit human read access; most people should never need to see a production secret.
- Log every read and alert on unusual access patterns.
Principle 3: Shorten
- Rotate static secrets on a schedule proportionate to risk, and immediately after exposure.
- Use dynamic secrets where possible: the manager creates a unique database credential or cloud token per request, with a short time-to-live, and revokes it automatically.
- Prefer short-lived tokens obtained from long-lived credentials held only by the manager.
| Static secret | Dynamic secret | Secretless (identity-based) | |
|---|---|---|---|
| Lifetime | Months to years | Minutes to hours | No stored secret; short-lived tokens issued by the platform |
| Shared? | Often | Unique per consumer | Unique per workload |
| Rotation effort | High | Automatic | None for the application |
| Impact if leaked | Long-lasting | Limited to TTL | Limited to token lifetime and audience |
| Example | Database password in a vault | Per-request database user from the vault | Cloud IAM role or managed identity to the database |
See the static versus dynamic secrets section of the Ultimate Guide.
Principle 4: Remove
The most secure secret is one that does not exist. Replace stored secrets with platform-issued identity wherever you can:
- Cloud IAM roles, managed identities and attached service accounts for workloads in the cloud.
- Workload identity federation for CI/CD and cross-cloud access.
- Kubernetes projected service account tokens mapped to cloud identities.
- SPIFFE identities and mTLS for service-to-service traffic.
The NHI Authentication Guide and Cloud Workload Identity Guide cover these in detail.
Solving "secret zero"
An application needs a credential to fetch its other credentials from the secrets manager. If that first credential is a static token in a configuration file, the problem has only moved. Better approaches:
- Authenticate to the manager with the platform identity: cloud IAM, Kubernetes service account tokens, or CI/CD OIDC tokens.
- Use agents or sidecars that authenticate with platform identity and deliver secrets to the application as files or environment variables.
- Where a bootstrap secret is unavoidable, make it single-use, short-lived and bound to a specific host or workload.
Delivering secrets to applications
- Runtime retrieval via SDK or API: flexible, supports dynamic secrets, requires application changes.
- Sidecar or agent injection into files: no code changes, supports rotation if the application re-reads files.
- Platform integrations such as Kubernetes secret store drivers and CI/CD secret integrations.
- Environment variables: convenient but easily leaked through process listings, crash dumps and logs; use with care.
Whatever the method, design applications to pick up rotated secrets without a restart.
Detection: finding secrets outside the manager
- Scan repositories, including history, on every commit and in pre-commit hooks.
- Scan CI/CD logs, container registries, infrastructure-as-code, wikis, tickets and chat.
- Validate findings where possible, since a live secret is urgent and a revoked one is not.
- Route findings to the secret's owner with a deadline, and track remediation.
Running secrets management as a programme
- Discover: scan for secrets everywhere and inventory existing vaults.
- Prioritise: start with production, privileged and externally exposed secrets.
- Centralise: migrate to approved managers, with owners and metadata.
- Automate rotation: begin with the highest-risk secrets and those easiest to rotate.
- Go dynamic and secretless: for new systems by default, and for existing ones as they change.
- Measure: secrets found outside managers, time to remediate, percentage rotated on schedule, percentage dynamic or secretless.
Practitioner checklist
- Approve a small set of secrets managers and migrate secrets into them, with owners and metadata.
- Give each application its own identity to the manager, scoped to its secrets.
- Solve secret zero with platform identity.
- Rotate static secrets automatically; adopt dynamic secrets for databases and cloud access.
- Replace secrets with workload identity wherever the platform supports it.
- Scan continuously for secrets outside managers and remediate through owners.
- Log and monitor secret access.
- Track programme metrics and report progress.
Standards and references
- OWASP Non-Human Identities Top 10 (2025): NHI2 Secret Leakage, NHI7 Long-Lived Secrets
- OWASP Secrets Management Cheat Sheet
- NIST SP 800-57 Part 1 Rev. 5: Recommendation for Key Management
- SPIFFE concepts
Related NHI Mgmt Group resources: Guide to the Secret Sprawl Challenge · Secrets Management Buyer's Guide · API Key Management Guide · Guide to NHI Rotation Challenges
