Secrets Manager vs KMS vs HSM: What Each Protects and How They Fit Together

Teams often treat a secrets manager, a key management service (KMS) and a hardware security module (HSM) as interchangeable vaults, then store database passwords in a KMS, expect an HSM to rotate API keys, or buy dedicated hardware when a managed service would meet the requirement. The three are layers, not alternatives. A secrets manager stores and hands out credentials that applications must read in plaintext. A KMS holds encryption keys that never leave it and performs cryptographic operations on request. An HSM is the tamper-resistant hardware that protects key material at the bottom of the stack. This guide explains what each protects, compares them property by property, shows how they fit together through envelope encryption, and gives decision guidance on when a dedicated or single-tenant HSM is genuinely required. It is for security architects, platform engineers and compliance leads. For key lifecycle in depth see the Cryptographic Key Management Guide.

Key takeaways

• Secrets managers hand secrets out; KMS keeps keys in. A secrets manager returns the secret value to an authorised caller. A KMS performs encrypt, decrypt and sign operations and does not export its root keys in plaintext.
• Most stacks use all three. The secrets manager encrypts what it stores under a KMS key, and the KMS protects its keys in HSMs, so the question is usually which layer you manage directly, not which one to buy.
• An HSM protects keys, not credential lifecycles. OWASP notes that HSMs do not manage application credential leasing or rotation, so they complement secrets management rather than replace it.
• FIPS validation and single tenancy are different requirements. Multi-tenant cloud KMS services already run on FIPS 140 Level 3 validated HSMs; a dedicated HSM is warranted when a regulation, contract or policy requires sole custody.
• The more control you keep, the more you operate. Customer-managed keys, single-tenant HSMs and external key managers move availability and recovery risk onto you; losing the key or its recovery material means losing the data.

What each one is

• Secrets manager: a service that stores credentials and other sensitive configuration (database passwords, API keys, tokens, certificates and their private keys), controls who can read each one, versions it, rotates it where possible and audits access. The consumer receives the secret value and uses it. Examples include AWS Secrets Manager, Google Cloud Secret Manager, HashiCorp Vault's key/value engine and secrets in Microsoft Azure Key Vault.
• Key management service (KMS): a service that creates, stores and controls cryptographic keys and performs operations with them on behalf of callers. The caller sends data or a data key and receives ciphertext, plaintext or a signature, but never the root key. AWS documents that KMS keys never leave its FIPS 140-3 Security Level 3 validated HSMs unencrypted and that to use a KMS key you must call the service.
• Hardware security module (HSM): a physical, tamper-resistant device validated against FIPS 140 that generates, stores and uses keys inside its boundary. You meet HSMs either indirectly, underneath a cloud KMS, or directly as a single-tenant service or an appliance you operate, using interfaces such as PKCS#11, JCA/JCE, CNG and KSP.

The layers stack: a secrets manager encrypts each stored secret with a key held in a KMS, and the KMS root keys live in HSMs. See the Secrets Management Guide, the HSM Deployment and Operations Guide and the FIPS 140-3 and Cryptographic Module Validation Guide.

Side-by-side comparison

How they fit together: envelope encryption

AWS defines envelope encryption as encrypting plaintext data with a data key and then encrypting the data key under another key. The pattern is how all three layers cooperate:

1. The application or service asks the KMS for a data key under a named KMS key (in AWS, the GenerateDataKey operation on a symmetric key).
2. The KMS returns a plaintext data key and the same key encrypted under the KMS key.
3. The caller encrypts the data locally with the plaintext data key, discards it from memory and stores the encrypted data key alongside the ciphertext.
4. To decrypt, the caller sends the encrypted data key to the KMS, which checks the caller's permission, decrypts it inside its HSM boundary and returns the plaintext data key.

A secrets manager does exactly this for each secret it stores, which is why read permission on a secret and decrypt permission on its KMS key are both needed when you choose a customer-managed key. OWASP's cheat sheet adds that keys should not be stored next to the secrets they encrypt unless they are themselves encrypted, which is what envelope encryption achieves. See the Envelope Encryption and KMS Integration Patterns Guide.

Cloud services at each layer

Each major cloud provides all three layers. Platforms are listed alphabetically; the table records each provider's own service names and documented options, not a ranking.

Validation levels and service names are taken from each provider's documentation as of November 2026; confirm current certificates on the NIST Cryptographic Module Validation Program before relying on them for compliance. For the provider guides see the AWS KMS Security Guide, the Cloud KMS and Cloud HSM Guide and the Azure Key Vault Security Guide.

When to use each

Do you need a dedicated HSM?

• Usually no: if the requirement is "keys protected by FIPS 140 Level 3 validated hardware", major cloud KMS services with HSM protection already meet it. OWASP's cheat sheet says a dedicated HSM is warranted only when regulatory, contractual or policy requirements mandate it.
• Yes, when: a scheme mandates specific certified payment HSMs; policy requires that the provider's administrators cannot operate the key store; you need direct PKCS#11 or CNG access for software that cannot call a KMS API; or you must hold keys outside the provider entirely.
• The cost of control: Microsoft documents for Managed HSM that the customer-owned security domain is the root of trust and that losing it means permanent loss of all keys. Microsoft also describes external key management as a last-resort option for legal or contractual requirements. Plan backup, quorum and recovery before you take custody.

Small organisations are usually best served by the provider's secrets manager plus provider-managed or customer-managed KMS keys, with no HSM to operate. Large and regulated organisations typically add customer-managed keys with separation of duties for all sensitive data, single-tenant HSMs for CA and signing keys, and external key management only where a legal requirement names it.

Putting the layers in place: a phased approach

1. Inventory both kinds of material. List credentials (passwords, API keys, tokens, certificates) and keys (data encryption, signing, CA) separately, with owner, location and consumer. Material in the wrong layer, such as KMS-encrypted passwords in config files or private keys in a key/value store, goes on the change list.
2. Centralise credentials first. Move secrets out of code, environment files and CI variables into the secrets manager, one application at a time, reading them at runtime. Rollback is restoring the previous configuration value, kept until the application has run cleanly through a release.
3. Decide key ownership. For each data store, record whether a provider-managed key is acceptable or a customer-managed key is required, and why.
4. Create customer-managed keys with separated roles. Key administrators manage policy and rotation but cannot decrypt; workloads can decrypt but cannot change policy. Turn on automatic rotation where the service offers it.
5. Test before switching. Re-encrypt a non-production store under the new key, confirm reads, backups and restores work, then move production. Keep the previous key enabled until everything encrypted under it has been re-encrypted.
6. Add an HSM only for a documented requirement. Provision it with recovery material under quorum, migrate the specific keys that need it, and rehearse a full restore before decommissioning anything.

Threat model, failure modes and a worked scenario for each option

Secrets manager

Threat model. The store holds usable secrets, so whoever can read a path gets the plaintext; MITRE ATT&CK tracks this as T1555.006 Cloud Secrets Management Stores. Failure modes. Broad read permissions and secrets fetched once and cached for ever. Controls. Per-secret access policies, audit logging and rotation. Decision criteria. Use it for anything an application must present to another system.

KMS

Threat model. Keys never leave the service, so the attacker targets permission to call decrypt. In AWS KMS, for example, root keys never leave FIPS 140-3 Security Level 3 validated HSMs unencrypted, and to use a key you must call the service. Failure modes. Key policies that grant decrypt widely, which make encryption a formality. Controls. Separate encrypt and decrypt rights and alert on unusual decrypt volume. Decision criteria. Use KMS for data encryption keys and envelope encryption.

Dedicated HSM

Threat model. You take custody of the hardware boundary, so the risk shifts to your own operations. Failure modes. Lost quorum cards and untested backups. Controls. Split knowledge, tested recovery. Decision criteria. Only when regulation or contract requires single-tenant key custody.

Worked scenario. A team stores a database password in a secrets manager and encrypts customer files with data keys. Each file is encrypted with its own data key, and that data key is encrypted under a KMS key and stored beside the file, so rotating the KMS key means re-encrypting only data keys, not the files. No dedicated HSM is needed unless a regulator demands it.

Common failure patterns

• Using KMS as a password store. Teams encrypt a password with KMS and keep the ciphertext in a config file, recreating a static secret with no versioning, rotation or per-secret audit. Use a secrets manager.
• Granting decrypt to everyone. A key policy that allows any principal in the account to decrypt makes the customer-managed key meaningless. Separate key administrators from key users and scope decrypt per workload.
• Buying an HSM to fix credential sprawl. An HSM protects keys but does not find, rotate or revoke API keys; sprawl needs a secrets manager and inventory.
• Single-tenant HSM without recovery planning. Security domain, quorum cards or backup keys held by one person or never tested.
• Accidental key deletion or disablement. Scheduled deletion of a KMS key makes everything encrypted under it unreadable. Restrict deletion rights and alert on scheduling.
• The secrets manager's own bootstrap credential left static. The credential a workload uses to reach the secrets manager becomes the new long-lived secret. See the Secrets Zero Problem Guide.

Detection and measurement

Log every secret read and every KMS decrypt, sign and key administration call, and send both to the SIEM. Alert on reads of many secrets by one identity, decrypt calls from new principals or networks, key policy changes, key disablement and scheduled deletion, and HSM administrative logins. Bulk reads from cloud secret stores map to ATT&CK T1555.006 Cloud Secrets Management Stores; stolen private keys to T1552.004 Private Keys.

• Secrets outside the secrets manager: findings from code and config scanning, trending to zero.
• Customer-managed key coverage: percentage of sensitive data stores encrypted under keys you control, where policy requires it.
• Key policy breadth: keys whose decrypt permission extends beyond named workloads, target zero.
• Key rotation compliance: keys rotated within the cryptoperiod your policy sets.
• Recovery drill success: successful restore of HSM or key backups in the last scheduled test.

Framework mapping

Practitioner checklist

• Map every credential to a secrets manager and every encryption or signing key to a KMS or HSM; remove crossover.
• Use envelope encryption for application data and cache data keys within policy.
• Separate key administrators from key users and scope decrypt permission per workload.
• Choose customer-managed keys where policy requires control over access or revocation.
• Document the requirement before buying a dedicated or single-tenant HSM.
• Back up HSM security domains or key material under quorum and test restore.
• Restrict and alert on key deletion, disablement and key policy changes.
• Remove static bootstrap credentials to the secrets manager with workload identity.

Standards and references

• NIST: SP 800-57 Part 1 Rev. 5, Recommendation for Key Management
• NIST: FIPS 140-3, Security Requirements for Cryptographic Modules
• OWASP: Secrets Management Cheat Sheet
• AWS: AWS KMS cryptography essentials
• Google Cloud: Cloud KMS protection levels
• Microsoft Learn: Key management in Azure
• Microsoft Learn: Azure Key Vault Managed HSM overview
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Cryptographic Key Management Guide, Secrets Management Guide, Envelope Encryption and KMS Integration Patterns Guide, HSM and Key Management Buyer's Guide, Cloud Secrets Managers Guide.