If the keys that protect your data, or the vault that holds your secrets, are deleted, destroyed or locked by an attacker, every backup encrypted under them may be useless. Most recovery plans focus on servers and data and assume the key management service will simply be there. It will not always be: keys can be scheduled for deletion by a compromised administrator, a secrets vault can be lost together with its unseal mechanism, and an HSM cluster can fail with its backups tied to the same region. This guide covers how to design for key and secrets recovery and how to run it: recovery windows on cloud key services, unseal and recovery keys for secrets vaults, what must be backed up and where, recovery order, and testing. It is for platform, cryptography and resilience teams. For responding to a key that has been stolen rather than lost, see the Encryption Key Compromise Playbook; for policy, see the Key Management Policy Template.
Products are named only as examples and are listed alphabetically; naming is not a ranking or an assessment. This is an independent guide from NHI Mgmt Group, based on each vendor's public documentation as of November 2026. It is not sponsored or reviewed by any vendor named. Product features change, so confirm settings against each vendor's current documentation.
Key takeaways
- Key loss is data loss. AWS states that data encrypted under a deleted KMS key becomes unrecoverable, and Google states that data encrypted with a destroyed key version cannot be decrypted. Recovery has to happen before the window closes.
- Waiting periods are your safety net, so make them long and watched. Cloud key services hold deleted keys for a configurable period; alert on every scheduled deletion and keep the window at the maximum your risk appetite allows.
- A vault backup without its seal is not a backup. HashiCorp states that if a Vault cluster's seal mechanism or its keys are permanently deleted, the cluster cannot be recovered, even from backups.
- Separate the people who can destroy keys from those who use them. Destroy, purge and schedule-deletion rights belong to a small, monitored group, ideally behind approval.
- Recovery order matters. Keys and secrets come back before the applications and backups that depend on them, and every secret exposed in the incident is rotated after recovery.
Recovery scenarios
| Scenario | Typical cause | What decides whether you recover |
|---|---|---|
| Cloud KMS key scheduled for deletion or destroyed | Mistake, automation error, or a compromised administrator (ATT&CK T1485 Data Destruction, T1490 Inhibit System Recovery) | Whether you notice inside the waiting period |
| Secrets vault lost | Storage corruption, ransomware on the hosts, region loss | Off-site snapshots plus a working seal and the operator key shares |
| Vault seal key unavailable | Auto-unseal KMS key disabled or deleted, HSM unreachable | Restoring the seal key; recovery keys alone are not enough |
| HSM cluster lost | Hardware failure, region outage | HSM backups in a second region, or security domain material and quorum of key shares |
| Administrators locked out | Attacker removes access (T1531 Account Access Removal), lost credentials | Break-glass identities and quorum-held recovery keys |
| Keys encrypted or held to ransom | Attacker re-encrypts data or wraps keys with their own key (T1486) | Isolated copies of key material or key backups outside the attacker's reach |
Recovery windows on cloud key services
Each major cloud key service delays permanent destruction. Platforms are listed alphabetically. Confirm current values in each vendor's documentation, as they change.
| Service | Deletion behaviour (vendor docs, November 2026) | Recovery action | Customer-side settings |
|---|---|---|---|
| AWS KMS | Schedule deletion with a waiting period of 7 to 30 days (default 30); the key state is Pending deletion and it cannot be used | Cancel the deletion, then re-enable the key (it stays disabled after cancellation) | Use the 30-day window; prefer disabling over deleting; CloudWatch alarm on attempted use of a key pending deletion |
| Google Cloud KMS | Key versions are scheduled for destruction, default 30 days, with a custom duration set at key creation | Restore the version while scheduled; it returns disabled and must be enabled | Organisation policy constraint for a minimum destroy scheduled duration; Cloud Monitoring alert on scheduled destruction |
| Microsoft Azure Key Vault | Soft delete keeps deleted vaults and objects for 7 to 90 days (default 90), set at vault creation; soft delete cannot be turned off once on | Recover the deleted vault or object within the retention period | Turn on purge protection (off by default) so deleted items cannot be purged before retention ends; restrict the purge permission |
Imported key material needs extra care. AWS documents that deleting only imported key material is reversible by reimporting it, but deleting the KMS key itself is not, and a new symmetric key cannot decrypt old ciphertexts even with the same material. Google documents that a destroyed imported key version can be re-imported with the same original material. If you rely on bring-your-own-key, your escrowed copy of the material is part of your recovery plan. See the Cloud Customer-Managed Keys and BYOK/HYOK Decision Guide.
Platform detail: AWS KMS Security Guide, Cloud KMS and Cloud HSM Guide, Azure Key Vault Security Guide.
Secrets vaults: seals, recovery keys and snapshots
A self-hosted secrets vault such as HashiCorp Vault encrypts its storage with a root key that is itself protected by a seal. HashiCorp documents two models:
- Shamir seal: the unseal key is split into shares, and a threshold of shares rebuilds it. Each node must be unsealed separately.
- Auto-unseal: a KMS or HSM decrypts the root key at start-up. Initialising this way produces recovery keys instead of unseal keys. HashiCorp warns that recovery keys cannot decrypt the root key: they authorise operations such as generating a root token or rekeying, but they cannot unseal Vault if the auto-unseal service fails.
The consequence is a hard dependency. HashiCorp states that if the KMS key used for auto-unseal becomes unavailable, access is blocked until it is available again, and if the seal mechanism or its keys are permanently deleted, the cluster cannot be recovered, even from backups. It recommends strict controls over the seal mechanism, such as AWS Service Control Policies.
For backups, HashiCorp states that all Vault editions support manual snapshots of cluster state, that Enterprise adds automated snapshots, and that snapshots should be kept outside the cluster's storage and outside the data centres where clusters run. A working restore therefore needs three things together: a recent snapshot, the seal (or Shamir shares) that protected it, and a quorum of operators who hold the unseal or recovery key shares.
See the HashiCorp Vault Guide and, for PAM vaults, Securing the PAM vault.
HSMs
HSM recovery depends on vendor-specific backup material such as cluster backups or a security domain with offline key shares. These are covered in the HSM Deployment and Operations Guide. The design rules on this page still apply: keep backup material in a second location, hold key shares under split custody, and restore into an isolated environment on a schedule. Use the Cryptographic Key Ceremony Template to record share handling.
Designing for recovery
- Inventory dependencies: for each key and vault, list what it protects (databases, backups, other keys) and what it depends on (a seal key, an HSM, an identity provider). Key hierarchies mean one lost wrapping key can take many data keys with it.
- Protect the destroy path: grant schedule-deletion, destroy and purge rights only to a dedicated role, separate from daily key administrators, with approval and alerting. In Google Cloud the destroy and restore permissions are cloudkms.cryptoKeyVersions.destroy and cloudkms.cryptoKeyVersions.restore; in Azure, purging requires the purge permission, for example through the Key Vault Purge Operator role.
- Guardrails at organisation level: use organisation policies or service control policies to block key destruction outside an approved role, and to protect seal keys from deletion.
- Isolate key backups from data backups: a backup that is encrypted under a key in the same account as the attacker's foothold may be lost with it. See the Backup Identity Isolation Guide.
- Split custody of key shares: unseal, recovery and HSM shares held by named people in separate locations, with a register of holders and handovers, and enough holders to reach quorum during holidays and staff turnover.
- Break-glass access: emergency identities that can reach the key service if the normal identity provider or admin accounts are unavailable. See the Break-Glass Account Guide.
Decision guidance: a small organisation using one cloud provider's managed key service can rely mostly on the provider's waiting period, provided it is set long, alarmed and protected by organisation policy. A larger organisation running self-hosted vaults, HSMs or bring-your-own-key needs escrowed key material, cross-region backups and a rehearsed quorum process, because there is no provider safety net for material it controls.
Recovery runbook
- Stop further damage: remove the destroy and purge rights of any suspected account, and freeze automation that manages keys.
- Find what is pending: list keys pending deletion, versions scheduled for destruction and soft-deleted vaults and objects, with their final dates (AWS records the scheduled deletion date; Azure lists deleted items, for example with az keyvault key list-deleted).
- Cancel and restore: cancel deletions, restore versions and recover soft-deleted items, starting with keys that protect backups and identity systems.
- Re-enable deliberately: restored keys return disabled on AWS and Google Cloud; enable them only once you have confirmed who will use them.
- Restore vaults: rebuild vault infrastructure in a clean environment, restore the seal key first, restore the latest clean snapshot, then unseal with the operator quorum.
- Verify: decrypt a known test object with each recovered key, and read a sample of secrets from the restored vault, before bringing applications back (CSF RC.RP-03 asks for integrity of restoration assets to be verified).
- Rotate: treat every secret and key the attacker could have read as exposed, and rotate in priority order. See the Secrets Rotation Runbook Template.
- Record: keep CloudTrail, Cloud Audit Logs and Azure activity records of the deletions, and the recovery actions, as incident evidence.
Detection and measurement
- Alert on destruction intent: every ScheduleKeyDeletion in AWS, every key version scheduled for destruction in Google Cloud, every delete and purge in Azure Key Vault, and every seal or snapshot configuration change in a vault.
- Alert on use of pending keys: AWS documents a CloudWatch alarm for attempts to use a key pending deletion, which shows what would break.
- Watch the destroy role: any use outside an approved change is an incident.
| Metric | Target (NHI Mgmt Group suggestion) |
|---|---|
| Production keys with the maximum waiting period or retention and purge protection on | 100% |
| Scheduled deletions acknowledged by a human within 24 hours | 100% |
| Vaults with an off-site snapshot under 24 hours old | 100% |
| Successful isolated restore tests of each vault and HSM in the last 12 months | 100% |
| Key share holders reachable in a quorum drill | Quorum reached within your recovery time objective |
Framework mapping
| Framework | Reference | How it applies |
|---|---|---|
| NIST SP 800-53 Rev 5 | SC-12, SC-12(1) | Key management and key availability |
| NIST SP 800-53 Rev 5 | CP-9, CP-10, CP-4 | Backup, recovery and reconstitution, and contingency plan testing for key and secrets services |
| NIST SP 800-53 Rev 5 | AC-6, AU-6 | Least privilege on destroy rights and review of key deletion events |
| NIST CSF 2.0 | PR.DS-11, RC.RP-01, RC.RP-03, PR.AA-05 | Tested backups, executing recovery, verifying restoration assets, separation of duties |
| NIST SP 800-57 Part 1 Rev 5 | Key backup, archive and recovery | General key management guidance on keeping keying material recoverable |
| MITRE ATT&CK | T1485, T1486, T1490, T1531 | Destruction, encryption for impact, inhibiting recovery and removing account access |
Common failure patterns
- Short windows nobody watches: keys deleted with the minimum waiting period and no alert, discovered when backups fail to decrypt.
- Seal key in the blast radius: the KMS key that auto-unseals the vault sits in the same account the attacker controls, so vault snapshots cannot be restored.
- Confusing recovery keys with unseal keys: teams assume auto-unseal recovery keys can open the vault in a disaster; they cannot decrypt the root key.
- Snapshots next to the cluster: vault backups stored on the same storage or in the same data centre are lost with it.
- Key shares that cannot reach quorum: holders have left, shares are lost, or nobody knows who holds them.
- No rotation after recovery: keys and secrets come back, but those the attacker could read stay valid.
Practitioner checklist
- Map every key and vault to what it protects and what it depends on.
- Set maximum waiting periods or retention, and turn on purge protection where available.
- Restrict destroy, purge and schedule-deletion rights to a monitored, approval-gated role.
- Alert on every scheduled deletion and every use of a key pending deletion.
- Keep vault snapshots off-site and protect the seal key from deletion with organisation-level guardrails.
- Escrow imported key material securely outside the cloud account it protects.
- Hold unseal, recovery and HSM shares under split custody with a current holder register.
- Restore each vault and HSM into an isolated environment at least yearly and decrypt a test object.
- Rotate every exposed key and secret after any recovery.
Standards and references
- NIST: SP 800-57 Part 1 Rev 5, Recommendation for Key Management: Part 1, General
- AWS: Deleting an AWS KMS key
- Google Cloud: Destroy and restore key versions
- HashiCorp: Vault seal and unseal
- HashiCorp: Vault backup standard procedure
- Microsoft Learn: Azure Key Vault soft-delete overview
Related NHI Mgmt Group resources: Cryptographic Key Management Guide · Encryption Key Compromise Playbook · Cyber Recovery Vault Guide · Cloud Ransomware Response Playbook