Encryption Guide: At Rest, In Transit and In Use

Most organisations can say their data is "encrypted", but far fewer can say who is able to decrypt it, and that second question decides whether encryption stops an attacker. Disk, storage service and transparent database encryption protect against lost hardware and stolen media, yet none of them stops a user, application or stolen cloud credential that is allowed to read the data. This is a decision guide: which data to encrypt, at which layer, under which key custody model, how to cover data in transit and in use, where post-quantum migration stands, and what GDPR says about encrypted data in a breach. It is for security architects, data owners and platform engineers. Algorithms and cipher suites are in the Cryptography Basics Guide and the key lifecycle in the Cryptographic Key Management Guide.

Key takeaways

• Encryption moves the problem to the key. It protects data only from parties who cannot use the key, so every design starts with a list of who and what can call decrypt.
• The higher the layer, the narrower the exposure. Disk and storage encryption defeat media theft; field-level and client-side encryption also defeat database administrators, backup readers and many cloud credential thefts, at the cost of search and complexity.
• Envelope encryption is the default pattern. Encrypt data with data keys, wrap them with a key that never leaves a KMS or HSM, and log every unwrap.
• Key custody is a trade-off, not a ranking. BYOK and HYOK reduce provider access but add outage and permanent data loss risk if the key is unavailable or destroyed.
• Internal traffic needs encryption too. TLS at the edge with cleartext database, replication and service traffic inside leaves the easiest interception points open, and hybrid ML-KEM groups now head the NCSC's TLS 1.3 key exchange list.

At rest, in transit, in use: what each actually stops

NIST SP 800-111 defines storage encryption as "the process of using encryption and authentication to restrict access to and use of stored information" and notes that the right solution depends on storage type, data volume, environment and threats. The useful question for every state of data is which attacker the control defeats.

Choosing the encryption layer for data at rest

SP 800-111 groups device options into full disk, volume and virtual disk, and file and folder encryption. Enterprise estates add storage service, database and application layers.

Endpoints and servers

• BitLocker: Microsoft documents XTS-AES 128-bit as the device encryption default and that changing the method requires decrypting first, so set policy before rollout. Use TPM plus PIN where pre-boot attacks matter. A device using only local accounts "remains unprotected even though the data is encrypted" because the recovery key is not escrowed, and device encryption does not cover USB media.
• Removable media: enforce encryption or block writes (CIS Controls v8.1 Safeguard 3.9).
• Servers: disk encryption protects retired drives and copied snapshots, not a running host. Put sensitive fields under data-level controls.
• Layering: multilayer encryption is normal, not redundant: disk encryption under TDE under field-level encryption, each with a different key holder, means one stolen credential exposes one layer. A sensitive field stored in clear above encrypted storage is still CWE-311 (Missing Encryption of Sensitive Data) for anyone with query access.

Databases: TDE versus column-level encryption

Microsoft's TDE documentation shows its limits: it performs "real-time I/O encryption and decryption of data and log files" and encrypts backups and tempdb, but "doesn't provide encryption across communication channels" and skips FILESTREAM data. Pages are decrypted into memory, so anyone who can query sees plaintext. The key chain runs from DPAPI through the service and database master keys to a certificate or EKM key that protects the database encryption key, and "Data loss results if the certificates are no longer available." A NULL pvt_key_last_backup_date in sys.certificates joined to sys.dm_database_encryption_keys means a TDE certificate has never been backed up.

Always Encrypted takes the opposite approach: keys are never exposed to the engine, protecting data from "on-premises database administrators, cloud database operators, or other high-privileged unauthorized users". A column encryption key encrypts data and a column master key in Azure Key Vault, the Windows certificate store or an HSM wraps it.

Field-level encryption design choices

• Deterministic or randomised: deterministic encryption supports lookups, equality joins and indexing, but Microsoft warns that patterns leak "especially if there's a small set of possible encrypted values, such as True/False". Reserve it for high-cardinality identifiers you must search.
• Blind indexes: store a keyed HMAC of the normalised value in a separate column for search, keep the field randomised, and use a separate HMAC key.
• Format-preserving encryption: SP 800-38G specified FF1 and FF3. The Rev. 1 second public draft (February 2025) states "The encryption method FF3 is no longer specified" after a tweak schedule weakness that also hit FF3-1, raises the FF1 domain size and bans floating point arithmetic in FF1. Use FF1 only, never on small domains.
• Tokenisation: removes cryptographic exposure downstream, but the detokenise API needs the same protection as a key.
• Bind ciphertext to context: pass record or tenant IDs as additional authenticated data so ciphertext copied elsewhere fails to decrypt.

Envelope encryption and the key hierarchy

AWS describes envelope encryption as "the practice of encrypting plaintext data with a data key, and then encrypting the data key under another key", ending in a root key. In AWS KMS that root never leaves "FIPS 140-3 Security Level 3 validated hardware security modules unencrypted". Other cloud KMSs and HSM-backed vaults follow the same pattern.

1. The application requests a data encryption key (DEK), for example with GenerateDataKey, and receives it in plaintext and wrapped under a key encryption key (KEK).
2. It encrypts locally with an AEAD cipher, stores the wrapped DEK beside the ciphertext and discards the plaintext DEK.
3. To read, it sends the wrapped DEK to the KMS for unwrapping: an authorised, logged call.
4. Rotating the KEK rewraps DEKs without touching bulk data; a suspected DEK compromise still means re-encrypting the data.

Design decisions: one DEK per object, file or tenant limits blast radius and stays well inside nonce limits. Caching plaintext DEKs cuts cost and latency but widens exposure to memory theft, so cap cache lifetime. Use separate KEKs per environment and classification, so each key policy grants one application one data set. The key policy and the identities allowed to decrypt are where most real protection lives; see the Secrets Management Guide.

Key custody: provider keys, CMK, BYOK, HYOK and HSM

Google Cloud's External Key Manager shows the HYOK trade-off. The key "resides on the external system, and is never sent to Google", and Key Access Justifications let the partner approve requests only for permitted reasons. But an unavailable key returns FAILED_PRECONDITION, a Spanner database can be deleted after 30 consecutive days of key unavailability, and "If you lose access to either key, your data cannot be recovered."

• CMK is the sensible default for regulated cloud data: auditable, revocable and cheap.
• BYOK fits when a regulator or contract requires key generation under your control or an offline escrow copy.
• HYOK fits only data where provider or foreign-jurisdiction access is a defined threat, with a tested availability design.
• Validated modules: NIST's CMVP moved FIPS 140-2 certificates to the Historical list after 21 September 2026, for use only in existing systems, so require FIPS 140-3 certificates in procurement.
• Scale: small organisations get most of the benefit from platform defaults plus CMK on the few stores holding personal or payment data. Large ones need a key hierarchy standard, per-environment KEKs, separation of key and data administrators, and HSM-backed roots.

Encryption in transit: closing the internal gaps

External TLS is usually in place. CWE-319 (Cleartext Transmission of Sensitive Information) findings mostly come from inside: application to database, replication, queues, service calls, backup traffic and management interfaces.

• TLS profile: the NCSC's TLS guidance (October 2026) prefers TLS 1.3, says "Only TLS 1.3 provides protection against quantum attacks when configured correctly", and requires TLS 1.0, 1.1 and SSL to be disabled. Its TLS 1.3 key exchange list starts with the hybrid groups X25519MLKEM768, SecP256r1MLKEM768 and SecP384r1MLKEM1024, with secp256r1 as the traditional fallback; enable 0-RTT only with replay mitigations.
• Database connections: require TLS on the server and verify certificate and host name on the client (for example PostgreSQL sslmode=verify-full). Encrypting without verifying stops sniffing but not an active interceptor.
• Service to service: mutual TLS through a service mesh authenticates both ends. Those certificates are machine identities; see the Machine Identity, PKI and Certificate Lifecycle Guide.
• Links and sites: MACsec (IEEE 802.1AE) encrypts Ethernet hop by hop and IPsec covers site-to-site tunnels. Both protect the link, not the application, so keep TLS on sensitive flows.
• Legacy ciphers: 64-bit block ciphers such as 3DES are exposed to the Sweet32 birthday attack in long sessions; remove them.

Encryption in use and end-to-end encryption

The Confidential Computing Consortium defines confidential computing as protecting "data in use by performing computation in a hardware-based, attested Trusted Execution Environment". It removes the host operator from the trust boundary, but you still trust the enclave code and the hardware maker. Release decrypt keys only to an enclave whose attested measurement matches policy.

• Database enclaves: Always Encrypted with secure enclaves (SQL Server 2019 and later, Azure SQL Database) adds pattern matching and comparisons on randomised columns.
• Homomorphic encryption: computes on ciphertext but is slow for general workloads and, the consortium notes, "does not directly provide integrity of the ciphertext nor of the computation". See the Privacy-Enhancing Technologies Guide.
• End-to-end and zero-knowledge designs: the provider never holds keys, which defeats provider compromise but removes provider-side recovery, search, malware scanning, DLP and e-discovery. Decide in advance how lost keys, leavers and legal holds are handled.

Post-quantum status for encrypted data

Quantum risk to encryption sits in RSA and elliptic curve key establishment and key wrapping, not in AES-256 bulk encryption. The urgent case is "harvest now, decrypt later": traffic recorded today that must stay confidential for a decade or more.

• Standards and timeline: ML-KEM (FIPS 203), ML-DSA (FIPS 204) and SLH-DSA (FIPS 205) were published in 2024. The NCSC sets discovery and an initial plan by 2028, highest-priority migration by 2031 and complete migration by 2035.
• Deploy now where it is cheap: enable hybrid ML-KEM groups on TLS 1.3 endpoints, starting with links carrying long-lived secrets, and keep ECDSA or RSA certificates until PQC authentication standards are finalised, as the NCSC advises.
• Stored data: envelope designs that wrap DEKs with RSA need a path to ML-KEM or symmetric wrapping; symmetric KEK hierarchies need no change. See the Post-Quantum Readiness Guide.

Encryption and data protection law

GDPR Article 32(1)(a) names "the pseudonymisation and encryption of personal data" as an example of appropriate security. Under Article 34(3)(a), communication to data subjects is not required where measures applied to the affected data "render the personal data unintelligible to any person who is not authorised to access it, such as encryption". In practice:

• The key must not be compromised with the data. Disk encryption on a server accessed while running, or ciphertext stolen with its key, leaves data intelligible to the attacker.
• The exemption covers individuals only. Notifying the supervisory authority under Article 33 is a separate assessment.
• Evidence must be ready. Record per store the layer, key and who could decrypt. See the GDPR Compliance Guide for Security Teams.

Rolling out an encryption programme

1. Classify and map: list stores holding personal, payment, health, credential and confidential data, with encryption layer, key owner and the identities that can decrypt.
2. Fix exposure first: unencrypted laptops and removable media, cleartext internal protocols carrying credentials or personal data, unencrypted backups, and keys stored beside their data.
3. Set the standard: media encryption by default, CMK for regulated cloud stores, field-level encryption for the highest-risk fields, TLS 1.3 preferred everywhere, and approved custody models.
4. Escrow before enforcing: confirm BitLocker recovery keys reach Entra ID or AD DS, back up TDE certificates and test restores on another server.
5. Pilot: log negotiated TLS versions and groups for two to four weeks before disabling anything; deploy disk and field-level encryption to a pilot ring and one application, measuring latency.
6. Roll out in rings with rollback: versioned TLS configuration; dual-read code (plaintext or ciphertext) until field migration completes; disable rather than delete KMS keys, with maximum deletion waiting periods.
7. Operate: review key policies quarterly, rotate KEKs, re-encrypt after suspected DEK exposure and include key unavailability in disaster recovery tests.

Detection and measurement

• KMS and HSM logs: alert on DisableKey, ScheduleKeyDeletion, PutKeyPolicy and new grants, decrypt calls from unusual identities or networks, and spikes in Decrypt or GenerateDataKey.
• Storage logs: alert on writes supplying unexpected or customer-provided keys and on mass object rewrites, early signs of T1486 (Data Encrypted for Impact) in cloud storage.
• Endpoints and network: device management reports of encryption and escrow status, enforced through conditional access; negotiated TLS versions and groups; cleartext on database or replication ports.

Metrics:

• Percentage of endpoints encrypted with recovery keys escrowed.
• Percentage of regulated data stores using CMK, and with field-level encryption on designated fields.
• Internal connections carrying sensitive data without TLS or certificate validation.
• Percentage of TLS 1.3 endpoints offering a hybrid ML-KEM group.
• Identities with decrypt rights per KEK, against a target.
• Time to restore an encrypted database on a clean server, from the last test.

Common failure patterns

• Key stored with the data: in the GoTo Breach 2023, attackers took encrypted backups together with an encryption key.
• Storage encryption treated as access control: a stolen cloud key with read rights gets plaintext. In the Codefinger S3 Ransomware 2025 case, stolen AWS keys were used to encrypt buckets with AWS's own encryption and a key the victim did not hold. Deny customer-provided key uploads by policy where you do not use them.
• Hard-coded keys: a key shipped in every installation is not secret; the Gladinet Hard-Coded Keys 2025 case shows static AES keys used to forge access tickets.
• Losing the key: deleted TDE certificates, unescrowed BitLocker keys and unreachable external key managers turn encryption into self-inflicted data loss.
• Deterministic encryption on small domains: yes/no flags or country codes leak through frequency analysis.
• Nonce reuse at scale: counters that reset on restart, or random 96-bit nonces used billions of times under one key, break AES-GCM (CWE-323). Per-object DEKs avoid the limit.

Framework mapping

Practitioner checklist

• Map every regulated data store to its encryption layer, key, key owner and decrypting identities.
• Enforce disk encryption on endpoints with recovery keys escrowed, and encrypt or block removable media.
• Use customer-managed keys for regulated cloud data and separate key administrators from data administrators.
• Apply randomised field-level encryption to the highest-risk fields; use deterministic mode or blind indexes only where search needs it.
• Adopt envelope encryption with per-object or per-tenant data keys and context bound as additional authenticated data.
• Require verified TLS on database, replication and service connections, and enable hybrid ML-KEM groups on TLS 1.3 where supported.
• Back up TDE certificates and test restores and key unavailability in disaster recovery exercises.
• Alert on key disable, deletion and policy changes, unusual decrypt volumes and unexpected customer-provided keys.
• Keep encryption evidence per store so a GDPR Article 34(3)(a) assessment takes hours, not weeks.

Standards and references

• NIST: SP 800-111, Guide to Storage Encryption Technologies for End User Devices
• NIST: SP 800-57 Part 1 Rev. 5, Recommendation for Key Management
• NIST: Cryptographic Module Validation Program (FIPS 140-3)
• NIST: SP 800-38G Rev. 1 (second public draft), Format-Preserving Encryption
• NCSC: Using TLS to protect data
• NCSC: Timelines for migration to post-quantum cryptography
• IETF: RFC 8446, The Transport Layer Security (TLS) Protocol Version 1.3
• AWS: AWS KMS cryptography essentials

Related NHI Mgmt Group resources: Cryptography Basics Guide · Cryptographic Key Management Guide · Machine Identity, PKI and Certificate Lifecycle Guide · Post-Quantum Readiness Guide · Privacy-Enhancing Technologies Guide