Cryptography Basics Guide for Security Teams

Most cryptographic failures in real systems come from choosing the wrong primitive, misusing a correct one or losing control of the keys, not from anyone breaking AES. Security teams are asked to approve TLS configurations, judge whether "encrypted" in a vendor questionnaire means anything, switch on FIPS mode, plan the move away from SHA-1 and RSA-2048, and respond when a key leaks. This guide covers the building blocks and what each one proves, the NIST status of common algorithms and key sizes, TLS cipher suites, FIPS 140-3 and the post-quantum timeline. It is for security engineers, architects and auditors who must make and defend these decisions.

Key takeaways

• Match the primitive to the property you need. Encryption gives confidentiality, a MAC gives integrity between parties who share a key, a signature gives integrity and origin that anyone with the public key can check. None of them substitutes for another.
• Use authenticated encryption by default. AES-GCM and ChaCha20-Poly1305 detect tampering; unauthenticated CBC and ECB invite padding oracle and pattern leaks, and a single repeated GCM nonce destroys its integrity guarantee.
• Know the NIST dates. Three-key TDEA encryption has been disallowed since the end of 2023, SHA-1 is being retired by 31 December 2030, and quantum-vulnerable RSA and elliptic curve algorithms at 112-bit strength are proposed for deprecation after 2030 and disallowance after 2035.
• FIPS mode is a compliance setting, not a security grade. It restricts a system to a validated module's approved algorithms; it does not fix key handling, and FIPS 140-2 certificates move to the Historical list after 21 September 2026.
• Keys are the real asset. A strong algorithm with a hard-coded, shared or unrotated key fails completely, so cryptography work starts with an inventory of keys and where they live.

The building blocks and what each one proves

NIST SP 800-57 Part 1 groups approved cryptography into hash functions, symmetric-key algorithms, asymmetric-key algorithms and random bit generators. Many design reviews go wrong here, by expecting a primitive to provide a property it was never meant to provide.

One rule from SP 800-57 is worth quoting to developers: "In general, a single key shall be used for only one purpose." A key that both encrypts and signs weakens both uses and widens the blast radius of a leak. Keys are also credentials: private keys, HMAC secrets and certificates held by workloads are non-human identities that need owners, vaulted storage, rotation and revocation. See the Cryptographic Key Management Guide and the Secrets Management Guide.

Symmetric encryption and modes of operation

AES is a block cipher: a pseudo-random permutation on 128-bit blocks under a 128, 192 or 256-bit key. On its own it encrypts exactly one block, so the mode of operation decides most of the real security.

• ECB: identical plaintext blocks give identical ciphertext, so structure leaks. The draft SP 800-131A Rev. 3 proposes disallowing it for data encryption.
• CBC: chains blocks from a random initialisation vector (IV) but provides no integrity; a server that reveals whether padding was valid becomes a padding oracle. Use it only with Encrypt-then-MAC.
• CTR: a fast stream mode, but malleable: flipping a ciphertext bit flips the plaintext bit. Needs a MAC.
• GCM: CTR encryption plus GMAC authentication, with additional authenticated data (AAD) for headers that need integrity but not secrecy. SP 800-38D recommends 96-bit IVs and caps random-IV use at 2³² invocations per key; if one IV repeats, "the authentication assurance essentially is lost."
• ChaCha20-Poly1305: a stream cipher plus one-time authenticator, fast and constant-time in software without AES hardware, so common on mobile and embedded devices. Its 96-bit nonce must also never repeat under a key.
• Releasing unverified plaintext: authenticated decryption must check the tag before returning any plaintext; streaming APIs that release it early reopen the attacks AEAD closes.

Legacy algorithms still turn up in audits: 3DES (three-key TDEA) encryption was disallowed after 2023 under SP 800-131A Rev. 2, with decryption kept for legacy data, and RC4 has no approved use.

Hashing, MACs, key derivation and randomness

A hash should resist preimage, second preimage and collision attacks. Because of the birthday bound, an n-bit hash gives about n/2 bits of collision resistance, so SHA-224 offers only 112 bits, which is why the draft SP 800-131A Rev. 3 proposes deprecating 224-bit hashes for applying protection through 31 December 2030.

• MD5 and SHA-1: practical collisions exist for both. SHA-1 is disallowed for signature generation, and NIST will retire it by 31 December 2030, advising users to "migrate to SHA-2 or SHA-3 as soon as possible."
• File integrity: sha256sum only helps if the expected digest came from a separate trusted channel, such as a signed release file. md5sum catches accidental corruption, not tampering.
• Length extension: SHA-256 is a Merkle-Damgard hash, so SHA-256(secret || message) lets an attacker append data and forge a valid digest. HMAC, SHA-3 and KMAC are not affected. Never build a MAC from a bare hash.
• HMAC: the default MAC; keys under 112 bits are disallowed for generation. Compare tags in constant time.
• HKDF: an extract-then-expand KDF that turns a high-entropy secret into separate keys, with the info field binding each key to one purpose. TLS 1.3 derives its traffic keys this way.
• Passwords: use Argon2id, scrypt, bcrypt or PBKDF2, never a fast hash or encryption. See the Password Security Guide.
• Randomness: generate keys, IVs, nonces and tokens from the OS CSPRNG (getrandom(2), Python secrets, Java SecureRandom, Go crypto/rand). OWASP lists Math.random(), java.util.Random and Python's random as unsafe. SP 800-90A defines Hash_DRBG, HMAC_DRBG and CTR_DRBG. Devices generating keys at first boot with little entropy, and cloned VM images, have produced duplicate keys.
• Nonce versus IV: a nonce must be unique; a CBC IV must also be unpredictable. A counter is a good GCM nonce and a bad CBC IV.

Asymmetric cryptography: signatures and key exchange

Public key cryptography rests on hard problems: integer factorisation (RSA), discrete logarithms in finite fields and on elliptic curves (DH, ECDH, ECDSA, EdDSA), and lattices or hash functions for the post-quantum standards.

• RSA padding matters more than key size: use RSA-PSS (probabilistic signature scheme) for signatures and RSA-OAEP (optimal asymmetric encryption padding) for encryption. PKCS#1 v1.5 encryption padding is the source of the Bleichenbacher family of padding oracle attacks, and SP 800-131A Rev. 2 disallowed it for key transport after 2023.
• ECDSA versus EdDSA: ECDSA needs a fresh, secret, unbiased nonce for every signature; a repeated or predictable nonce reveals the private key. Ed25519 (on Curve25519) derives its nonce deterministically and avoids that class of bug. P-256 and P-384 are the usual choice where FIPS validation is required.
• Forward secrecy: ephemeral Diffie-Hellman (ECDHE with X25519 or P-256) uses a new key pair per session, so a later theft of the server's long-term key does not decrypt recorded traffic. TLS 1.3 removed static RSA and static DH key exchange for this reason.
• Validate what you receive: public key validation stops invalid-curve and small-subgroup attacks; use libraries that do it rather than raw curve arithmetic.
• Signature failures are hard failures: a signature that does not verify against the expected key means the data changed or came from someone else. A bank or any relying party should reject the transaction, not fall back to a weaker check, and log it.
• Key fingerprints: compare full SHA-256 fingerprints out of band, not the first few characters.

Key sizes and NIST status

NIST expresses strength in bits of security from the set {80, 112, 128, 192, 256}; SP 800-57 states that 80 bits "is no longer considered sufficiently secure." The table combines the final SP 800-131A Rev. 2 (March 2019), the Rev. 3 initial public draft (October 2024) and the NIST IR 8547 initial public draft (November 2024). Draft dates may change before they are final.

A practical baseline for new systems: AES-256-GCM or ChaCha20-Poly1305, SHA-256 or stronger, ECDHE with X25519 or P-256, and Ed25519, ECDSA P-256 or RSA-3072 PSS signatures, with a plan for ML-KEM and ML-DSA. See the Post-Quantum Readiness Guide.

TLS cipher suites and configuration

NIST SP 800-52 Rev. 2 requires government TLS servers and clients to support TLS 1.2 with FIPS-based cipher suites and required TLS 1.3 support by 1 January 2024. TLS 1.3 is now specified in RFC 9846, which obsoletes RFC 8446 and the TLS 1.2 specification RFC 5246.

• TLS 1.3 removes most choices: all its suites are AEAD, key exchange is always ephemeral, and renegotiation, compression and DSA are gone. Configuration effort shifts to groups (x25519 is code point 0x001D, secp256r1 0x0017) and signature schemes (rsa_pss_rsae_sha256 0x0804, ecdsa_secp256r1_sha256 0x0403, ed25519 0x0807).
• TLS 1.2 cipher strings: allow only ECDHE key exchange with AES-GCM or ChaCha20-Poly1305. Remove static RSA key exchange (no forward secrecy), CBC-mode suites where clients allow, 3DES, RC4, NULL and EXPORT suites. Disable TLS 1.0 and 1.1.
• 0-RTT: RFC 8446 warns that early data "is not forward secret" and has "no guarantees of non-replay between connections". Leave 0-RTT off unless the application only accepts idempotent requests in it.
• Downgrade protection: an attacker in the path (ATT&CK T1557, Adversary-in-the-Middle) exploits any fallback to weaker versions or suites (T1689, Downgrade Attack). Disable what you do not need rather than trusting clients to prefer strong options.
• Libraries: OpenSSL, BoringSSL, LibreSSL, GnuTLS and Windows Schannel each use different syntax and defaults, so test the deployed result, not the configuration file. Certificates are the identity half of TLS; see the Machine Identity, PKI and Certificate Lifecycle Guide.

FIPS mode and validated modules

FIPS 140-3 is the standard for cryptographic modules, validated through NIST's Cryptographic Module Validation Program (CMVP). Its four security levels rise from Level 1 (basic requirements) to Level 4 (the strongest physical protection and tamper response). The CMVP no longer accepts FIPS 140-2 submissions and states that FIPS 140-2 modules can be accepted by agencies "through September 21, 2026", after which those certificates move to the Historical list and may be used only in existing systems.

• What FIPS mode does: it makes the platform use a validated module in its approved mode, blocking non-approved algorithms. It is needed for US federal systems and contracts that require it.
• Windows: the Group Policy setting "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing" (registry HKLM\System\CurrentControlSet\Control\Lsa\FipsAlgorithmPolicy\Enabled) restricts CNG (Cryptography API: Next Generation) and Schannel to approved algorithms. Applications using their own libraries are not affected.
• Linux: distributions with validated modules enable FIPS mode through a kernel boot parameter and system crypto policy; /proc/sys/crypto/fips_enabled reads 1 when active. Follow the vendor's procedure; enabling at installation ensures keys are generated inside the approved module.
• OpenSSL 3: FIPS support is a separately loaded provider configured for approved-only use. Check the exact provider version against its CMVP certificate; a newer build is not automatically validated.
• Trade-off: FIPS mode breaks applications that call non-approved algorithms, even for non-security uses such as MD5 checksums. Enable it where required, pilot first, and do not treat it as evidence that keys are handled well.

Choosing between options

• Small organisations: rely on current platform defaults, cloud KMS and managed TLS, and spend effort on inventory, key storage and library patching.
• Large or regulated organisations: need a written cryptographic standard, a crypto inventory across code, certificates and HSMs, FIPS-validated modules where contracts require them, and a post-quantum plan with owners and dates.

Rolling out a cryptography baseline

1. Inventory: scan TLS endpoints, certificate stores, code repositories, HSM and KMS key lists, and VPN and SSH configurations. Record algorithm, key size, library, owner and data protected.
2. Write the standard: approved, legacy-only and prohibited algorithms based on SP 800-131A, minimum TLS settings, key storage rules and rotation triggers.
3. Fix the dangerous items first: hard-coded or shared keys, unauthenticated encryption, SHA-1 or MD5 signatures, TLS 1.0 and 1.1, and RSA keys under 2048 bits.
4. Test in audit mode: log negotiated TLS versions and suites for two to four weeks before disabling anything, and list the clients that would break.
5. Roll out in rings: internal services, then a pilot group of external endpoints, then the rest. Keep the previous configuration versioned so rollback is a redeploy.
6. Rotate and re-encrypt: where weak algorithms or exposed keys protect data, re-encrypt under new keys rather than wrapping old ciphertext.
7. Add guardrails: CI rules for weak primitives and insecure random calls, and secret scanning for private keys.

Detection and measurement

• Log: negotiated TLS version, cipher and group at load balancers and sensors (Zeek's ssl.log records them); certificate issuance and expiry; KMS and HSM key use, export and deletion; signature verification failures.
• Scan: run TLS scans (for example testssl.sh or Nmap's ssl-enum-ciphers) on a schedule and after changes.
• Alert on: private key access from unexpected identities, new signing keys, repeated signature or MAC failures, and any connection negotiating TLS 1.0, 1.1 or a discouraged suite.

Metrics that show progress:

• Percentage of TLS endpoints offering only TLS 1.2 and 1.3 with AEAD suites.
• Number of SHA-1, MD5 or sub-2048-bit RSA uses remaining in the inventory.
• Hard-coded keys found in code per month, and median time to remove and rotate them.
• Percentage of keys with a named owner, storage location and rotation date.
• Percentage of quantum-vulnerable key establishment uses with a migration plan.

Common failure patterns

• Hard-coded keys: a key compiled into a product is shared by every customer and can never be truly secret. The Gladinet Hard-Coded Keys 2025 case shows static AES keys used to forge access tickets.
• Poorly protected signing keys: the holder can mint trusted tokens or software. The Microsoft Storm-0558 Key Breach 2023 involved a stolen signing key used to forge access, and the MSI Signing Keys Leak 2023 exposed firmware signing keys. ATT&CK tracks related abuse under T1649 (Steal or Forge Authentication Certificates) and T1553.002 (Code Signing).
• Encryption key stored with the data: in the GoTo Breach 2023, attackers took encrypted backups together with an encryption key. Encryption at rest helps only if the key sits in a separate system with separate access.
• Nonce reuse: counters that reset on restart, or random nonces used for billions of messages under one key, quietly break GCM.
• Padding oracles: CBC without a MAC, with distinguishable errors, lets attackers decrypt data byte by byte.
• Fixing the setting but not the keys: when a crypto misconfiguration has been exploited, contain first: stop the exposure, identify which keys and data were reachable, then revoke and rotate those keys before deploying the corrected configuration.

Framework mapping

Practitioner checklist

• Build and maintain a crypto inventory: algorithm, key size, library, key location and owner.
• Publish an approved-algorithm standard based on SP 800-131A, with legacy and prohibited lists.
• Use AEAD (AES-GCM or ChaCha20-Poly1305) and enforce unique nonces per key.
• Remove MD5 and SHA-1 from signatures, certificates and integrity checks that face tampering.
• Restrict TLS to 1.2 and 1.3 with ECDHE and AEAD suites; leave 0-RTT off by default.
• Use OS CSPRNG APIs and store keys apart from the data they protect, in an HSM, KMS or vault, never in code.
• Enable FIPS mode only where required, after a pilot, and confirm module certificates are FIPS 140-3 or still valid.
• Plan the post-quantum move for 112-bit and other quantum-vulnerable keys against the 2030 and 2035 dates.

Standards and references

• NIST: SP 800-57 Part 1 Rev. 5, Recommendation for Key Management
• NIST: SP 800-131A Rev. 2, Transitioning the Use of Cryptographic Algorithms and Key Lengths
• NIST: IR 8547, Transition to Post-Quantum Cryptography Standards (initial public draft)
• NIST: FIPS 203, Module-Lattice-Based Key-Encapsulation Mechanism Standard
• NIST: FIPS 204, Module-Lattice-Based Digital Signature Standard
• NIST: SP 800-38D, Galois/Counter Mode (GCM) and GMAC
• NIST: Cryptographic Module Validation Program (FIPS 140-3)
• OWASP: Cryptographic Storage Cheat Sheet

Related NHI Mgmt Group resources: Cryptographic Key Management Guide · Machine Identity, PKI and Certificate Lifecycle Guide · Post-Quantum Readiness Guide · Secrets Management Guide · Network Ports and Protocols Security Guide