Every token an identity provider or authorisation server issues is only as trustworthy as the signing key behind it, yet many organisations never rotate those keys because rotation has broken something before. The result is signing keys that live for years, verifiers that hard-code a single public key, and no plan for an emergency rollover. This guide covers how JSON Web Key Sets (JWKS) publish keys, how kid selection and caching should work on the verifier side, a staged rotation procedure that does not cause outages, emergency rollover, and how to detect forged tokens. It is for identity platform owners, API and gateway teams, and anyone running their own token issuer, from a Keycloak instance to a custom service that signs internal JWTs.
Key takeaways
- Publish before you sign. A new key must appear in the JWKS and survive one full cache period before the issuer signs anything with it.
- Verifiers must fetch, not pin. Resolve keys from the issuer's
jwks_uribykid, refresh on an unknownkidwith a throttle, and never hard-code a public key. - Retire means remove. A key that is "retired" but still in the JWKS, or still trusted by a verifier, can still forge tokens, as the 2023 MSA signing key incident showed.
- Keep signing keys in an HSM or KMS. Rotation limits the damage window; non-exportable storage stops the key leaving in the first place.
- Rehearse the emergency rollover. The day you need it is the day a key is suspected stolen, and every verifier must cope with a key that disappears without notice.
What a JWKS is and how verifiers use it
A JSON Web Key Set (RFC 7517) is a JSON object with a keys array. Issuers publish it at a jwks_uri advertised in their metadata (OpenID Connect discovery at /.well-known/openid-configuration or OAuth metadata per RFC 8414). A verifier reads the token header, takes the kid, finds the matching key in the set and checks the signature. The JWT Security Guide covers token validation in general; this page focuses on the keys.
| JWK member | Meaning | Rotation relevance |
|---|---|---|
kty | Key type (RSA, EC, OKP) | Required (RFC 7517 Section 4.1). Changing algorithm family is a rotation too |
kid | Key identifier | RFC 7517 Section 4.5: used to select among keys during rollover; keys in a set SHOULD have distinct values |
use / key_ops | Signature (sig) or encryption (enc); allowed operations | RFC 8414: if a set holds signing and encryption keys, every key needs use |
alg | Intended algorithm | RFC 8725 Section 3.1: each key MUST be used with exactly one algorithm |
x5c, x5t#S256 | Certificate chain and thumbprint | Used where verifiers also check a certificate, such as SAML-aligned IdPs |
n, e / crv, x, y | Public key material | RSA at least 2048 bits, EC at least 224 bits in FAPI 2.0 Section 5.4 |
- Order means nothing: RFC 7517 Section 5 says array order does not indicate preference by default. Microsoft's guidance also warns not to rely on key order.
- Unknown members and types: unrecognised members MUST be ignored and keys with unrecognised
ktySHOULD be ignored, so adding a new key type does not break compliant verifiers. - Duplicate kids: FAPI 2.0 Section 5.4.2 says a JWK set should not contain multiple keys with the same
kid; where it does, Section 5.4.3 requires verifiers to considerkty,use,algandcrvto pick the right one.
Where rotation goes wrong: the two failure modes
| Failure mode | Typical cause | Consequence | Fix |
|---|---|---|---|
| Outage | Verifier pins one key; cache longer than the pre-publication window; no refresh on unknown kid | All tokens signed with the new key are rejected | Pre-publish, bounded cache, throttled refresh on miss |
| Outage | Old key removed while tokens it signed are still valid | Long-lived tokens fail mid-session | Keep the old key published until max token lifetime has passed |
| Forgery risk | Key never rotated, or retired key left in JWKS or in verifier trust | A stolen key signs valid tokens indefinitely | Scheduled rotation and true removal |
| Forgery risk | Verifier accepts keys from token headers (jku, x5u, jwk) | Attacker supplies their own key | Trust only the configured jwks_uri |
| Forgery risk | One key signs tokens for several issuers, tenants or token types | A token valid in one context is accepted in another | Bind keys to an issuer and token type (RFC 8725 Sections 3.8 and 3.12) |
Organisations often swing between these modes: a rotation causes an outage, rotation is paused, and the key then ages for years. Fixing the verifier side first is what makes routine rotation safe.
Verifier rules: kid lookup and caching
Microsoft's signing key rollover guidance for Entra ID, which states there is "no set or guaranteed time between these key rolls", gives a concrete caching model that works for any issuer:
- Look up by issuer and kid: cache keys per issuer (and per tenant for multi-tenant IdPs), each keyed by
kid. - Bounded cache: a 24-hour time to live with a background refresh every hour, or honour the issuer's
Cache-Controlheaders. Okta documents that it returns a short cache time near the end of the safe period so clients pick up changes. - Refresh on miss, with a throttle: on an unknown
kid, refresh once and retry, but not more often than about every 5 minutes, so a flood of tokens with randomkidvalues cannot turn the verifier into a denial of service against the IdP. - Keep last known good: if a refresh fails, keep using the cached set rather than failing closed on every request.
- Add jitter: spread refreshes (Microsoft suggests 12 hours plus or minus 1 hour in its library-style algorithm) so a fleet does not stampede the endpoint.
- Treat kid as untrusted input: RFC 8725 Section 3.10 warns that
kidvalues used in database or directory lookups can carry SQL or LDAP injection. Match against cached values only. - Ignore header-supplied key locations: FAPI 2.0 advises against
jkuandx5u; RFC 8725 requires an allowlist if they are used at all. - Bind key to issuer: RFC 8725 Section 3.8 requires that the signing key belongs to the token's
iss. A key from issuer A must never validate a token claiming issuer B. - Use the library: mainstream JWT and OIDC libraries implement these rules. Hand-written verification code is where pinning and missing refresh logic usually appear.
A zero-downtime rotation procedure
Rotation has three time windows: the verifier cache period (C), the maximum lifetime of tokens signed by the old key (T), and a safety margin (M). Every step below waits for one of them.
- Generate: create the new key pair in the HSM or KMS with a new, unique
kid(a random value or the RFC 7638 thumbprint style). Do not reusekidvalues. - Publish: add the new public key to the JWKS alongside the current key. Do not sign with it yet.
- Wait C plus M: every verifier that respects caching now holds both keys. Check JWKS fetch logs or a canary verifier to confirm.
- Switch signing: start signing new tokens with the new key. Watch signature validation failures at gateways and relying parties for at least one hour.
- Wait T plus M: tokens signed with the old key expire naturally. For refresh tokens that are self-contained JWTs, T is the refresh token lifetime, which is a reason to keep those opaque.
- Remove: delete the old public key from the JWKS and disable or destroy the private key version in the KMS according to your retention policy.
- Record: log the rotation in change control with timestamps,
kidvalues and the person or pipeline that performed it.
Testing: rehearse in a staging issuer with production-like verifiers, including the oddest ones (legacy gateways, partner integrations, mobile apps with bundled config). Rollback: during step 4, switching back to the old key is safe because it is still published; after step 6, rollback means re-publishing it, so only remove the old key once step 4 has run cleanly.
Choosing a cadence
| Context | Practical cadence | Notes |
|---|---|---|
| Large hosted IdPs | Set by the provider | Okta documents about four times a year, subject to change without notice; Entra ID publishes no fixed schedule |
| Self-hosted IdP or AS, keys in HSM or KMS | Every 3 to 12 months, automated | Automation matters more than the exact interval |
| Keys that have ever been exportable or on developer machines | Rotate now, then move to non-exportable storage | Treat as possibly copied |
| Internal service-to-service JWT issuers | Same as self-hosted, with shorter token lifetimes | Short T makes frequent rotation cheap |
| Suspected compromise | Immediately (emergency rollover) | See below |
NIST SP 800-57 Part 1 frames this as cryptoperiods, with separate originator-usage (signing) and recipient-usage (verification) periods. That distinction is exactly the publish, sign, retire sequence above. Small organisations using a hosted IdP mainly need verifiers that follow the caching rules; organisations running their own issuers own the whole procedure.
Emergency rollover
When a signing key may be compromised, the aim reverses: stop trust in the old key as fast as possible and accept some disruption.
- Decide: the identity platform owner and incident lead agree that the key is suspected compromised. Record the decision time.
- Generate and publish a new key and switch signing to it immediately, without the cache wait.
- Remove the compromised key from the JWKS at once and push a forced refresh to verifiers you control (restart, cache flush, configuration push).
- Revoke what it signed: invalidate sessions and refresh tokens issued under the old key, and require re-authentication for high-risk applications.
- Hunt: search for tokens with the old
kidpresented after the cut-over, and for tokens with no matching issuance record. - Preserve evidence: keep KMS or HSM access logs, signing service logs, JWKS change history and gateway logs for the investigation.
Verifiers that follow the caching rules recover within one refresh; pinned verifiers fail until someone updates them, which is why the emergency plan should list them by name. Microsoft's guidance explicitly calls hard-coded keys "inherently less secure and less reliable" because of emergency out-of-band rollovers.
Detection and measurement
- Issuance versus use: correlate tokens seen at gateways and APIs with the issuer's issuance log. A validly signed token with no issuance record points to a stolen key (ATT&CK T1606 Forge Web Credentials). The Coupang breach is a case where forged access tokens signed with a former engineer's unrevoked key were accepted as genuine.
- Key access: alert on any export, unusual sign volume or new principal calling sign on the token signing key in the KMS or HSM (T1552.004 Private Keys).
- JWKS changes: alert on any change to a published JWKS outside the rotation pipeline.
- Old kids: alert when a removed
kidappears in presented tokens.
| Metric | Target |
|---|---|
| Age of the oldest active signing key per issuer | Within policy (for example under 12 months) |
| Signing keys in non-exportable HSM or KMS storage | 100% for production issuers |
| Verifiers using dynamic JWKS resolution rather than pinned keys | 100%, with named exceptions |
| Signature failures during the last planned rotation | Near zero |
| Time to complete an emergency rollover in the last rehearsal | Measured in hours, not days |
| Validly signed tokens without an issuance record | Zero, each one investigated |
Lessons from incidents
- The 2023 MSA signing key incident: as summarised in our Storm-0558 and Cloud Key Theft Profile, a Microsoft consumer signing key created in 2016 was still trusted in 2023. Its retirement had been planned for 2021, manual rotation was halted after an outage, and nothing alerted on the old key remaining valid. Token validation flaws let it sign tokens accepted for enterprise mail. The Cyber Safety Review Board, reporting in April 2024, called the intrusion preventable. Microsoft later reported moving its signing keys into HSMs with automatic rotation.
- Coupang 2025: a token signing key held by a departed developer was never revoked or rotated, and keys were kept on developer computers. Offboarding must include every signing key a leaver could reach.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev. 5 | SC-12 Cryptographic Key Establishment and Management | Generation, distribution, rotation and destruction of signing keys |
| NIST SP 800-53 Rev. 5 | SC-13 Cryptographic Protection; CM-3 Configuration Change Control | Approved algorithms; JWKS changes under change control |
| NIST SP 800-53 Rev. 5 | IR-4 Incident Handling; AU-6 Audit Record Review | Emergency rollover and forged-token hunting |
| NIST SP 800-57 Part 1 Rev. 5 | Cryptoperiods | Originator-usage and recipient-usage periods |
| NIST CSF 2.0 | PR.AA-01, PR.DS-02, DE.CM-03 | Credentials managed; integrity of tokens in transit; monitoring |
| OWASP ASVS 5.0 | V9.1.1, V9.1.2, V9.1.3 | Signature validation, algorithm allowlist, keys from trusted pre-configured sources |
| MITRE ATT&CK | T1606, T1552.004 | Forged web credentials; theft of private keys |
Common failure patterns
- Rotation paused after an outage: the fix was to stop rotating rather than to fix the verifiers. Years later, a key nobody remembers is still trusted.
- Pinned keys in partner integrations: a partner copied your public key into their config. Your rotation breaks them, so you stop rotating. Publish a JWKS and make dynamic resolution a contract term.
- Signing key on disk: a PEM file in a container image or on a build server can be copied silently. Rotation shortens the exposure; non-exportable storage removes it.
- Unthrottled refresh on unknown kid: every forged token triggers a JWKS fetch, and an attacker turns your verifiers into a load test against the IdP.
- One key for everything: the same key signs ID tokens, access tokens and internal service tokens across environments, so a test environment leak compromises production.
- No issuance correlation: nobody can tell a forged token from a real one because issuance is not logged in a way that can be matched.
Practitioner checklist
- List every token issuer you run or rely on, its
jwks_uri, key ages and storage location. - Find and fix verifiers that pin keys; make dynamic JWKS resolution the default in shared libraries.
- Enforce kid lookup per issuer, bounded caching, throttled refresh on miss and last known good fallback.
- Ignore
jku,x5uand embeddedjwkheaders; allowlist algorithms per key. - Move production signing keys into non-exportable HSM or KMS storage.
- Automate the publish, wait, switch, wait, remove sequence and record each run in change control.
- Write and rehearse an emergency rollover runbook that names pinned verifiers.
- Correlate presented tokens with issuance logs and alert on unknown or removed
kidvalues. - Add signing key rotation to offboarding for anyone who had key access.
Standards and references
- IETF: RFC 7517 JSON Web Key (JWK)
- IETF: RFC 8725 JSON Web Token Best Current Practices
- IETF: RFC 8414 OAuth 2.0 Authorization Server Metadata
- OpenID Foundation: FAPI 2.0 Security Profile
- NIST: SP 800-57 Part 1 Rev. 5 Recommendation for Key Management
- Microsoft Learn: Signing key rollover in the Microsoft identity platform
- Okta Developer: Key rotation
Related NHI Mgmt Group resources: JWT Security Guide · Storm-0558 and Cloud Key Theft Profile · Cryptographic Key Management Guide · JWT Client Authentication Guide · Token Lifetime Policy Design Guide