When a service account, API key, OAuth token or agent credential is compromised, the usual incident metrics (time to detect, time to contain) miss the questions that decide whether the attacker is really gone. Did you revoke every credential the identity could reach, or only the one you found? How many systems did it actually touch? How long were secrets that it exposed still valid? This guide defines a practical set of measures for incidents involving non-human identities (NHIs): what each one means, how to calculate it, where the data comes from, and how to use it in the post-incident review and in quarterly reporting. It is for incident responders, identity and secrets teams, and security leaders who report on incident outcomes. Programme-level identity KPIs are covered in the Identity Security Metrics Guide; this page is about measuring a specific compromise and the response to it.
Key takeaways
- Measure the credential, not just the incident. For NHI compromises the clock that matters runs from first exposure of the secret to the moment it stops working, which can be weeks longer than the incident ticket.
- Rotation completeness is the headline number. Cloudflare rotated thousands of credentials after the Okta compromise, but one service token and three service accounts were missed, and that is how the attacker got in.
- Blast radius needs two numbers. Record what the identity could reach (potential) and what it actually touched (observed), so the gap shows how much least privilege would have saved.
- Use NIST SP 800-55 measure types. Split metrics into implementation, effectiveness, efficiency and impact, as NIST SP 800-55 Vol. 1 does, so a dashboard does not mix "did we do it" with "how fast" and "what did it cost".
- Few metrics, measured every time. NIST notes that with quantitative metrics "less is often more". Eight measures collected for every NHI incident beat thirty collected once.
Why NHI incidents need their own metrics
NIST SP 800-61 Rev. 3 (April 2025) frames incident response inside the CSF 2.0 functions and sets the goals of reducing the number and impact of incidents and improving the efficiency and effectiveness of detection, response and recovery. For human account compromise the response is usually bounded: reset the password, revoke sessions, re-register MFA. NHI compromises behave differently.
- Credentials fan out: one stolen CI token or vault credential exposes every secret it can read, so the number of affected credentials grows after the first finding. The CircleCI breach began with one engineer's stolen SSO session and ended with customers told to rotate every secret stored on the platform during the affected period.
- Owners are unclear: rotation waits on someone confirming what an identity is for, which shows up as long rotation times.
- Rotation can break production: teams delay or skip rotation of credentials they believe are unused or fragile. That belief was wrong in the Cloudflare Thanksgiving breach.
- Logs are thin: many SaaS and legacy systems do not log NHI activity well, so "what did it touch" is often an estimate.
The timeline: anchor points to record
Every NHI incident record should capture these timestamps. Most metrics below are differences between them.
| Anchor | Definition | Typical source |
|---|---|---|
| T0 Exposure | Earliest time the credential could have been obtained (commit time, log write, compromised host infection, supplier breach date) | Git history, secret scanner, EDR, supplier notice |
| T1 First malicious use | Earliest confirmed use by the attacker | Cloud audit logs, SaaS audit logs, gateway logs |
| T2 Detection | First alert, report or finding that led to the incident | SIEM, scanner, third-party report |
| T3 Declaration | Incident formally opened | Case management |
| T4 First revocation | The first compromised credential stops working | Identity provider, secrets manager, provider console |
| T5 Scope complete | Responders sign off the list of affected identities and credentials | Incident decision log |
| T6 Last revocation | The last credential in scope stops working, verified by a failed authentication test | Rotation records, test results |
| T7 Closure | Incident closed after verification and review | Case management |
Record decisions and these timestamps as the incident runs; the Incident Log and Decision Record Template has a suitable structure.
The core NHI incident metrics
The table groups measures by the four types defined in NIST SP 800-55 Vol. 1: implementation (was it done), effectiveness (did it work), efficiency (how fast) and impact (what it meant for the organisation).
| Metric | Type | Calculation | What it tells you |
|---|---|---|---|
| Credential exposure window | Efficiency | T6 minus T0, per credential; report median and maximum | How long a stolen secret was usable. The number attackers care about |
| Dwell time | Efficiency | T2 minus T1 | How long the attacker operated unseen with the identity |
| Time to first revocation | Efficiency | T4 minus T2 | Speed of the first containment action |
| Time to full revocation | Efficiency | T6 minus T2 | Whether containment finished or stalled on hard cases |
| Rotation completeness | Effectiveness | Credentials verified revoked, divided by credentials in final scope | Whether anything was missed. Anything under 100% at closure is an open risk |
| Scope growth | Effectiveness | Credentials in final scope divided by credentials in initial scope at T3 | How well first triage understood the fan-out |
| Potential blast radius | Impact | Resources, data stores and identities the compromised NHIs were permitted to reach | Exposure created by the permissions |
| Observed blast radius | Impact | Resources actually accessed by the attacker, from logs | Real impact; the ratio to potential shows excess privilege |
| Ownerless identities in scope | Implementation | In-scope NHIs with no current owner at T3 | Where ownership gaps slowed the response |
| Log coverage | Implementation | In-scope NHIs whose activity was logged for the whole T0 to T6 period | Confidence in the observed blast radius |
| Recurrence | Effectiveness | Repeat incidents within 12 months with the same root cause category | Whether lessons were actually applied |
| Response cost | Impact | Staff hours, outage minutes from rotation, external costs | Business cost; supports the case for prevention |
Worked example from public reporting
Using the dates in the NHI Mgmt Group analysis of the Cloudflare incident: Cloudflare's Okta instance was accessed on 18 October 2023; the attacker used unrotated credentials from 14 November, was detected on 23 November and cut off on 24 November. The remediation programme rotated more than 5,000 production credentials and triaged 4,893 systems. The missed items were one service token and three service accounts. Rotation completeness after the Okta incident was therefore very high by count yet still insufficient, which is why the metric must be read alongside a list of the exceptions and the privilege of each one, not as a percentage alone.
Measuring blast radius
Blast radius is the metric teams most often skip because it is hard. A workable method:
- List in-scope identities: every compromised NHI, plus any identity whose credentials were readable by it (secrets in variables, vault paths, files).
- Compute potential reach: export effective permissions for each identity from the cloud provider, SaaS admin console or vault policy. Count resources, data stores and further identities reachable.
- Compute observed reach: query audit logs for each identity from T0 to T6. Count distinct resources accessed, separating reads, writes and administrative actions.
- Record confidence: mark each identity's observed figure as complete, partial or unknown based on log coverage.
- Report the ratio: observed divided by potential. A low ratio with high potential means the attacker had far more access than they needed, and that the permissions are the finding.
Map the attacker's use of the identity to ATT&CK so findings compare across incidents: typically T1552.001 Credentials In Files for secrets found in code or configuration, T1528 Steal Application Access Token and T1550.001 Application Access Token for OAuth and API tokens, and T1078.004 Cloud Accounts for cloud identities.
Data sources and how to collect them
| Metric input | Data source | Collection tip |
|---|---|---|
| Exposure time (T0) | Secret scanner findings, Git history, EDR timeline, supplier breach notice | Take the earliest defensible time and record the evidence; do not default to detection time |
| First use and observed reach | Cloud audit logs, SaaS audit logs, API gateway logs, vault audit logs | Pre-build saved queries keyed on identity ID so they run in minutes during an incident |
| Revocation times | Identity provider and secrets manager events, provider console audit | Log the revocation event, then a failed authentication test with the old credential |
| Potential reach | Effective permission exports, entitlement tooling | Snapshot at T3, before remediation changes permissions |
| Ownership | NHI inventory, agent registry | Record owner status at T3, not after the incident has fixed it |
| Cost | Time tracking, change records, outage records | Tag rotation-caused outages to the incident |
Saved queries per identity type are described in the Detection Engineering for Non-Human Identities Guide, and fast scoping depends on a reliable inventory, covered in NHI Discovery Across Cloud, SaaS, CI/CD and On-Premises Environments.
Implementation: adding NHI metrics to your incident process
- Add fields to the case template: the eight anchor timestamps, an in-scope credential list with status, and owner status at declaration.
- Define closure criteria: no NHI incident closes until rotation completeness is 100% or each remaining credential has a documented, approved exception with a deadline.
- Build the queries before you need them: one saved search per identity type for first use and observed reach. Test them in an exercise.
- Pilot on the next three incidents: collect the metrics manually, note which inputs were hard to get, and fix the data sources first.
- Automate what repeats: pull revocation events from the identity provider and secrets manager into the case record automatically.
- Review quarterly: trend the efficiency measures, list every incident below 100% rotation completeness, and feed findings into the programme roadmap.
- Rollback: if the extra fields slow live response, make them mandatory at closure rather than during containment, and keep the timestamps as the minimum.
Use the Secrets Rotation Runbook Template so rotation evidence is captured in the same format each time, and the Leaked Credential Response Playbook for the response steps the metrics measure.
Targets and decision guidance
NIST SP 800-55 Vol. 1 describes a target as a range or a bound that marks satisfactory performance, recommends interim and final targets, and suggests setting acceptable ranges from historical averages. The targets below are illustrative only; set your own from your first quarter of data.
| Metric | Illustrative interim target | Illustrative mature target |
|---|---|---|
| Time to first revocation (critical NHI) | Under 4 hours | Under 1 hour |
| Time to full revocation | Under 5 days | Under 24 hours |
| Rotation completeness at closure | 100% or approved exceptions | 100%, verified by failed authentication test |
| Ownerless identities in scope | Under 20% | Zero |
| Log coverage of in-scope NHIs | Over 70% | Over 95% |
| Scope growth | Tracked | Under 1.5 times initial scope |
- Small organisations: collect the timestamps and rotation completeness only. They need no tooling beyond the case record and give most of the insight.
- Large organisations: add blast radius and cost, and report the efficiency measures by identity type (cloud, SaaS OAuth, CI/CD, agent) so slow areas stand out.
- Supplier-originated incidents: T0 is the supplier's exposure date, not your notification date. Track the gap between them as a third-party measure.
Common failure patterns
- Clock starts at detection: exposure windows look short because T0 is set to the alert time, hiding weeks of validity.
- Rotation by assumption: credentials believed unused are skipped, as in the Cloudflare case. Measure verified revocation, not intended revocation.
- Percentages without exceptions: a 99.9% completeness figure hides the one token with administrative rights.
- No failed-authentication test: a rotation ticket marked done while the old secret still works in a second system or cached configuration.
- Blast radius from permissions only: reporting potential reach as impact, or observed reach without noting that logs were incomplete.
- Metrics collected once: a detailed post-incident report with numbers that are never trended, so recurrence is invisible.
Framework mapping
| Framework | Reference | Relevant metric |
|---|---|---|
| NIST CSF 2.0 | RS.AN-03, RS.AN-08 | Observed blast radius, scope growth |
| NIST CSF 2.0 | RS.MI-01, RS.MI-02 | Time to first and full revocation, rotation completeness |
| NIST CSF 2.0 | ID.IM-03, ID.IM-04 | Recurrence, quarterly review |
| NIST SP 800-53 Rev. 5 | IR-4, IR-5, IR-8, PM-6 | Incident handling, monitoring and measures of performance |
| NIST SP 800-53 Rev. 5 | IA-5, AC-2, AU-6 | Authenticator revocation, account management, log review for observed reach |
| OWASP NHI Top 10 (2025) | NHI1 Improper Offboarding, NHI2 Secret Leakage, NHI5 Overprivileged NHI, NHI7 Long-Lived Secrets | Ownerless in scope, exposure window, blast radius ratio, credential exposure window for static secrets |
| CIS Controls v8.1 | Control 17 | Incident response management and post-incident review |
Practitioner checklist
- Add the eight anchor timestamps to the incident case template.
- Set T0 from evidence of exposure, never from detection time.
- Keep an in-scope credential list with verified revocation status for each item.
- Test every revoked credential with a failed authentication attempt.
- Snapshot effective permissions at declaration for potential blast radius.
- Pre-build audit log queries per identity type for observed blast radius.
- Block closure below 100% rotation completeness without approved exceptions.
- Trend the measures quarterly and report incidents with exceptions by name.
- Mark all targets as interim until you have a baseline.
Standards and references
- NIST: SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management
- NIST: SP 800-55 Vol. 1, Measurement Guide for Information Security
- NIST: Cybersecurity Framework 2.0
- NIST: SP 800-53 Rev. 5
- OWASP: Non-Human Identities Top 10 (2025)
- CISA: Federal Government Cybersecurity Incident and Vulnerability Response Playbooks
- MITRE ATT&CK: Credentials In Files
- MITRE ATT&CK: Application Access Token
Related NHI Mgmt Group resources: Identity Incident Response Playbook, Post-Incident Review and Root Cause Analysis Template, Security Metrics and KPIs Programme Guide, Incident Severity Classification Matrix Template.