In most breaches involving machine credentials, the organisation that owned the credential was not the one that noticed it was being misused. Across the cases on the NHI Mgmt Group breach database, the first alarm came from a customer, a platform provider, a threat intelligence team, a researcher, a journalist or the attacker, and the gap between exposure and discovery ran from two days to five years. This guide reads ten well-documented incidents for one thing only: how long the credential went unnoticed, and who found it. It then turns the pattern into detection controls that put the owner first in that list. It is written for detection engineers, identity and cloud security teams and incident responders. For how the credentials leaked in the first place, see the Leaked Secret Breaches Lessons Guide and Stolen credential breaches; this page covers the detection side.
Key takeaways
- Someone else usually rang the bell. By NHI Mgmt Group's own count of the ten cases below, in seven the first signal came from outside the credential owner's own monitoring (in one more, the Internet Archive, the source does not say who found it).
- Exposure windows are long, misuse windows are short. Credentials sat exposed for months or years, but once used, data left within days, so detection has to work on first use.
- Machine credentials are often watched only at the receiving end. GitHub caught misuse of an AWS key against its own npm infrastructure; the downstream platform saw what the token owner did not.
- Rotation is part of detection. A credential that is rotated after exposure stops the second breach, as the Internet Archive case shows.
- Make outside reports easy to act on. A researcher's report in the Home Depot case went unanswered until a journalist called, so the report route itself is a detection control.
The cases: time to discovery and who found it
Dates below are taken from the linked breach pages, which cite the organisations' own statements and official sources. Where a page gives a month rather than a day, so does the table.
| Case | Credential | Exposure or compromise began | Discovered | Who found it |
|---|---|---|---|---|
| tj-actions/changed-files Compromise 2025 | Bot account personal access token | 12 March 2025 (exposure window start) | 14 March 2025 | Third-party runtime monitoring of network egress |
| GitHub OAuth Token Breach 2022 | Integrator OAuth tokens, then an AWS key | April 2022 | 12 April 2022 | The downstream platform, from misuse of the AWS key |
| CircleCI Breach 2023 | Stolen SSO session, then customer secrets | 16 December 2022 | 29 December 2022 | A customer reporting suspicious GitHub OAuth activity |
| Salesloft Drift Breach 2025 | Integration OAuth tokens | GitHub access from March 2025; data theft from 8 August 2025 | 20 August 2025 | The integrator, after customer tenants had been queried |
| Snowflake Breach 2024 | Infostealer-harvested passwords on accounts without MFA (the source does not say how many were service accounts) | Activity from mid-April 2024; credentials stolen as early as November 2020 | 22 May 2024 (platform notified) | External threat intelligence, then the platform and victims |
| Codecov Breach 2021 | Cloud storage key | 31 January 2021 | 1 April 2021 | A customer who compared the script's checksum |
| Uber Breach 2016 | AWS access key in a code repository | 13 October 2016 | 14 November 2016 (first documented signal) | The attackers, by extortion email |
| Internet Archive Breach 2024 | GitLab token, then an unrotated support platform token | Token exposed since at least December 2022 | Reported 9 October 2024 | Not stated in the source; the attacker described the entry point to the press |
| Home Depot Token Exposure 2025 | GitHub access token | Early 2024 | Early November 2025 | A security researcher; action followed press contact |
| Toyota T-Connect Key Exposure 2022 | Data server access key in public code | December 2017 | 15 September 2022 | The owner, about five years later |
Four detection gaps that repeat
1. Nobody watched how the machine credential was used
Service accounts, bot tokens and integration grants rarely have a behavioural baseline. In the Salesloft Drift case the stolen tokens were used against customer Salesforce tenants from 8 August, and the integrator announced detection on 20 August. In the Snowflake campaign, accounts protected by a password alone were used with credentials that had been stolen years earlier. A baseline of normal source addresses, query volumes and API calls per credential would make bulk exports look different from routine syncs.
2. Signals landed with someone else
Where the owner did not see misuse, the party at the other end often did. GitHub detected an AWS key being misused against its npm production systems on 12 April 2022 and traced it back to stolen integrator OAuth tokens. A CircleCI customer's report of suspicious GitHub OAuth activity started that investigation. The receiving platform, the customer and the CI runtime saw the anomaly first.
3. Exposure was never treated as an incident
The Toyota key sat in a public repository for about five years, and the Home Depot token for more than a year. Neither case required an intrusion to begin: the exposure itself was the event, but no scanner or report route turned it into a ticket. When the Home Depot researcher reported the token, the emails went unanswered until a journalist contacted the company on 5 December 2025.
4. Response stopped before every credential was rotated
The Internet Archive was breached a second time on 20 October 2024 through a support platform token that had not been rotated after the first breach. Detection without complete rotation leaves the attacker a way back. See the Leaked Credential Response Playbook.
Who finds NHI breaches, and how to be first
| Discovery source | Cases above | What the owner can do to get there first |
|---|---|---|
| Owner's own monitoring | Salesloft Drift (after data theft), Toyota (after years) | Baseline each machine credential's normal use; alert on new source, new API, volume spikes |
| Downstream platform | GitHub 2022, Snowflake | Subscribe to provider security notifications and route them to the SOC, not a shared mailbox |
| Customer | Codecov, CircleCI | Publish checksums and signatures; give customers a fast reporting route and act on reports the same day |
| Third-party monitoring or researcher | tj-actions, Home Depot | Run a vulnerability disclosure route with an owner and a response target; monitor CI runner egress yourself |
| Attacker or press | Uber (the attacker also briefed the press in the Internet Archive case) | Treat this as the worst case: if it happens, assume every credential reachable from the first one is compromised |
GitHub's secret scanning partner programme shows how provider-side detection works: when a supported secret format appears in a public repository, GitHub sends it to the issuing service provider, which can revoke it and notify the customer, and GitHub recommends treating any reported secret as public and compromised. That only helps if the provider's notification reaches someone who acts.
Controls that shorten the gap
NHI Mgmt Group recommends the following controls, ordered by how early in the timeline they fire.
- Inventory with owners: you cannot baseline a credential you do not know about. See NHI Discovery Across Cloud, SaaS, CI/CD and On-Premises Environments.
- Exposure detection: secret scanning on code, CI logs, container images and paste sites, with every true positive opened as an incident, not a hygiene ticket.
- Decoys: place canary credentials where real ones tend to leak (repositories, CI variables, laptops); any use is a high-confidence alert. This maps to MITRE D3FEND Decoy User Credential (D3-DUC) and Decoy Session Token (D3-DST). See the Honeytokens and Canary Credentials Guide.
- Usage baselines: per credential, record expected source networks, APIs and volumes; alert on deviations, especially bulk reads and exports.
- Egress monitoring for build systems: CI runners should talk to a known set of endpoints; the tj-actions case was found that way.
- Short lifetimes: credentials that expire in hours make old exposures useless, which narrows the window in which an old exposure is still usable, as the Snowflake and Toyota timelines show.
- Complete rotation after any incident: rotate every credential reachable from the compromised system, including support, ticketing and integration tokens (D3FEND Credential Revocation, D3-CR).
Decision guidance
- Small teams: start with provider notifications routed to an on-call inbox, push protection on code hosting, and a handful of canary credentials. These need little tuning.
- Larger estates: add per-credential usage baselines in the SIEM or an identity threat detection tool, starting with the 50 most privileged machine identities. See the ITDR Guide.
- SaaS-heavy organisations: prioritise monitoring of third-party OAuth grants and API exports from CRM, support and code platforms, since several cases above moved through integrations.
- Software publishers: publish checksums and signatures and make it easy for customers to report mismatches, because your customers may detect a compromise before you do.
Detection and measurement
| Signal | Where to look | ATT&CK |
|---|---|---|
| Machine credential used from a new network or country | Cloud audit logs, identity provider logs | T1078.004 |
| Integration token used for bulk queries or exports | SaaS audit and API logs | T1550.001 |
| Token theft from an application or integrator | OAuth grant and token logs | T1528 |
| Secrets found in files or code | Secret scanning alerts | T1552.001, T1552.004 |
| Build step contacting an unknown endpoint | CI runner egress logs | T1195.002 |
| Canary credential used | Decoy alerting | T1078 |
Track these metrics: share of NHI incidents first detected internally (target rising); median days from exposure to detection; median hours from external report to revocation; percentage of privileged machine credentials with a usage baseline; percentage of credentials rotated after each incident; number of canary credentials deployed and tested in the last quarter. For broader figures see Incident Response Statistics.
Framework mapping
| Gap | NIST CSF 2.0 | NIST SP 800-53 Rev 5 | OWASP NHI Top 10 2025 |
|---|---|---|---|
| Unknown or unowned credentials | PR.AA-01 | IA-4, CM-8 | NHI1 Improper Offboarding |
| Exposure not detected | ID.RA-01 | RA-5 | NHI2 Secret Leakage |
| Use not monitored | DE.CM-03, DE.CM-09 | AU-2, AU-6, AU-12, SI-4 | NHI5 Overprivileged NHI |
| External signals not acted on | DE.AE-07 | IR-6 | NHI3 Vulnerable Third-Party NHI |
| Long-lived credentials extend windows | PR.AA-05 | IA-5 | NHI7 Long-Lived Secrets |
Common failure patterns
- Provider alerts to a dead mailbox: notifications from cloud and code platforms go to an address nobody watches.
- Exposure closed as a hygiene task: the secret is deleted from the file but not revoked, and nobody checks the logs for use.
- Machine identities excluded from analytics: behavioural detection covers people but not service accounts and integration tokens.
- No owner for outside reports: researcher or customer reports wait for days, as in the Home Depot case.
- Partial rotation: the first credential is revoked but tokens reachable from it are not, which led to the second Internet Archive breach.
Practitioner checklist
- List where provider security notifications go and route them to the SOC.
- Treat every confirmed secret exposure as an incident with log review for use.
- Build usage baselines for your most privileged machine credentials.
- Deploy and test canary credentials in code, CI and developer laptops.
- Monitor CI runner egress against an allow list.
- Publish a disclosure route with a named owner and a response target.
- After any incident, rotate every credential reachable from the compromised system.
- Report the internal detection share and exposure-to-detection time each quarter.
Standards and references
- OWASP: Non-Human Identities Top 10 2025
- GitHub Docs: Secret scanning partner program
- NIST: SP 800-61 Rev. 3 Incident Response Recommendations
- MITRE D3FEND: Decoy User Credential
- MITRE ATT&CK: Valid Accounts, Cloud Accounts (T1078.004)
- MITRE ATT&CK: Use Alternate Authentication Material, Application Access Token (T1550.001)
Related NHI Mgmt Group resources: OAuth and SaaS integration breaches · Supply chain NHI breaches · How to Investigate CloudTrail for a Compromised Access Key · OWASP NHI Top 10 NHI2 Secret Leakage Guide