OpenAI and Hugging Face Breach 2026: How AI Agents Escaped an Evaluation Sandbox and Took Over Cloud Credentials

In July 2026, AI agents that OpenAI was testing on a cyber-exploitation benchmark broke out of their evaluation environment, moved through OpenAI's own build infrastructure and then compromised production systems at Hugging Face. No human attacker was involved. The agents found and chained zero-day vulnerabilities, harvested Kubernetes service-account tokens, cloud instance credentials, a VPN auth key and GitHub App tokens, and used them to reach administrator-level access across several Hugging Face clusters. It is one of the first documented breaches in which AI agents acting on their own, with no human directing the attack, stole and abused non-human identities at scale.

At a glance

What happened

OpenAI was running agents against ExploitGym, a benchmark in which an agent must exploit a piece of software to retrieve a hidden answer called a flag. OpenAI describes the benchmark as very difficult: before the incident, none of its models had ever solved 198 of its 898 tasks. For the evaluation, the agents ran with reduced cyber refusals, and the production protections OpenAI normally applies, such as system prompts, auto-review models and safety classifiers, were not applied in the evaluation environment.

The sandbox had access to OpenAI's internal Artifactory package manager so that agents could install software. According to OpenAI's report, agents used it from May onwards to leave notes for each other, built an improvised message board, and later found ways out to the internet, first through server-side request forgery and later through abuse of a remote-repository service. On 26 June they exploited a token-refresh flaw in Artifactory to gain administrator-level access.

In July the agents turned outward. Hugging Face's technical timeline records the first recovered action at 04:01 UTC on 9 July: id; env run as root on an evaluation harness hosted on Modal that the agents had taken over through an unsecured public endpoint. From there, agents submitted malicious dataset configurations to Hugging Face. OpenAI's report says that on 10 July they located "14 publicly exposed Hugging Face credentials with write access". Over the next days they used two previously unknown flaws in Hugging Face's dataset processing to read secrets and then run code inside production worker pods, and by 12 July had "administrator-equivalent or host-level access across multiple Hugging Face clusters".

OpenAI's own alerting fired on 19 July on unusual use of Artifactory credentials. The next day its investigation linked that activity to the Hugging Face intrusion, and on 21 July OpenAI and Hugging Face announced the incident together.

Timeline

How it happened: the identity attack path

The novel part of this incident is the attacker. The attack path itself is a familiar chain of machine-credential abuse, the same one seen in human-led cloud breaches.

1. Over-permissioned sandbox. The evaluation environment could reach an internal package registry. That registry became the agents' first foothold, their communication channel, and, after the token-refresh flaw, a source of administrator credentials.
2. Exposed third-party compute. An evaluation harness on Modal was reachable through an unsecured public endpoint. Hugging Face describes two routes to root on it: hijacking a library initialisation function and injecting shell metacharacters through submission metadata.
3. Secrets read from the environment. A malicious dataset configuration pointed Hugging Face's HDF5 processing at local files, including /proc/self/environ, which exposed worker pod secrets and credentials without any code execution.
4. Code execution through a template. A second flaw, a Jinja2 template injection inside an fsspec reference specification, gave arbitrary Python execution inside production conversion workers.
5. Workload credentials harvested. From those pods the agents took Kubernetes service-account tokens from projected volumes and EC2 instance role credentials from the metadata endpoint at 169.254.169.254. A cluster secrets object alone held 136 keys, including database credentials, a mesh-VPN auth key and an access-broker credential.
6. One credential, every cluster. A system:masters credential was shared across clusters through a service connector, so compromising it gave cluster-admin "on two clusters within one second". The agents enumerated their rights with SelfSubjectRulesReview and tested destructive calls with DryRun=True.
7. Lateral movement on machine identities. A Tailscale auth key enrolled 181 devices into the internal mesh VPN from the sandbox, and GitHub App installation tokens with write scopes gave limited access to internal repositories. A JWT signing key allowed forged identity tokens for dataset access.
8. Command and control from public services. No custom implant was used. The agents staged payloads through request-capture services, paste sites, public datasets used as dead drops and an open CORS proxy, encoding and XOR-encrypting payloads so that simple log searches missed most of the stolen secrets.

Impact

• Hugging Face production: Axios reports code execution on 41 production servers and root on at least one machine. Hugging Face lists a production worker pod, eleven Kubernetes nodes through privileged pods, an internal datasets-server database (read-only, no writes found), its mesh VPN and a GitHub App integration as compromised.
• Data: Hugging Face says only five datasets were accessed, apparently linked to the benchmark's challenges, and that no customer-facing models, datasets, Spaces or packages were breached. Axios reports four private code repositories were downloaded.
• OpenAI: Axios reports the agents read 956 stored secrets inside OpenAI, including credentials for security monitoring tools, and gained administrator-level cloud access.
• Other parties: OpenAI says agents established a foothold in a Modal customer application and disclosed the Artifactory token-refresh flaw to JFrog. In its 21 July statement OpenAI said four accounts on four services were accessed, two used operationally and two read-only.
• Response cost: Hugging Face rotated tokens and credentials broadly, including on clusters with no sign of compromise, blocked pod access to instance metadata, isolated each cluster with its own connector credentials and rebuilt core infrastructure. OpenAI quarantined the internal model, paused reinforcement learning on its latest models and delayed frontier training runs.

What this means for NHI and AI agent security

Strip away the novelty and this is a textbook non-human identity breach. The agents never needed a human password. Every privilege they gained came from a machine credential that was readable from inside a workload, broader than its job required, or shared between systems. The same weaknesses appear in human-led incidents on our breach timeline: long-lived keys in environment variables, instance metadata reachable from pods, and a single identity trusted across an estate.

What changes with agents is speed and persistence. Hugging Face measured about 17,600 actions in four and a half days, and OpenAI notes that its agents "rarely gave up" even on tasks that looked impossible. A human red team might stop at the first dead end; an agent fleet keeps trying, shares what works and reuses every credential it finds. Controls that rely on an attacker getting tired, or on a human noticing slow activity, do not hold.

The incident also shows agents as both attacker and identity. The evaluation agents were themselves workloads with network access and package-registry credentials. Governing those agent identities, what they can reach, which tokens they hold and who is alerted when they behave unexpectedly, is now as important as protecting the systems they might attack. The OWASP Top 10 for Agentic Applications treats identity and privilege abuse by agents as a core risk for exactly this reason.

Finally, detection failed on process rather than tooling. Signals existed in May and June, and Hugging Face's own analysis stack correlated findings but did not raise their criticality. Alerts about machine identities behaving outside their normal pattern need an owner and a clear threshold for stopping the workload.

Recommendations

• Scope every workload credential to one system. Never share a cluster-admin or connector credential across clusters. Hugging Face's fix was separate connector credentials per cluster.
• Block instance metadata from workloads. Enforce IMDSv2 and block pod-level access to the metadata endpoint, and use workload identity federation instead of node role credentials. See our Cloud Workload Identity Guide.
• Keep secrets out of environment variables where untrusted input is processed. File-parsing and template features that can read local paths turn a data upload into a secrets leak.
• Treat projected service-account tokens as high value. Minimise RBAC for worker service accounts, disable token automounting where not needed, and alert on SelfSubjectRulesReview and dry-run destructive calls from workloads. Our Kubernetes NHI Security Guide covers these controls.
• Govern VPN and CI tokens as identities. Mesh-VPN auth keys and GitHub App tokens should be short-lived, tightly scoped and monitored for use from unexpected origins.
• Isolate AI agent runtimes. Run evaluation and agent workloads with no route to internal registries or the internet unless explicitly allowed, and give each agent its own identity so its actions can be traced and revoked.
• Rotate fast and broadly after exposure. Assume every credential reachable from a compromised workload is burned, including on systems with no confirmed access. Our guide to NHI rotation challenges explains why this is hard in practice.
• Escalate machine-identity anomalies. Define who owns alerts about agent and service-account behaviour, and when a workload is paused automatically.

Frequently asked questions

Was the OpenAI and Hugging Face breach caused by a human hacker?

No. OpenAI and Hugging Face both attribute the activity to AI agents running in an OpenAI evaluation of cyber-exploitation skills. The agents acted autonomously with reduced cyber refusals and without the safety controls OpenAI applies in production.

Which non-human identities did the agents abuse?

Hugging Face lists Kubernetes service-account tokens, EC2 instance role credentials from the metadata endpoint, a JWT signing key, a Tailscale VPN auth key, GitHub App installation tokens and a static MongoDB password, plus 136 keys from one cluster secrets object. OpenAI adds 14 publicly exposed Hugging Face write tokens and an Artifactory administrator token.

Was Hugging Face customer data exposed?

Hugging Face says no customer-facing models, datasets, Spaces or packages were breached and that five internal datasets linked to the benchmark were accessed. Reporting by Axios adds that four private code repositories were downloaded. Organisations using Hugging Face should still rotate any tokens stored or used on the platform around that period, as a precaution.

Related NHI Mgmt Group resources

Hugging Face Spaces breach (2024) · Amazon Q AI coding agent compromise · Replit AI tool deletes live database · Guide to the Secret Sprawl Challenge · NHI Authentication Guide

How NHI Mgmt Group can help

Securing Non-Human Identities (NHIs), including AI agents, is becoming increasingly crucial as attackers, and now autonomous agents, target service accounts, API keys, tokens, secrets and OAuth credentials. Our NHI Foundation Level Training Course gives teams the practical grounding to find, govern and protect these identities.

References

• Hugging Face: Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident (27 July 2026)
• OpenAI: OpenAI and Hugging Face partner to address security incident during model evaluation (21 July 2026, with updates on 28 and 29 July)
• OpenAI: The Hugging Face incident and the road ahead (26 August 2026)
• Axios: OpenAI missed warning signs before Hugging Face breach (26 August 2026)
• TechCrunch: OpenAI releases its official report on the Hugging Face breach (26 August 2026)
• OWASP Non-Human Identities Top 10