Leaked API keys can fund model distillation and charge victims

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Truffle Security's blog post: API keys can be used to fund model distillation attacks, and OpenCode's breach shows how stolen credentials can turn model access into victim-billed training data.

Why it matters: Identity and security teams need to treat API keys as spend-bearing non-human credentials, because compromise can create both access loss and direct financial exposure.

By the numbers:

• OpenCode disclosed a breach affecting 1.4% of its Console users.

👉 Read Truffle Security's analysis of model distillation abuse and API key risk

Context

Model distillation is the practice of training one model on another model’s responses, which means the attacker needs access to the teacher model and enough budget to collect examples. In identity terms, that makes an API key more than an authentication token. It becomes a spend-bearing credential that can finance abuse if it is stolen, shared, or retained in the wrong service boundary.

The security gap is not only theft. It is the assumption that model access stays inside a controlled requester and only serves legitimate usage patterns. When a key is forwarded through a coding service, resold, or embedded in a transcript, the trust boundary expands beyond the organisation that originally owns the credential.

This is why secret governance for AI services needs the same lifecycle discipline as any other API key management problem, but with added scrutiny on usage cost, downstream retention, and delegated access paths.

Key questions

Q: What breaks when an API key is stolen in a model platform?

A: The key stops being a simple authenticator and becomes a funded abuse path. An attacker can use it to generate responses, collect training examples, burn quota, and create charges on the victim’s account. The control failure is that access and billing were treated as separate concerns when they were actually the same exposure surface.

Q: Why do leaked model API keys create financial risk as well as access risk?

A: Because each request can consume paid capacity under the victim’s account. Even if the attacker never changes the model or the service configuration, they can spend the organisation’s allowance while building their own dataset. The outcome is a direct transfer of cost, not just a loss of confidentiality.

Q: How do organisations know if their AI governance is too weak?

A: Weak AI governance shows up when legitimate accounts can create harmful content, export it repeatedly, and feed it into other systems without raising scrutiny. If abuse can move from prompt to campaign with little friction, the organisation is treating AI as a tool feature rather than as an identity and trust boundary.

Q: What should organisations do when intermediaries store or forward API keys?

A: Treat the intermediary as part of the credential boundary and require lifecycle controls over storage, logs, backups, and offboarding. If the service can replay the secret, it can also prolong exposure. That means procurement, engineering, and security all share responsibility for where the key can persist.

Technical breakdown

Why stolen API keys are monetisable in model abuse

A model API key authorises requests against an account, and that account usually carries billing, rate limits, and usage entitlements. When an attacker steals the key, they do not need to break the model provider itself. They can simply send prompts, store the responses, and push the victim’s account toward its own limits or billing ceiling. In abuse cases, the key is not just an access secret. It is a prepaid path to compute, output, and data collection. That makes the key valuable even when the attacker never gains control of the model or its infrastructure.

Practical implication: Treat spend-bearing API keys as high-value secrets with explicit revocation and usage monitoring.

Why billing limits do not equal governance limits

A published monthly limit is not the same thing as a security boundary. Some providers apply alerts while requests continue, and some usage is delayed in billing systems, so the dashboard is not a reliable real-time cutoff. From an identity perspective, this means the effective blast radius of a compromised key can outlast the team’s assumption about when abuse will stop. The control problem is not only whether a limit exists. It is whether abuse can continue after the organisation believes the account has been contained. Financial guardrails and identity guardrails are related, but they are not interchangeable.

Practical implication: Instrument key usage as an identity risk signal, not just a finance alert.

How intermediary services expand the credential boundary

When a platform stores a customer’s key and forwards requests on its behalf, that platform becomes part of the security boundary for the credential. Its storage, administration, backups, and retired systems can all expose the key, even if the customer never commits it to source control. The same issue appears when users paste live secrets into prompts or configuration snippets that an intermediary retains. In both cases, the original owner loses direct control over where the secret travels, how long it persists, and who can later replay it. That is a lifecycle problem, not just a leakage problem. Read Guide to the Secret Sprawl Challenge.

Practical implication: Map every service that stores, forwards, or retains API keys into the credential lifecycle.

Threat narrative

Attacker objective: The attacker wants to collect model responses at scale while shifting the compute cost and account burden onto the victim.

1. Entry occurs when a valid model API key is stolen, copied from an intermediary, or otherwise obtained from the victim’s credential estate.
2. Credential access follows as the attacker uses the key to send legitimate model requests and collect responses under the victim’s account.
3. Impact appears when the attacker accumulates training examples while the victim absorbs the charges, consumes quota, or loses access to the shared allowance.

Breaches seen in the wild

• Microsoft Azure OpenAI abuse by Storm-2139: Storm-2139 used API keys leaked by Microsoft customers to hijack Azure OpenAI, bypass safety guardrails and resell access to generate harmful content.
• 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

API keys have become spend-bearing identities, not just access tokens: In model ecosystems, a stolen key can finance the attack itself because each request converts into billable output and training data. That changes how identity teams should think about blast radius. The risk is no longer limited to unauthorized access. It includes a direct transfer of compute cost and account capacity to an attacker, which means key governance is now a financial-control issue as well as an authentication issue.

Long-lived API key governance fails when abuse is economically self-funding: Traditional secret management assumes compromise is an access event that can be contained before the next cycle of review. In model distillation abuse, the attacker can keep requesting outputs until the provider enforces a limit or the account is revoked. That makes the real failure mode sustained, monetised misuse of a legitimate credential. Organisations should read that as a governance gap in how they classify and monitor high-value machine identities.

Delegated model access widens the accountability gap around third-party retention: When a service stores or forwards customer keys, it inherits part of the trust boundary even if the customer still owns the bill. That creates a shared-control problem across procurement, security, and engineering. The practical implication is that offboarding, backups, and retained transcripts become credential exposure surfaces, not just operational details. NHI governance has to extend to every intermediary that can replay a secret or preserve it beyond its intended use.

Model distillation abuse is a secret-life-cycle problem with an AI-specific outcome: The same exposure pattern that leaks ordinary API keys now also enables training-data harvesting and unauthorized model monetisation. That creates a distinct category of credential abuse where the asset is not only the secret itself, but the ongoing ability to produce usable responses under someone else’s account. Practitioners need to treat that as a lifecycle and billing boundary failure, not as a narrow application incident.

Credential verification without continuous revocation is too slow for AI service abuse: Truffle Security’s example shows that exposed keys can remain valuable long enough to generate both responses and charges before teams detect them. That is a strong signal that inventory alone is insufficient. The control gap is speed between discovery and invalidation, especially where third-party services, shared gateways, or billing delays extend exposure. In practice, the organisation needs to know which keys can still spend before an attacker does.

From our research library:

• DeepSeek alone generated 113,000 new exposed API keys in 2025, illustrating how new AI providers create credential exposure before security guardrails catch up, according to the State of Secrets Sprawl 2026.
• Only 20% have formal processes for offboarding and revoking API keys, and even fewer have procedures for rotating them, according to the Ultimate Guide to NHIs.
• Read next: API Key Management Guide

What this signals

Secret governance now has a billing dimension: When an API key can generate paid model output, revocation speed becomes part of financial resilience. Teams should expect AI platforms to be judged not only on authentication strength but on how quickly leaked credentials are rendered useless in practice.

Identity programmes that still treat API keys as low-friction developer secrets will miss the real exposure pattern. The more an AI service can be reached through intermediaries, the more likely the credential boundary has already expanded beyond the team that thinks it owns it.

For practitioners

• Inventory spend-bearing API keys Classify every model-facing key by owner, billing impact, provider, and revocation path so teams can prioritise the secrets that can generate direct cost if abused.
• Revoke keys through provider controls first Use the provider-side revocation path as the containment step, then validate that downstream gateways, proxies, and mirrors can no longer replay the credential.
• Audit intermediary retention of secrets Review coding assistants, gateways, ticketing tools, and prompt logs for any place a live credential can be stored, replayed, or copied into training workflows.
• Separate model access from shared billing exposure Where possible, avoid letting one reused key fund broad organisational model consumption. Use tighter scoping, separate accounts, or delegated credentials with narrower blast radius.
• Measure how fast leaked keys become inert Track the elapsed time between exposure detection, revocation, and confirmation that the key can no longer spend against the account.

Key takeaways

• A stolen model API key can function as both an access credential and a cost-bearing resource, which makes compromise financially as well as operationally damaging.
• The OpenCode breach illustrates how response harvesting and billing exposure can occur together when model credentials are mismanaged.
• Identity teams need faster revocation, clearer ownership, and tighter intermediary controls if they want secret governance to keep pace with AI service abuse.

Related NHI Mgmt Group resources: API Key Management Guide · Guide to the Secret Sprawl Challenge · NHI Authentication Guide · Ultimate Guide to NHIs

Key terms

• Model Distillation: Model distillation is a technique for transferring knowledge from a larger model into a smaller one that is easier and cheaper to run. It helps reduce training and inference costs, especially when organisations want to preserve useful accuracy while cutting computational demand and emissions.
• Spend-Bearing API Key: An API key that can generate direct financial consumption when used, because each request draws from a paid account, quota, or billing arrangement. For identity governance, this means the credential carries both access authority and cost exposure, so revocation timing matters.
• Credential Retention Boundary: The point at which a service stops being a temporary handler of a secret and becomes a place where the secret can persist, be replayed, or be reused. For AI workflows, this boundary often includes logs, backups, prompt stores, and forwarding gateways.

What's in the full article

Truffle Security's full blog post covers the operational detail this post intentionally leaves for the source:

• Provider-by-provider discussion of published monthly spend limits and how they affect abuse potential
• Explanation of why alerts, hard limits, and billing delay do not stop all malicious requests at once
• Examples of how coding assistants and retained transcripts can carry live secrets into downstream workflows
• Context on TruffleHog detection and verification workflow for exposed credentials

👉 The full Truffle Security post covers the distillation mechanics, spending examples, and credential retention paths in more detail.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.