Model weights, training datasets, notebooks and the AI API keys around them are stolen the old-fashioned way far more often than through clever model attacks. MITRE ATLAS tracks this as AML.T0025 Exfiltration via Cyber Means: adversaries move AI artifacts or other useful information out through traditional channels, and ATLAS points to the ATT&CK Exfiltration tactic (TA0010) for the detail. This guide maps the AI-specific assets to the ATT&CK channels, shows where AI infrastructure differs from ordinary servers (GPU clusters, model registries, notebooks, agent runtimes), and sets out detection, prevention and response for AI platform and SOC teams. Agent-driven exfiltration through tools is covered separately in the Agent Egress Control and Network Isolation Guide.
Key takeaways
- Same channels, different loot. AML.T0025 uses the ordinary ATT&CK exfiltration channels; what changes is the target: weights, datasets, embeddings, evaluation data and AI service credentials.
- ATLAS records it in real incidents. Case studies linked to AML.T0025 include the PyTorch torchtriton dependency attack, ShadowRay and the Hugging Face organisation confusion research.
- Size is a signal, but not always. A frontier model's weights are large and noisy to move; an API key, a LoRA adapter or a fine-tuning dataset is small and quiet.
- AI compute is often over-connected. GPU clusters and notebooks commonly have broad internet egress for package and model downloads, which is the same path data leaves by.
- The one ATLAS mitigation is access control. ATLAS links AML.M0005 Control Access to AI Models and Data at Rest to this technique; egress control, monitoring and short-lived credentials have to be added from general practice.
How the technique works
ATLAS defines AML.T0025 in two sentences: adversaries may exfiltrate AI artifacts or other information relevant to their goals via traditional cyber means, and the ATT&CK Exfiltration tactic gives more information. ATLAS rates it as realised. In practice it is the last step of an intrusion into AI infrastructure. ATLAS techniques such as AML.T0035 AI Artifact Collection and AML.T0036 Data from Information Repositories come first; AML.T0025 is how the collected material leaves.
| ATT&CK channel (v20) | How it applies to AI assets |
|---|---|
| T1041 Exfiltration Over C2 Channel | Implants on GPU nodes or notebook servers send collected keys and small artifacts back over their control channel |
| T1048 Exfiltration Over Alternative Protocol | DNS or other non-HTTPS protocols; the PyTorch torchtriton binary sent data out as encrypted DNS queries |
| T1567 Exfiltration Over Web Service | Uploads to attacker-owned model hubs, code repositories, cloud storage or webhooks, which blend with normal ML traffic to the same services |
| T1537 Transfer Data to Cloud Account | Copying weights or datasets from your bucket to an attacker-controlled account in the same cloud, often without crossing your network edge |
| T1020 Automated Exfiltration and T1030 Data Transfer Size Limits | Scripted collection from training jobs, and chunking large checkpoints to stay under alert thresholds |
The channel guides go deeper: see T1567 Exfiltration Over Web Service and T1048 Exfiltration Over Alternative Protocol.
What gets taken
| AI asset | Where it lives | Typical size and noise |
|---|---|---|
| Base or fine-tuned model weights | Model registry, object storage, serving nodes, training checkpoints | Gigabytes to terabytes; large transfers are visible if anyone watches |
| Adapters (for example LoRA) and small task models | Experiment trackers, notebook home directories | Megabytes; easy to move quietly |
| Training and evaluation datasets | Data lake, feature store, labelling platforms | Variable; often contain personal or licensed data |
| Embeddings and vector indexes | Vector databases used for retrieval | Moderate; can reveal the source documents |
| System prompts, agent configurations and tool definitions | Code repositories, agent platforms | Tiny; reveal guardrails and integrations |
| AI service and platform credentials | Environment variables, notebooks, CI secrets, cluster nodes | Bytes; the highest value per byte |
Weight protection levels and storage are covered in Securing AI Model Weights. This page focuses on the exfiltration step across all of these assets.
Where it shows up
- Exposed AI compute frameworks: ATLAS case study AML.CS0023 records ShadowRay: Ray's Jobs API allows remote execution without authentication, clusters exposed to the internet were exploited from September 2023, and adversaries could use the victims' compute and steal valuable information. See the ShadowRay 2024 breach page and the Ray and AI Compute Cluster Security Guide.
- Poisoned ML dependencies: ATLAS case study AML.CS0015 records the PyTorch-nightly compromise. PyTorch's advisory states that a malicious
torchtritonpackage on PyPI, installed between 25 and 30 December 2022, collected system information, environment variables,$HOME/.ssh/*and the first 1,000 files in the home directory, and uploaded them via encrypted DNS queries. See the PyTorch torchtriton Supply Chain Attack 2022. - Model hub organisation abuse: ATLAS case study AML.CS0027 describes a researcher who created look-alike organisations on Hugging Face; employees joined them, giving the researcher access to models they uploaded.
- Notebook and experiment servers: notebooks hold tokens in cells and environment variables and often run with broad egress. See the Jupyter Server Security Guide.
- Insiders: people with legitimate registry or bucket access can copy checkpoints to personal storage through the same ordinary channels, so the same controls and logs apply.
Detection
| Data source | What to look for | D3FEND |
|---|---|---|
| Object storage access logs (model and dataset buckets) | Bulk reads of checkpoint or dataset prefixes by an identity that normally only writes, or reads from a new network location | D3-UBA |
| Cloud control plane logs | Bucket policy changes that add an external principal, replication rules to an unknown account, snapshot sharing (T1537) | D3-OAM |
| Egress proxy and firewall logs from GPU, training and notebook subnets | Large or sustained uploads, new destinations, uploads to model hubs or storage under accounts that are not yours | D3-NTA |
| DNS logs | High-entropy subdomains or unusual query volume from build and training hosts (the torchtriton pattern) | D3-NTA |
| Model registry audit logs | Downloads of production or unreleased models outside deployment pipelines | D3-UBA |
| Endpoint and container runtime telemetry | Archive creation over checkpoint directories, reads of credential files and environment dumps by unexpected processes | D3-SFA |
Baseline first. Training jobs legitimately pull terabytes and push checkpoints, so absolute volume rules produce noise. Build baselines per identity and per destination: which workload identities write checkpoints, which pipelines download models, which external hosts each subnet reaches. Then alert on departures.
Prevention and hardening
- AML.M0005 Control Access to AI Models and Data at Rest: ATLAS asks for access controls on internal model registries, limited internal access to production models, and training data access only for approved users. ATLAS notes access controls can prevent exfiltration.
- Default-deny egress from AI compute: route package and model downloads through an internal mirror or proxy with an allowlist, and block direct internet egress from training and serving nodes. This removes most channels in the table above. See the Cloud Egress Filtering guide.
- Data perimeters: restrict model and dataset buckets so they can only be read from your own network and only shared with principals in your own organisation, which blocks T1537 copies to outside accounts.
- Keep control planes private: NHI Mgmt Group recommends not binding Ray to 0.0.0.0, blocking unauthorised access with firewall rules or security groups, and putting an authorisation proxy in front of the dashboard if it must be reachable.
- Encrypt and separate keys: AML.M0012 Encrypt Sensitive Information applies to weights at rest; keep the decryption keys under a different identity from the storage reader, so a copied bucket is not a usable model.
- Remove credentials from compute: replace static AI and cloud keys in environment variables with short-lived workload identity, so harvested environments yield little.
Implementation and decision guidance
Egress control on AI compute breaks jobs if it is switched on in one step. A phased rollout we recommend (NHI Mgmt Group):
- Observe (two to four weeks): log all egress from training, serving and notebook networks through a proxy in pass-through mode. Record destinations per workload identity.
- Build the allowlist: group destinations into package indexes, model hubs, your own storage and AI APIs. Move package and model downloads to an internal mirror where possible.
- Enforce on a pilot ring: switch one cluster or team to default-deny, with a documented exception route and a named owner who can approve new destinations quickly.
- Expand and harden: roll out to the remaining rings, add data perimeter conditions to model and dataset buckets, and turn on alerts for policy and replication changes.
- Roll back safely: keep the pass-through policy ready to reapply per ring, so a broken training run can be restored in minutes while the allowlist is fixed.
| Situation | Start with | Then add |
|---|---|---|
| Small team using hosted model APIs and a few notebooks | Remove keys from notebooks, scope hub tokens, turn on storage logging | Egress allowlist on the notebook environment |
| Organisation fine-tuning its own models on private data | Registry and bucket access control (AML.M0005), data perimeter on dataset buckets | Baselined storage analytics per identity |
| Organisation training or serving high-value proprietary models | Default-deny egress on all AI compute, encrypted weights with separate key custody | Insider controls and the weight security levels in Securing AI Model Weights |
Measuring it
- Egress coverage: percentage of AI compute subnets behind default-deny egress.
- Static key count: number of long-lived AI and cloud keys found on compute hosts and notebooks, trending to zero.
- External principals: number of model or dataset buckets readable by principals outside your organisation.
- Time to rotate: median time from a host being declared compromised to every readable credential being rotated.
- Alert precision: share of storage and egress alerts on AI subnets that analysts confirm as worth investigating.
Where identity fits
Almost every case study linked to AML.T0025 involves stolen or exposed machine credentials: Hugging Face and OpenAI tokens on Ray nodes, SSH keys in home directories, cloud keys in environment variables. The stolen credential is both the loot and the means of the next exfiltration. Our recommendations (NHI Mgmt Group):
- Inventory AI credentials: list every AI service key, model hub token and cloud identity used by training, serving and notebooks, with an owner.
- Scope hub tokens: give pipelines read-only, repository-scoped model hub tokens; keep write tokens for publishing pipelines only. See the Hugging Face Security Guide.
- Rotate on suspicion: treat any host where exfiltration is suspected as having lost every credential it could read, and rotate them all.
Response
- Identify the asset and the channel: which artifact, how much, to where, through which identity. Use storage and egress logs together.
- Contain the identity: disable the workload identity or user, revoke its sessions and keys, and remove any external principals or replication rules added to buckets.
- Contain the host: isolate affected nodes or notebook servers, preserving memory and disk where the platform allows.
- Rotate exposed credentials: every AI, cloud, database and SSH credential readable on the host, as PyTorch advised for affected nightly installs.
- Assess what the loss means: stolen weights may allow offline attack staging against your model; stolen datasets may be a personal data breach; stolen system prompts reveal guardrails to bypass.
- Close the path and record: fix the egress or access gap, and log the incident against AML.T0025 and the ATT&CK channel used. The Data Exfiltration Playbook covers the general response steps.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0025 (tactic AML.TA0010); preceding AML.T0035, AML.T0036 | Technique and collection steps |
| MITRE ATLAS mitigations | AML.M0005, AML.M0012 | Access control and encryption of AI assets |
| MITRE ATT&CK v20 | TA0010; T1041, T1048, T1567, T1537, T1020, T1030 | Channels used |
| NIST SP 800-53 Rev. 5 | AC-4, SC-7, AC-6, SI-4, AU-6 | Information flow, boundary protection, least privilege, monitoring |
| NIST CSF 2.0 | PR.DS-01, DE.CM-01, DE.CM-09, RS.MI-01 | Data at rest, network and runtime monitoring, containment |
| MITRE D3FEND | D3-OTF, D3-NTA, D3-UBA, D3-OAM | Outbound filtering and analytics |
Common failure patterns
- Open egress for convenience: GPU nodes allowed to reach the whole internet so jobs can pull packages and models, which also lets data leave anywhere.
- Volume rules on training subnets: thresholds tuned for office networks either fire all day or get disabled; baselines per identity work better.
- Ignoring small artifacts: watching only for terabyte transfers while API keys, adapters and system prompts leave in kilobytes.
- Control planes on the internet: compute framework dashboards exposed without authentication, as in ShadowRay.
- Personal hub accounts: staff publishing or syncing company models through personal model hub accounts or organisations nobody verified, which AML.CS0027 shows can be impersonated.
Practitioner checklist
- List every location that holds weights, adapters, datasets, embeddings and system prompts, with the identities that can read each.
- Enforce default-deny egress on training, serving and notebook networks, with downloads through an allowlisted mirror or proxy.
- Restrict model and dataset buckets to your own network and organisation, and alert on policy or replication changes.
- Turn on and retain object storage, model registry and DNS logs for AI subnets, and baseline them per identity.
- Keep compute framework control planes private and authenticated.
- Replace static AI and cloud keys on compute with short-lived workload identity and scoped hub tokens.
- Write the AI exfiltration response steps into the incident plan, including full credential rotation for affected hosts.
- Tag incidents with AML.T0025 and the ATT&CK channel so trends are visible.
Standards and references
- MITRE ATLAS: Exfiltration via Cyber Means
- MITRE ATLAS: Control Access to AI Models and Data at Rest
- MITRE ATT&CK: Exfiltration
- MITRE ATLAS: ShadowRay
- PyTorch: Compromised PyTorch-nightly dependency chain
- NIST: SP 800-53 Rev. 5
Related NHI Mgmt Group resources: Securing AI Model Weights · Agent Egress Control and Network Isolation Guide · Data Exfiltration Guide · MITRE ATLAS Guide