Once an adversary holds a copy of your model's architecture and weights, they can build and test attacks offline where you cannot see them, then use them against your live system with the first query. MITRE ATLAS tracks this as AML.T0044 Full AI Model Access, in the AI Model Access tactic (AML.TA0000), and rates its maturity as demonstrated. ATLAS defines it as full "white-box" access: complete knowledge of the model architecture, its parameters and its class ontology. This guide covers how models end up in white-box hands, the two ATLAS mitigations, how to protect model files in registries, containers, repositories and edge apps, how to design for models you must assume are exposed, and how to detect access. It is for ML platform, application and security teams, and goes deeper than the MITRE ATLAS Guide.
Key takeaways
- White-box access moves the attack offline. ATLAS notes adversaries may exfiltrate the model to craft adversarial data (AML.T0043) and verify attacks (AML.T0042) where their behaviour is hard to detect.
- Shipping a model is distributing it. Two of the four ATLAS case studies recovered models from mobile apps, including encrypted ones decrypted at runtime (AML.CS0058).
- Write access is worse than read access. In AML.CS0027 and AML.CS0028 researchers gained the ability to replace models, turning exposure into a supply chain risk.
- ATLAS maps two mitigations. AML.M0005 Control Access to AI Models and Data at Rest and AML.M0017 AI Model Distribution Methods.
- Some models are white box by design. For open-weight models and on-device models, plan as if the adversary has the weights, and put the defences around the model.
How the technique works
AML.T0044 was created in the first ATLAS release (May 2021) and last modified in May 2026; it applies to predictive, generative and agentic AI and has no sub-techniques. It is an access state rather than a single action: the adversary ends up holding the model files or equivalent knowledge. ATLAS contrasts it with AML.T0040 AI Model Inference API Access, where the adversary can only query the model.
| Access level | What the adversary has | ATLAS technique | Typical consequence |
|---|---|---|---|
| Black box | Query and response only | AML.T0040 | Attacks developed through many visible queries |
| Grey box | Partial knowledge, such as preprocessing methods or model family | ATLAS M0017 refers to grey-box attacks where preprocessing is known | More efficient query-based attacks |
| White box | Architecture, parameters and class ontology | AML.T0044 | Gradient-based adversarial inputs and verification offline; model tampering if write access is also held |
The access is gained through other techniques: finding artefacts (AML.T0007 Discover AI Artifacts), collecting them (MITRE ATLAS AML.T0035 AI Artifact Collection), using valid accounts on a model repository, or downloading a model the victim has published. What follows is usually offline crafting, and where write access is held, model tampering such as MITRE ATLAS AML.T0076 Corrupt AI Model.
Real cases in ATLAS
All four case studies ATLAS links to AML.T0044 are research exercises; they show the routes rather than a named attacker.
| ATLAS case study | Route to full access | Lesson for defenders |
|---|---|---|
| AML.CS0013 Backdoor Attack on Deep Learning Models in Mobile Apps (Microsoft Research, 2021) | Models extracted from Android apps on Google Play, in compiled binary form; 54 apps found vulnerable to backdoor injection, including face authentication and financial apps | A model inside an app package is available to anyone who installs it |
| AML.CS0027 Organization Confusion on Hugging Face (2023) | A researcher created Hugging Face organisations impersonating real companies; employees asked to join and uploaded private models, giving the researcher read and write access | Verify the organisation before uploading; control who may publish models externally |
| AML.CS0028 AI Model Tampering via Supply Chain Attack (Trend Micro, 2023) | Over 8,000 misconfigured private container registries exposed online, about 70 percent with write access; over 1,000 unique AI models in images that could be pulled without authentication | Registries and container images holding models need the same protection as the models |
| AML.CS0058 Google Photos AI Model Extraction (Skyld, 2025) | TensorFlow Lite models recovered from the Android app: some unencrypted in assets, most embedded in a native library, and encrypted models captured in memory after runtime decryption | Encryption at rest does not protect a model that must be decrypted on the device to run |
Skyld reported the Google Photos findings to Google in December 2024 and published in March 2025; the researchers recovered 20 models covering tasks such as face detection, segmentation and depth estimation.
Where models leak
Map every place a full copy of a model exists. Each is a route to AML.T0044.
| Location | Exposure | Primary control |
|---|---|---|
| Training and experiment storage | Checkpoints in object storage, notebooks and shared drives | Private buckets, scoped service accounts, retention for old checkpoints |
| Model registry | Broad read rights for data science groups and pipelines | Role-based access per model and stage (AML.M0005) |
| Container images and registries | Models baked into images; registries open to the internet | Authenticated pulls, no anonymous access, image signing |
| Public model hubs | Uploads to look-alike or wrong organisations; private repositories made public | Approved organisations list, publishing workflow, token scopes |
| Mobile, desktop and edge apps | Model files in the package or decrypted in memory | Serve sensitive models from the cloud (AML.M0017); accept exposure for on-device models |
| Inference servers | Weights on disk of serving hosts; model download endpoints | Host hardening, no model file download routes, restricted shell access |
| Third parties | Models shared with vendors, partners or fine-tuning services | Contracts, scoped sharing, deletion confirmation |
Prevention: the ATLAS mitigations
| ATLAS mitigation | ATLAS guidance | Practical application |
|---|---|---|
| AML.M0005 Control Access to AI Models and Data at Rest | Establish access controls on internal model registries, limit internal access to production models and limit training data access to approved users | Per-model permissions, separate read and write roles, short-lived pipeline credentials, quarterly access review |
| AML.M0017 AI Model Distribution Methods | Deploying models to edge devices increases attack surface; consider serving models in the cloud, and computing features in the cloud to prevent grey-box attacks | Keep high-value or security-relevant models server-side; ship only models whose exposure you accept |
Additional controls
- Integrity, not just confidentiality: sign model artefacts and verify signatures at load time, so write access to a registry or container cannot silently replace a model. See Securing AI Model Weights.
- Runtime protection for on-device models: encryption at rest raises effort but, as AML.CS0058 shows, the model is exposed once decrypted. Combine obfuscation and runtime protection with the assumption that a determined adversary will obtain it. See the On-Device AI Security Guide.
- Do not let security depend on model secrecy: for fraud, malware or content classifiers, add server-side checks and ensembles so a white-box evasion of the shipped model is not enough.
- Control external publishing: keep a list of the organisation's verified accounts on public model hubs and route uploads through an approved process.
- Fine-tuned derivatives: adapters and fine-tuned checkpoints reveal the base model plus your data; protect them as production models. See the Fine-Tuning and LoRA Security Guide.
Designing for assumed white-box access
Open-weight models and on-device models are white box to anyone who wants them. Here AML.T0044 is a planning assumption, not an incident.
- Threat model the public weights: adversaries can develop jailbreaks and adversarial inputs against the same weights offline, so guardrails must live outside the model. See the Open-Weight Model Security Guide.
- Keep the value in the system: retrieval data, tools, system prompts and business rules on the server are what the adversary cannot download.
- Test with white-box methods: red-team with gradient-based adversarial testing, since that is what an adversary with the weights will do.
- Watch for the payoff: monitor the live service for the inputs a white-box adversary produces, such as high-confidence misclassification of perturbed inputs.
Where identity fits
Most routes to full access run through a non-human identity: a pipeline service account with read on the whole registry, a hub access token with write scope, a container registry credential. Inventory these identities per model, give each the narrowest scope, prefer short-lived workload credentials over long-lived tokens, and alert on their use from new places. ATLAS tracks repository account abuse through AML.T0012 Valid Accounts.
Detection
| Data source | What to look for |
|---|---|
| Model registry audit logs | Downloads of production models by identities that do not deploy them; bulk downloads; downloads outside pipeline runs |
| Object storage access logs | Reads of checkpoint and weight files from new principals or networks |
| Container registry logs | Anonymous or external pulls of images that contain models; pushes that replace model-bearing tags |
| Public model hub monitoring | New organisations using your company name; your models appearing in repositories you do not own |
| Data loss prevention and egress | Large transfers of model file types (for example .safetensors, .onnx, .tflite) to unapproved destinations |
| Inference telemetry | Sudden success of carefully perturbed inputs, which may indicate attacks crafted offline |
Measuring it
- Model inventory coverage: share of production models with every storage location recorded.
- Access breadth: number of identities with read on each production model; trend downwards.
- Signed artefacts: share of production models verified by signature at load.
- Edge exposure decisions: share of shipped models with a recorded decision that exposure is acceptable.
- Anonymous access: number of registries or buckets holding models that allow unauthenticated read; target zero.
Implementation steps
- Inventory: list every production model and every location that holds a full copy, using the table above.
- Classify: rate each model by business value and by how much security depends on its secrecy.
- Close anonymous access: fix public buckets, open registries and unauthenticated image pulls first.
- Tighten registry roles: split read, write and promote rights per model; move pipelines to short-lived credentials. Pilot on one team and keep the old role available for a short rollback window.
- Add integrity checks: sign models at build and verify at load.
- Decide on distribution: for each shipped model, decide whether to move it server-side (AML.M0017) or accept exposure and add server-side checks.
- Monitor: enable registry, storage and container audit logs and alert on the signals above. For incident handling see the AI Model Theft Response Playbook.
A small team using one managed registry can focus on steps 1, 3 and 4. Organisations shipping models in apps or devices also need the distribution decision and runtime protection.
Common failure patterns
- Everyone in data science can read every model: registry permissions set at workspace level rather than per model.
- Treating encryption at rest as sufficient on devices: the model is decrypted to run and can be captured then.
- Models baked into public or open container images: the image is published for convenience and the weights go with it.
- Unchecked hub organisations: staff upload private models to an organisation they assume is official.
- Security that relies on the model being secret: a client-side classifier is the only check, so offline evasion is enough.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AC-3 Access Enforcement; AC-6 Least Privilege; SC-28 Protection of Information at Rest; SI-7 Software, Firmware, and Information Integrity; AU-12 Audit Record Generation | Model file access, storage protection, integrity checks and logging |
| NIST CSF 2.0 | PR.AA-05; PR.PS-04; DE.CM-09 | Least privilege, logging and monitoring for model stores |
| OWASP Top 10 for LLM Applications 2025 | LLM03:2025 Supply Chain | Tampered or replaced models in registries and hubs |
| ISO/IEC 27001:2022 Annex A | 5.15 Access control; 8.3 Information access restriction; 8.24 Use of cryptography | Access and cryptographic controls for model artefacts |
Practitioner checklist
- Every production model has a recorded list of locations holding a full copy.
- No bucket, registry or image holding a model allows anonymous read.
- Registry access is per model with separate read, write and promote roles.
- Pipeline and hub credentials are short-lived and narrowly scoped.
- Models are signed at build and verified at load.
- Each model shipped to devices has a recorded exposure decision.
- Security decisions do not depend only on a model the adversary can download.
- Registry, storage and container audit logs feed alerting.
Standards and references
- MITRE ATLAS: AML.T0044 Full AI Model Access
- MITRE ATLAS: AML.M0005 Control Access to AI Models and Data at Rest
- MITRE ATLAS: AML.M0017 AI Model Distribution Methods
- MITRE ATLAS: AML.CS0058 Google Photos AI Model Extraction
- MITRE ATLAS: AML.CS0028 AI Model Tampering via Supply Chain Attack
- MITRE ATLAS: AML.CS0027 Organization Confusion on Hugging Face
- Skyld: Google Photos model extraction research
- OWASP: LLM03:2025 Supply Chain
Related NHI Mgmt Group resources: MITRE ATLAS Guide, Fine-Tuning and LoRA Security Guide, Securing AI Model Weights, Model Extraction and Distillation Defence Guide.