Models, training sets, notebooks and inference logs are scattered across more systems than most security teams have mapped, and an adversary inside the network only has to find one of them. MITRE ATLAS tracks this as AML.T0035 AI Artifact Collection, in the Collection tactic (AML.TA0009): adversaries collect AI artifacts for exfiltration or for use in AI attack staging. ATLAS defines AI artifacts as models and datasets, plus other telemetry data produced when interacting with a model, and rates the technique as realised. This guide is for security engineers, MLOps and data platform teams: where artifacts live, how to restrict and encrypt them, how to detect collection, and how to respond. Protecting frontier model weights in depth is covered in the Securing AI Model Weights guide; this page covers the wider artifact estate. For the framework, see the MITRE ATLAS Guide.
Key takeaways
- Artifacts are more than weights. Datasets, feature stores, experiment runs, notebooks, evaluation sets, prompts and inference logs are all collectable, and many are less protected than the model.
- Collection is the step before loss. AML.T0035 feeds exfiltration (AML.TA0010) and AI attack staging (AML.TA0001, now named AI Attack Adaptation), so detecting it early stops both theft and better-targeted attacks.
- ATLAS maps four mitigations. AML.M0001 Limit Model Artifact Release, AML.M0005 Control Access to AI Models and Data at Rest, AML.M0012 Encrypt Sensitive Information and AML.M0017 AI Model Distribution Methods.
- AI platforms are often built for trusted networks. Compute clusters, experiment trackers and notebooks need authentication turned on and network isolation applied by the customer.
- Machine identities do most of the reading. Training jobs, pipelines and notebooks hold broad storage rights, so a stolen workload credential is the usual route to bulk collection.
What counts as an AI artifact
| Artifact | Typical location | Why an adversary wants it |
|---|---|---|
| Model weights and checkpoints | Model registries, object storage, serving hosts, edge devices | Intellectual property; white-box access to craft evasions |
| Training and fine-tuning datasets | Data lakes, buckets, feature stores, labelling tools | Proprietary or personal data; material to poison or to train a proxy |
| Evaluation and golden sets | Experiment trackers, repositories | Shows what the model is tested against, and where it is weak |
| Experiment metadata | Experiment tracking servers, pipeline stores | Architectures, hyperparameters and data lineage that speed up replication |
| Notebooks and code | Notebook servers, repositories, shared drives | Credentials, data paths and preprocessing logic |
| Prompts and agent configuration | Application config, prompt stores, agent frameworks | Maps guardrails and tools for later attacks |
| Inference telemetry | Logging pipelines, observability tools, gateways | Inputs and outputs that may contain personal data or model behaviour at scale |
ATLAS treats finding these as a separate step, AML.T0007 Discover AI Artifacts, and moving them out as AML.T0025 Exfiltration via Cyber Means. Collection sits between the two.
How collection happens
ATLAS case studies show three recurring patterns:
- Red team inside the network: in AML.CS0010, a red team attacking an internal cloud AI service found the target model file and the training data it needed after gaining valid account access.
- Code execution in a notebook: AML.CS0018 describes searching a victim system for private data, including model artifacts, after code execution in a hosted notebook environment, where notebooks can run shell commands.
- Exposed AI compute: AML.CS0023 ShadowRay records Ray clusters reachable without authentication, where adversaries could collect production models and data. The incident is covered on the ShadowRay 2024 breach page, which also lists cloud, AI service and database credentials exposed on the same nodes.
The common thread is ordinary access, a valid account, a notebook or an open control plane, followed by reading artifacts that were reachable from there.
Prevention and hardening
| ATLAS mitigation | ATLAS rationale | What to do |
|---|---|---|
| AML.M0005 Control Access to AI Models and Data at Rest | Access controls on model registries, limited internal access to production models, training data only for approved users | Separate registry and storage permissions for read, write and promote; deny broad bucket reads; review access quarterly (our cadence) |
| AML.M0012 Encrypt Sensitive Information | Protect AI artifacts with encryption | Encrypt artifact stores with customer-managed keys and restrict key use to the services that need it, so storage access alone is not enough |
| AML.M0001 Limit Model Artifact Release | Limit public release of data, algorithms, architectures and checkpoints used in production or representative of it | Review papers, model cards, public repositories and conference demos for production details before release |
| AML.M0017 AI Model Distribution Methods | Serving models in the cloud, and computing features there, reduces adversary access compared with edge deployment | Where models must ship to devices, assume they will be extracted; see the On-Device AI Security Guide |
Platform settings the customer controls
Several common AI platforms place security at the deployment boundary, so the settings below are customer responsibilities. Platforms are listed alphabetically.
- Jupyter notebook servers: require authentication, avoid running kernels with broad cloud credentials, and keep notebooks out of shared storage that holds production data. See the Jupyter Server Security Guide.
- MLflow tracking server: the MLflow docs describe built-in basic authentication, enabled with the basic-auth app name, with permission levels READ, USE, EDIT, MANAGE and NO_PERMISSIONS. All users get READ by default through the default_permission setting; set it to NO_PERMISSIONS where experiments hold sensitive artifacts and grant access per experiment and model. See the MLflow and Kubeflow Security Guide.
- Ray clusters: the Ray docs state that security and isolation must be enforced outside the cluster and that anyone who can reach the dashboard, Jobs or Client ports can run code. Keep clusters on an isolated network, turn on token authentication (the docs describe it as added in Ray 2.52.0, with default enablement extended in later releases), and use separate clusters for workloads that need isolation. See the Ray and AI Compute Cluster Security Guide.
Decision guidance
- Organisations using only hosted models: the artifact estate is mostly prompts, fine-tuning data and inference logs. Focus on logging pipelines and fine-tuning data stores.
- Organisations training their own models: registries, datasets and compute clusters dominate. Start with an artifact inventory and storage access review.
- Research-heavy teams: notebooks and experiment trackers sprawl fastest. Default-deny access in trackers and short-lived credentials in notebooks give the most risk reduction per hour.
Where identity fits
Most artifact reads are made by machine identities: training jobs, data pipelines, serving containers and notebook kernels. Our recommendations (NHI Mgmt Group):
- Scope workload identities per pipeline: a training job should read its own dataset and write its own outputs, not list every bucket in the account.
- Use short-lived credentials: replace static cloud keys in notebooks and cluster nodes with workload identity federation, so a compromised node does not yield long-lived access.
- Separate promote from read: the identity that can promote a model to production should not be the one that experiments use.
- Treat exposed nodes as credential leaks: if an AI compute node was reachable, rotate every credential on it, as the ShadowRay lessons show.
See the Securing the data science notebook platform itself guide for notebook identity patterns.
Detection
| Signal | What it suggests | Data source |
|---|---|---|
| Bulk reads or listings across model registries or dataset buckets by one identity | Collection sweep | Object storage access logs, registry audit logs |
| Model or dataset downloads by an identity that has never accessed them | Stolen credential or misused access | Registry and storage logs joined to identity baselines |
| Notebook kernels or cluster jobs reading paths outside their project | Code execution used to search for artifacts | Cloud audit logs, cluster job logs |
| Large archive creation or compression on AI hosts | Staging for exfiltration | Endpoint and process telemetry |
| New or unknown jobs submitted to AI compute clusters | Unauthorised code on the cluster | Cluster job history, dashboard access logs |
| Exports from inference logging or observability tools | Collection of telemetry and user data | Logging platform audit trail |
In MITRE D3FEND terms, these map to D3-FAPA File Access Pattern Analysis and D3-UBA User Behavior Analysis. Turn on data access audit logs for artifact stores; on many cloud storage services they are a separate setting from administrative logs.
Metrics
- Inventory coverage: share of artifact stores (registries, buckets, trackers, clusters) in the AI asset inventory with an owner.
- Audit coverage: share of those stores with data access logging enabled and sent to the SOC.
- Broad access count: number of identities with read access to all models or all datasets.
- Static credential count: long-lived keys found on notebooks and AI compute nodes.
- Exposed control planes: AI dashboards and APIs reachable from the internet, target zero.
Implementation steps
- Inventory: list every artifact store and AI platform, with owner, data classification and network exposure. The AI Supply Chain and AI-BOM Guide helps structure the record.
- Close exposure: remove internet exposure from AI control planes and turn on authentication on trackers, notebooks and clusters.
- Tighten access: review who and what can read each store; remove wildcard reads; split read, write and promote roles.
- Encrypt and log: apply customer-managed keys and data access logging to sensitive stores.
- Detect: deploy the analytics above, starting with bulk reads and first-time downloads, in alert-only mode for two weeks to tune (our recommendation).
- Review releases: add an artifact release check to the publication process for papers, demos and public repositories.
Roll back any access change that breaks a pipeline by restoring the previous policy version, then narrow it again with the pipeline owner.
Response
- Scope: identify which artifacts were read, by which identity, from which host, and over what period.
- Contain: revoke or rotate the identity's credentials, isolate affected notebooks or clusters, and block further access to the stores.
- Preserve evidence: keep storage, registry and cluster logs and host images.
- Assess impact: decide whether collected datasets contained personal data, which triggers privacy assessment, and whether collected models enable evasion or extraction, which triggers model hardening.
- Follow on: check for exfiltration (AML.T0025) and record the incident; see the AI Incident Reporting Guide.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| MITRE ATLAS | AML.T0035 in AML.TA0009; preceded by AML.T0007; followed by AML.T0025 | Technique and its neighbours |
| MITRE ATLAS mitigations | AML.M0001, AML.M0005, AML.M0012, AML.M0017 | Mitigations ATLAS maps to the technique |
| NIST SP 800-53 Rev. 5 | AC-3, AC-6, SC-28, CM-8, AU-12 | Access enforcement, least privilege, encryption at rest, inventory, logging |
| NIST CSF 2.0 | ID.AM-07, PR.AA-05, PR.DS-01, DE.CM-09 | Data inventory, access, data at rest, monitoring |
| MITRE D3FEND | D3-FAPA, D3-UBA | Access pattern and behaviour analytics |
Common failure patterns
- Protecting the model, not the data: the registry is locked down while the training data bucket is readable by every pipeline.
- Default read for everyone: experiment trackers left at a default that lets all users read every run and artifact.
- AI control planes on the internet: dashboards and job APIs exposed for convenience, as in ShadowRay.
- Static keys in notebooks: long-lived cloud keys pasted into notebooks give anyone who runs code there the same reach.
- No data access logs: only administrative events are logged, so bulk reads leave no trace.
Practitioner checklist
- Maintain an inventory of every AI artifact store and platform with owner and exposure.
- Remove internet exposure from AI dashboards, job APIs and trackers, and require authentication.
- Set experiment tracker default permissions to no access where artifacts are sensitive.
- Split read, write and promote permissions on registries and datasets.
- Encrypt sensitive artifact stores with customer-managed keys and restricted key use.
- Replace static keys on notebooks and AI compute with short-lived workload credentials.
- Enable data access logging on artifact stores and alert on bulk and first-time reads.
- Review public releases for production artifact details before publication.
Standards and references
- MITRE ATLAS: AI Artifact Collection
- MITRE ATLAS: Control Access to AI Models and Data at Rest
- MITRE ATLAS: ShadowRay case study
- Ray: Security
- MLflow: Basic HTTP authentication
- NCSC: Guidelines for secure AI system development
- NIST: AI 100-2 E2025 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
Related NHI Mgmt Group resources: Securing AI Model Weights · MLflow and Kubeflow Security Guide · Ray and AI Compute Cluster Security Guide · MITRE ATLAS Mitigations Guide