Machine learning pipelines run untrusted code, move valuable artefacts between loosely governed systems and hold some of the most powerful credentials in the organisation, yet they are rarely secured with the discipline applied to ordinary software delivery. MLOps security is the practice of protecting the whole machine learning lifecycle: data ingestion, notebooks and experimentation, training compute, experiment tracking and model registries, packaging, deployment and inference. This guide is the overview page for that field. It explains the threats at each stage, the controls that matter most, how to roll them out and how to measure them, then points to the deeper NHI Mgmt Group guides on each platform and topic. It is written for platform engineers, ML engineers and security architects who own or assure ML infrastructure.
Key takeaways
- Model files can be code. Pickle-based formats execute instructions when loaded, so loading an unvetted checkpoint is equivalent to running an unvetted binary; prefer safetensors and restricted loaders.
- ML developer tools are not perimeter services. Ray, Jupyter and self-hosted MLflow assume a trusted network; anyone who reaches their APIs can often run code or change models.
- The pipeline's identities are the real crown jewels. Hub tokens, cloud roles on training clusters and registry write access let an attacker steal or replace models without touching the model itself.
- Integrity must be provable end to end. Hash and sign datasets and models, record lineage, and verify signatures at deployment rather than trusting a registry tag.
- Treat ML like any other supply chain. Third-party models, datasets, Python packages and container images need the same intake, pinning and scanning controls as application dependencies.
What MLOps security covers
MLOps borrows the CI/CD model from software delivery but adds artefacts and tools that conventional AppSec programmes do not see. The table maps each lifecycle stage to its typical components, the assets at risk and the main control objective.
| Stage | Typical components | Assets at risk | Primary control objective |
|---|---|---|---|
| Data ingestion and preparation | Object storage, data lakes, feature stores, labelling tools | Training data, labels, personal data | Provenance, integrity and access control on data |
| Experimentation | Jupyter, hosted notebooks, IDE remote kernels | Credentials in notebooks, data extracts, code | Authenticated, isolated notebook servers with no standing secrets |
| Training | Ray, Kubernetes, Kubeflow, managed training jobs, GPU clusters | Compute, cloud roles, intermediate checkpoints | Network isolation, least-privilege job identities, trusted images |
| Tracking and registry | MLflow, Weights and Biases, cloud model registries | Model versions, metrics, artefact storage keys | Write control, approval gates, immutable versions |
| Packaging and release | CI pipelines, container builds, model hubs | Serialised models, images, signing keys | Safe formats, signing, scanning, provenance attestations |
| Serving and monitoring | Inference servers, endpoints, gateways, vector stores | Model weights, inference API, user data | Authenticated endpoints, rate limits, drift and abuse monitoring |
The UK NCSC's Guidelines for secure AI system development, published in November 2023, organise the same territory into four stages: secure design, secure development (including supply chain security and asset management), secure deployment (protecting infrastructure and models, incident management) and secure operation and maintenance (logging, monitoring and update management). It is a useful frame for assigning ownership across teams.
Threats across the ML lifecycle
Most real MLOps compromises are not exotic adversarial ML. They are exposed management interfaces, leaked tokens and poisoned dependencies, which then give the attacker access to models, data and cloud accounts. MITRE ATLAS captures the AI-specific techniques; MITRE ATT&CK still covers the conventional ones.
| Threat | Where it occurs | Framework reference | Key controls |
|---|---|---|---|
| Malicious serialised model executes on load | Notebooks, training jobs, serving containers | ATLAS AML.T0011 User Execution; AML.T0058 Publish Poisoned Models | safetensors, weights_only=True, intake scanning, sandboxed loading |
| Compromised dependency or model in the supply chain | pip installs, model hubs, base images | ATLAS AML.T0010 AI Supply Chain Compromise; ATT&CK T1195.002 | Pinned hashes, private mirrors, AI-BOM, provenance checks |
| Training data poisoning | Data lakes, scraped corpora, feature stores | ATLAS AML.T0020 Training Data Poisoning | Dataset hashing, write controls, source allowlists, validation sets |
| Model tampering or backdoor in the registry | Registries, artefact buckets | ATLAS AML.T0018 Manipulate AI Model | Signed models, immutable versions, two-person promotion |
| Theft of model weights and artefacts | Buckets, registries, experiment trackers | ATLAS AML.T0035 AI Artifact Collection | Least-privilege storage access, encryption, egress monitoring |
| Model extraction through the inference API | Public or partner endpoints | ATLAS AML.T0024 Exfiltration via AI Inference API | Authentication, rate limits, query pattern monitoring |
| Credential theft from notebooks and configs | Notebooks, .env files, job specs | ATT&CK T1552.001 Credentials In Files | Secret managers, workload identity, secret scanning |
| Remote code execution on exposed cluster APIs | Ray dashboard and Jobs API, Jupyter, MLflow | ATT&CK T1190 Exploit Public-Facing Application | Private networking, authentication proxies, token auth |
For the full taxonomy of adversarial attacks on models themselves, see the Adversarial ML Taxonomy Guide and the OWASP Machine Learning Security Top 10.
Securing the core components
Model artefacts and serialisation
Hugging Face's own security documentation explains that pickle, the default format for PyTorch weights, can execute arbitrary code when a file is unpickled, because opcodes can import and call functions such as exec. The Hub scans uploads with ClamAV and a pickle import scanner, but states that the scan is not foolproof and that checking a file remains the user's responsibility.
- Format policy: accept safetensors (or framework formats that store only tensors) for third-party weights; treat
.bin,.pt,.pkland joblib files as executable content. - Loader settings: call
torch.loadwithweights_only=Truewhere pickle cannot be avoided, and set it explicitly so older PyTorch versions behave the same way. - Intake: pull external models into a quarantine bucket, scan them, inspect pickle imports with
pickletools(which disassembles without executing), then promote to an internal registry.
Notebooks and experimentation
Notebook servers combine interactive code execution, data access and long-lived credentials. Require authentication on every server, bind kernels to private networks, run per-user containers rather than shared servers, and keep tokens out of notebook cells and outputs. See the Jupyter Server Security Guide.
Training compute and orchestration
Ray's security documentation is explicit: Ray runs arbitrary code, expects a safe network and trusted code, and does not enforce isolation between jobs in the same cluster. Anyone who can reach the dashboard, Jobs API or Ray Client ports can run code on the cluster. Built-in token authentication arrived in Ray 2.52.0 and is described as defence in depth, not a replacement for network isolation. The same principle applies to Kubeflow and self-managed GPU clusters: private networking first, then authentication, then separate clusters for workloads that must not share a trust boundary. See the Ray and AI Compute Cluster Security Guide and GPU and AI Cluster Multi Tenancy Security.
Experiment tracking and model registries
Self-hosted MLflow has no authentication unless it is started with --app-name basic-auth, after which permissions are READ, USE, EDIT, MANAGE or NO_PERMISSIONS per resource, with READ as the default for all users. Change that default to NO_PERMISSIONS for sensitive experiments, put the server behind single sign-on, and restrict who can transition a model version to production. See the MLflow and Kubeflow Security Guide and the Weights and Biases Security Guide.
Signing and provenance
The OpenSSF model signing project (sigstore/model-transparency) provides a model_signing CLI with sign, verify and digest subcommands. It hashes every file in a model directory and produces a Sigstore bundle containing an in-toto statement, signed either keylessly via Sigstore and OIDC, with a private key, with certificates or with a PKCS #11 device. Verification recomputes the hashes, so a swapped file fails even if the registry tag is unchanged. See the Model Signing and OpenSSF Model Signing Guide and the AI Supply Chain and AI-BOM Guide.
Serving
Inference servers should load only signed models from the internal registry, run without write access to model storage, sit behind an authenticating gateway and enforce rate limits that make extraction expensive. See the vLLM, Triton and NVIDIA NIM Inference Server Security Guide.
Where identity fits
An ML platform is a dense cluster of non-human identities: training job roles, notebook service accounts, CI runners that push to registries, hub tokens and the storage keys behind artefact URLs. Hugging Face recommends one token per application, fine-grained tokens only in production, and Trusted Publishers so CI pipelines exchange an OIDC token for a short-lived Hub token instead of storing a secret. Apply the same pattern everywhere: workload identity federation for cloud access, per-job roles scoped to one bucket prefix, no shared "ml-admin" account, and owners recorded for every pipeline identity. See the CI/CD Pipeline Identity Security Guide and the Databricks Service Principals and Token Security Guide.
Implementation roadmap
- Inventory (weeks 1 to 3): list notebook servers, clusters, trackers, registries, buckets and endpoints, and every identity and token they use. Scan external attack surface for exposed ML ports such as the Ray dashboard and Jupyter.
- Close exposure (weeks 2 to 6): move management interfaces behind private networking or an authenticating proxy; enable MLflow auth and Ray token auth. Test in a staging cluster first and keep the previous network rules ready to restore if jobs fail.
- Fix identities (weeks 4 to 10): replace static keys in notebooks and pipelines with federation and secret managers; split shared roles per job; revoke unused hub tokens.
- Control intake (weeks 6 to 12): set a format policy, build the quarantine and scan step, pin Python dependencies by hash and mirror them internally. Run the policy in warn mode for one release cycle before blocking.
- Prove integrity (quarter 2): sign models at registration, verify at deployment, record dataset hashes and lineage in the registry. Start with verification logging only, then enforce for production endpoints.
- Monitor and test (ongoing): feed tracker, registry and cluster audit logs to the SOC; include ML platforms in penetration tests and AI red team exercises.
Decision guidance
| Decision | Option A | Option B | Choose A when |
|---|---|---|---|
| Platform | Managed (SageMaker, Vertex AI, Databricks) | Self-hosted open source stack | The team is small or lacks platform security engineers; managed services provide IAM integration, private endpoints and audit logs by default |
| Third-party weights | safetensors only | Pickle allowed after scanning | Almost always; reserve B for specific legacy models loaded in an isolated sandbox |
| Cluster tenancy | Dedicated cluster per team or sensitivity level | Shared cluster with quotas | Workloads handle different data classifications, because Ray and similar tools do not isolate jobs |
| Signing | Keyless (Sigstore, OIDC) | Organisation keys in HSM or KMS | Pipelines already use OIDC and public transparency logging is acceptable; choose B for air-gapped or confidential models |
Small organisations get most of the benefit from three steps: a managed platform with private networking, no long-lived tokens in notebooks, and safetensors-only intake. Large organisations with many teams need a central registry with promotion gates, signing and an AI-BOM so that the security team can answer "which endpoints run this model" within minutes.
Managed platforms have their own hardening guides: see the Amazon SageMaker Security Guide, the Google Vertex AI Security Guide and the Databricks Security Guide.
Detection and measurement
- Log sources: cloud audit logs for model buckets and registries, MLflow or tracker audit events, Ray and Kubernetes API logs, notebook server access logs, hub organisation audit logs and inference gateway logs.
- High-value alerts: registry stage transitions to production outside the pipeline identity, new public exposure of ML ports, bulk reads of model artefacts, job submissions from unknown sources, and inference query volumes far above a client's baseline.
| Metric | Target direction |
|---|---|
| ML management interfaces reachable from the internet | Zero |
| Production models with verified signatures at deployment | Towards 100% |
| Third-party models in pickle format in production | Falling, with documented exceptions |
| Long-lived tokens and keys used by ML pipelines | Falling each quarter |
| Pipeline identities with a named owner | 100% |
| Mean time to answer "where is this model deployed" | Minutes, not days |
For model behaviour monitoring, see ML model evaluation and monitoring for security teams.
Common failure patterns
- Cluster dashboards on the internet: in ShadowRay, attackers used exposed Ray Jobs APIs to run code and steal cloud credentials and AI API tokens.
- Hub tokens leaked in code: the Hugging Face API token exposure showed write-capable tokens in public repositories opening hundreds of organisations' model repositories.
- Over-scoped storage links: the Microsoft SAS token exposure began with an AI research team sharing training material through a link that granted far more than intended.
- Leftover publishing tokens in CI: the Ultralytics PyPI compromise combined a workflow injection with a stored PyPI token to ship a malicious AI library release.
- Dependency confusion in ML packages: the PyTorch torchtriton attack stole SSH keys from users of nightly builds through a package name that resolved to a public index.
- Secrets stored in hosted AI apps: the Hugging Face Spaces breach exposed secrets that app owners had stored on the platform, forcing mass rotation.
Framework mapping
| Control area | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | Other |
|---|---|---|---|
| Inventory of ML systems and artefacts | CM-7, SA-15 | ID.AM-02 | NCSC secure AI guidelines: asset management |
| Model and data provenance | SR-4, SR-11 | GV.SC-05 | NIST SP 800-218A |
| Integrity of models and datasets | SI-7 | PR.DS-01 | ATLAS AML.T0018, AML.T0020 |
| Supply chain intake | SR-3 | GV.SC-05 | ATLAS AML.T0010; ATT&CK T1195.002 |
| Pipeline identities and least privilege | AC-6, IA-5 | PR.AA-01, PR.AA-05 | ATT&CK T1552.001 |
| Network isolation of ML tools | SC-7 | PR.IR-01 | ATT&CK T1190 |
| Logging and monitoring | AU-2, SI-4 | DE.CM-01 | ATLAS AML.T0024 |
NIST SP 800-218A, published in July 2024, is the SSDF community profile for generative AI and dual-use foundation models and extends SSDF 1.1 with AI-specific practices for model producers, system producers and acquirers.
Where to go next
- Platforms: MLflow and Kubeflow Security Guide (tracking, pipelines, auth), Weights and Biases Security Guide (hosted tracking and API keys), Jupyter Server Security Guide (notebook hardening), Hugging Face Security Guide (hub tokens and model intake), Databricks Mosaic AI and Agent Bricks Security.
- Compute: Ray and AI Compute Cluster Security Guide, Confidential AI and GPU Enclaves Guide (protecting data in use).
- Models and data: Securing AI Model Weights, AI Data Poisoning and Backdoors Guide, Open-Weight Model Security Guide, AI Model Security Guide.
- Supply chain: Model Signing and OpenSSF Model Signing Guide, AI Supply Chain and AI-BOM Guide, AI Model Scanning and Supply Chain buyer's guide.
- Response: AI Model Poisoning Incident Playbook and AI Model Theft Response Playbook.
Practitioner checklist
- Inventory every notebook server, cluster, tracker, registry, model bucket and endpoint, with an owner.
- Confirm no Ray, Jupyter, MLflow or similar interface is reachable from the internet.
- Enable authentication on self-hosted trackers and clusters, and set default permissions to none for sensitive projects.
- Adopt a safetensors-first format policy and set
weights_only=Truewherever pickle remains. - Route all external models and datasets through quarantine, scanning and an internal registry.
- Replace static cloud keys and hub tokens in pipelines with federation and short-lived credentials.
- Sign models at registration and verify signatures before deployment to production.
- Pin Python dependencies by hash and serve them from an internal mirror.
- Send registry, tracker and cluster audit logs to the SOC with alerts on production promotions.
- Include ML platforms in the scope of penetration tests and red team exercises.
Standards and references
- NCSC: Guidelines for secure AI system development
- NIST: SP 800-218A Secure Software Development Practices for Generative AI and Dual-Use Foundation Models
- MITRE ATLAS: AI Supply Chain Compromise
- Ray: Security documentation
- MLflow: Basic HTTP authentication
- Hugging Face: Pickle scanning and security
- Hugging Face: User access tokens
- OpenSSF and Sigstore: Model transparency (model signing)
Related NHI Mgmt Group resources: AI Threats and Model Security Guide, CI/CD Security Guide, AI Red Teaming Guide, NHI Discovery Across Cloud, SaaS, CI/CD and On-Premises Environments.