Why can pre-trained machine learning models become a supply chain risk for enterprise environments?

Pre-trained models can carry malicious content in their parameters or serialization logic, which means the model itself becomes a delivery vehicle. If teams assume models are inert data, they may skip inspection and import hidden code, malware, or exfiltration logic. That turns model acquisition into a supply chain control point rather than a routine download.

Why pre-trained models change the supply chain model

Pre-trained models are not just static artifacts. They can embed harmful behavior in learned weights, serialized objects, custom layers, or loader logic, so the risk is not limited to the model file itself. If an enterprise treats a model like inert data, it may import executable behavior without the review, provenance, and validation steps it would apply to other software dependencies.

That is why model acquisition belongs in the same trust and provenance conversation as packages, build artifacts, and third-party code. A model pulled from a repository, marketplace, or shared workspace can arrive with hidden functionality that only appears when the runtime deserializes it, executes hooks, or passes it through a downstream pipeline.

For teams building secure ML workflows, the closest parallel is a software dependency with opaque provenance and potentially active behavior. The model may be consumed by inference code, but the supply chain question is whether the artifact was produced, signed, scanned, and approved in a way that makes its behavior inspectable before deployment. SLSA is a useful reference point for thinking about provenance and integrity even when the artifact is a model rather than a binary. NIST SSDF (SP 800-218) reinforces the need to control what enters the software pipeline, including third-party components and their integrity checks. OpenSSF also remains relevant because the same supply chain discipline that protects packages applies to model distribution paths.

Where the hidden risk actually lives

The practical danger is usually not that a model “thinks badly,” but that it creates a control bypass. A model can carry malicious serialization logic, unsafe deserializers, or embedded code paths that trigger when engineers load it into notebooks, CI pipelines, or serving infrastructure. Once imported, it can also become a vehicle for data exfiltration, unexpected outbound requests, or privileged access to adjacent systems if the hosting environment is loosely governed.

This is especially important when models are shared across teams, copied from public sources, or integrated into automated retraining and deployment pipelines. The more reusable the artifact, the more likely it is to be moved by people who trust the label and skip inspection of the actual contents. In that sense, the model is part of the attack surface before it ever produces a prediction.

NHIMG’s Ultimate Guide to Non-Human Identities is useful background here because supply chain abuse often succeeds by exploiting excessive trust in non-human systems, tokens, and automated workflows. The same pattern shows up when a model artifact reaches environments with broad execution or network permissions. For a concrete supply chain analogue, the LiteLLM PyPI package breach and the Reviewdog GitHub Action supply chain attack both show how trusted artifacts can become delivery mechanisms once they are consumed automatically.

One useful signal from NHIMG’s research is that 92% of organisations expose non-human identities to third parties, which illustrates how often external trust relationships become the weak link in modern pipelines. The same trust problem applies when teams source models externally and then grant them broad runtime access without treating the model as an untrusted input.

Risk and Threat Considerations

The main risk is silent trust expansion: once a model is accepted as “just a model,” organisations often stop applying the checks they would use for code, dependencies, or executables. That creates a path for malicious behavior to enter the enterprise through a normal ML workflow rather than an obvious software installation.

Failure mechanism: A model can exploit weak artifact governance by hiding code, unsafe serialization payloads, or exfiltration logic inside the object that engineers load, convert, or deploy. When deserialization, plugin loading, or pipeline automation runs without inspection, the malicious behavior is activated inside trusted infrastructure.

Impact: The result can be credential theft, data leakage, poisoned inference outputs, or a foothold for broader supply chain compromise. At enterprise scale, the blast radius increases when the same model is reused across many environments or when downstream systems inherit the model’s permissions and network reach.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

MITRE ATT&CK address the attack and risk surface, while CIS Controls v8, NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
CIS Controls v8	CIS 6 — Access Control Management	Controls who can load and run untrusted model artifacts.
	CIS 3 — Data Protection	Protects model files and associated secrets from tampering or leakage.
	CIS 16 — Application Software Security	Applies secure handling to software-like artifacts such as serialized models.
Recommendation — Restrict model registry and runtime access to approved users and pipelines. Protect model artifacts and secrets with encryption, integrity checks, and secure storage. Scan and validate model artifacts before promotion into production workflows.
NIST CSF 2.0	ID.SC-4 — Supply Chain Risk Management	Directly addresses trust and integrity of third-party artifacts entering the environment.
	PR.DS-6 — Data Integrity	Model files must retain integrity to prevent hidden modification or payload insertion.
	PR.PS-3 — Configuration Management	Model ingestion depends on controlled handling of approved artifacts and versions.
Recommendation — Assess model suppliers, provenance, and integrity before accepting artifacts. Verify model integrity before loading, converting, or deploying it. Track model versions and approve only controlled artifact sources.
NIST AI RMF	MAP 1.3 — AI system supply chain and provenance	Directly covers provenance, integrity, and supplier trust for AI artifacts.
	MEASURE 2.2 — AI system security testing	Testing should detect malicious or unsafe model behavior before release.
	MANAGE 1.2 — AI risk treatment and monitoring	Requires governance over ongoing AI-related risk after model adoption.
Recommendation — Document model provenance and verify artifact integrity before deployment. Test models for unsafe behaviors, hidden payloads, and unexpected execution paths. Monitor deployed models for drift, abuse, and supply chain compromise indicators.
MITRE ATT&CK	T1195 — Supply Chain Compromise	Pre-trained models can function as a compromised delivery artifact.
Recommendation — Hunt for tampered artifacts and compromised upstream model sources.

Practitioner Guidance

What to verify: Treat model intake as a controlled change. Verify source provenance, checksum or signature expectations, and whether the model format or loader can execute code during import, conversion, or serving. If the answer is unclear, the artifact should be handled like untrusted software, not approved data.

Decision rule: If a model can influence code execution, filesystem access, network calls, or downstream automation, require a higher trust threshold than you would for a plain file upload. If the artifact is only used in a sandboxed evaluation flow, the acceptable exposure is lower, but the provenance check still matters.

What good looks like: Approved models have a documented source, are scanned before use, are stored in controlled registries, and are loaded only by runtime environments with tightly bounded permissions. The key control question is whether the model can do anything other than produce inference results once it is admitted.

Practitioner takeaway: The right mindset is not “can we trust this model’s predictions,” but “can we trust everything that happens when this model is introduced into our environment.”