Malicious Models on Hugging Face are model artefacts uploaded to the repository that contain harmful weights, unsafe code or booby-trapped dependencies. Attackers may use them to execute code, steal data or poison downstream training and inference. Defenders inspect publishers, review artefacts, scan dependencies and run models in isolated environments.
What malicious model artefacts change about the risk model
Model repositories make distribution easy, which also makes it easier to hide unsafe logic inside weights, serialised artefacts, or helper packages. For a team evaluating models, the key change is that trust is no longer limited to the model’s output quality, because the artefact itself can carry execution or persistence risk.
That is why model review needs to go beyond benchmark performance and into provenance, packaging format, and dependency behaviour. A model can look legitimate while still being designed to trigger code execution, exfiltrate data, or contaminate downstream fine-tuning.
How malicious models are typically used
Malicious uploads usually aim at one of three outcomes: executing code during load or inference, stealing secrets from the runtime environment, or poisoning downstream systems that reuse the model. The malicious behaviour may sit in the model weights, in a deserialisation path, or in a dependency pulled in as part of the bundle.
In practice, the danger is strongest when teams treat a model like a static file instead of an executable supply-chain object. That assumption can turn a routine pull from a public repository into an attacker-controlled entry point.
For a broader supply-chain lens on model artefacts, packages, and provenance, see AI Supply Chain Security and AI-BOM Guide.
Why repository trust is not enough
Repository reputation helps, but it is not a control. Attackers can upload convincing artefacts, impersonate known publishing patterns, or slip malicious behaviour into a dependency that only activates after installation or loading.
That is especially important for organisations that reuse public models in pipelines, demos, agentic workflows, or internal experimentation. Once the artefact enters a trusted workflow, the malicious behaviour can inherit the same access to data, tokens, and infrastructure that the surrounding environment already has.
Exposure is not only about the model hub itself. If teams have stored credentials, API keys, or tokens near the execution environment, a malicious model can turn that proximity into credential theft or later movement.
On that point, Hugging Face API tokens exposed 2023 is a useful reminder that public model ecosystems and secret exposure often overlap.
What defenders should verify before use
Defenders should treat model intake as a verification problem. That means checking the publisher, reviewing the artefact format, inspecting bundled dependencies, and testing the model in isolation before it is allowed anywhere near production data or production credentials.
Isolation matters because many malicious behaviours only become visible when the model is loaded, imported, or executed in a real runtime. Sandboxing and environment separation reduce the blast radius if the artefact contains hidden code, poisoned components, or unexpected network behaviour.
Where available, artefact provenance, signing, and dependency hygiene should be part of the intake decision, not an afterthought. For practical control mapping, NIST guidance and supply-chain controls are often the right reference points, but the operational test remains simple: can this artefact be trusted to run without changing the environment around it?
The same supply-chain discipline also applies to poisoned or booby-trapped builds, which is why AI supply-chain and AI-BOM controls belong in the review process alongside model scanning.
Risk and Threat Considerations
Malicious model artefacts are dangerous because they collapse the boundary between “content” and “code”. A single download can become an execution path, a credential theft path, or a poisoning path for every system that trusts the model after import.
Failure mechanism: The artefact is designed to trigger harmful behaviour during loading, dependency resolution, or downstream reuse, often in an environment that already holds secrets or privileged access.
Impact: The result can be code execution, secret theft, poisoned training data, compromised inference pipelines, or broader compromise of adjacent systems that relied on the model as a trusted input.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while CIS Controls v8, NIST SP 800-53 Rev 5 and SLSA set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP API Security Top 10 | API8 — Security Misconfiguration | Malicious model artefacts exploit unsafe runtime and deployment assumptions. |
| Recommendation — Harden model runtimes so untrusted artefacts cannot execute with excessive system access. | ||
| CIS Controls v8 | CIS-16 — Application Software Security | Model files and dependencies behave like software supply-chain inputs. |
| Recommendation — Review untrusted model artefacts and their dependencies before allowing execution. | ||
| NIST SP 800-53 Rev 5 | SI-7 — Software, Firmware, and Information Integrity | This term centers on hostile artefacts that tamper with integrity and execution trust. |
| Recommendation — Verify artefact integrity and block untrusted model packages from trusted environments. | ||
| SLSA | Supply-chain Levels for Software Artifacts | Model artefacts and bundled dependencies need provenance and build-trust assurance. |
| Recommendation — Require provenance evidence for model artefacts before they enter downstream systems. | ||
| OWASP Non-Human Identity Top 10 | NHI-02 — Secret Leakage | Malicious models may steal credentials, tokens, or other secret material from their runtime. |
| Recommendation — Isolate model execution so stolen secrets cannot be accessed from the model runtime. | ||
Practitioner Guidance
Why practitioners should care: Model review should be handled like a software intake and supply-chain control, not a simple file download. If a model can execute logic or call dependencies, it needs the same scrutiny you would apply to untrusted code.
What to watch for: Be suspicious of opaque provenance, unusual serialisation formats, unexpected dependency graphs, and any model that requires broad runtime permissions just to load. Those are common signs that the artefact may be carrying more than weights.
Practitioner takeaway: The safest default is to test every new model in a restricted environment first, then promote it only after the artefact, publisher, and dependency chain are all understood.