Malicious Models on Hugging Face

What malicious model artefacts change about the risk model

Model repositories make distribution easy, which also makes it easier to hide unsafe logic inside weights, serialised artefacts, or helper packages. For a team evaluating models, the key change is that trust is no longer limited to the model’s output quality, because the artefact itself can carry execution or persistence risk.

That is why model review needs to go beyond benchmark performance and into provenance, packaging format, and dependency behaviour. A model can look legitimate while still being designed to trigger code execution, exfiltrate data, or contaminate downstream fine-tuning.

How malicious models are typically used

Malicious uploads usually aim at one of three outcomes: executing code during load or inference, stealing secrets from the runtime environment, or poisoning downstream systems that reuse the model. The malicious behaviour may sit in the model weights, in a deserialisation path, or in a dependency pulled in as part of the bundle.

In practice, the danger is strongest when teams treat a model like a static file instead of an executable supply-chain object. That assumption can turn a routine pull from a public repository into an attacker-controlled entry point.

For a broader supply-chain lens on model artefacts, packages, and provenance, see AI Supply Chain Security and AI-BOM Guide.

Why repository trust is not enough

Repository reputation helps, but it is not a control. Attackers can upload convincing artefacts, impersonate known publishing patterns, or slip malicious behaviour into a dependency that only activates after installation or loading.

That is especially important for organisations that reuse public models in pipelines, demos, agentic workflows, or internal experimentation. Once the artefact enters a trusted workflow, the malicious behaviour can inherit the same access to data, tokens, and infrastructure that the surrounding environment already has.

Exposure is not only about the model hub itself. If teams have stored credentials, API keys, or tokens near the execution environment, a malicious model can turn that proximity into credential theft or later movement.

On that point, Hugging Face API tokens exposed 2023 is a useful reminder that public model ecosystems and secret exposure often overlap.

What defenders should verify before use

Defenders should treat model intake as a verification problem. That means checking the publisher, reviewing the artefact format, inspecting bundled dependencies, and testing the model in isolation before it is allowed anywhere near production data or production credentials.

Isolation matters because many malicious behaviours only become visible when the model is loaded, imported, or executed in a real runtime. Sandboxing and environment separation reduce the blast radius if the artefact contains hidden code, poisoned components, or unexpected network behaviour.

Where available, artefact provenance, signing, and dependency hygiene should be part of the intake decision, not an afterthought. For practical control mapping, NIST guidance and supply-chain controls are often the right reference points, but the operational test remains simple: can this artefact be trusted to run without changing the environment around it?

The same supply-chain discipline also applies to poisoned or booby-trapped builds, which is why AI supply-chain and AI-BOM controls belong in the review process alongside model scanning.

Risk and Threat Considerations

Malicious model artefacts are dangerous because they collapse the boundary between “content” and “code”. A single download can become an execution path, a credential theft path, or a poisoning path for every system that trusts the model after import.

Failure mechanism: The artefact is designed to trigger harmful behaviour during loading, dependency resolution, or downstream reuse, often in an environment that already holds secrets or privileged access.

Impact: The result can be code execution, secret theft, poisoned training data, compromised inference pipelines, or broader compromise of adjacent systems that relied on the model as a trusted input.

Practitioner Guidance

Why practitioners should care: Model review should be handled like a software intake and supply-chain control, not a simple file download. If a model can execute logic or call dependencies, it needs the same scrutiny you would apply to untrusted code.

What to watch for: Be suspicious of opaque provenance, unusual serialisation formats, unexpected dependency graphs, and any model that requires broad runtime permissions just to load. Those are common signs that the artefact may be carrying more than weights.

Practitioner takeaway: The safest default is to test every new model in a restricted environment first, then promote it only after the artefact, publisher, and dependency chain are all understood.

Related resources from NHI Mgmt Group

• What should teams do first after running a malicious Hugging Face repository?
• Malicious Hugging Face Model
• What should security teams do if a Hugging Face repo may have exposed browser and cloud credentials?
• What breaks when Hugging Face API tokens are exposed in public code?