Unsafe Model Loading

Expanded Definition

Unsafe model loading describes a trust failure in the loading path, not a property of the model weights alone. The key boundary is whether import-time behaviour can trigger code execution through deserialisation hooks, tokenizer setup, or helper functions that assume the file is benign. In practice, the risk appears when convenience features are used before provenance, integrity, and format checks are enforced.

That distinction matters because two models with identical task behaviour can carry very different load-time risk. A safe workflow treats model artefacts as data until validated, while an unsafe workflow allows the loader to act on embedded instructions. Guidance on this point is still evolving across the industry, but the consensus is clear that import paths should not be allowed to execute arbitrary logic by default. For adjacent terms, this is broader than “malicious model” and narrower than general software supply-chain risk.

A common misunderstanding is to assume that a model registry or familiar source name makes the loading step safe. In reality, the control problem is the loader’s execution surface and the assumptions made by the surrounding tooling.

Examples and Use Cases

Unsafe model loading appears in ordinary workflows whenever developers prioritise convenience over validation. It is especially visible in environments that automate experimentation, promote copy-pasted snippets, or rely on libraries that silently enable richer loading behaviour.

• A data scientist imports a model file that triggers deserialisation logic before the file is verified against an expected schema.
• An engineering team enables a helper option that restores custom objects during load, expanding the execution surface beyond the intended model data.
• A tokenizer package runs setup code at initialisation time, so a seemingly harmless dependency change alters what executes during model startup.
• An internal platform accepts externally sourced artefacts without checking provenance, making the loading step the easiest place for hidden logic to run.
• A CI pipeline tests only model output, not load-time behaviour, so unsafe artefacts can pass review until deployment.

The tradeoff is straightforward: permissive loading reduces friction for experimentation, but it also reduces the boundary between content and code. That is why loader defaults deserve the same scrutiny as any other execution path.

Security Implications

When unsafe model loading is present, the first failure is often hidden code execution during import rather than during inference. That means an attacker, compromised upstream package, or tampered artefact can gain an execution foothold before normal application controls, logging, or policy enforcement have fully engaged. The practical consequence is that the organisation may believe it is merely “loading a model” while it is actually running logic from an untrusted source.

The blast radius can extend beyond the model runtime. Load-time execution can touch filesystem paths, environment variables, credentials, network endpoints, or downstream automation that assumes the artefact is passive. In mature environments, this often shows up as unexpected network calls, unexplained file changes, or a model that behaves normally only after its initial loading side effects have already occurred.

Impact: compromise of the loading host, contamination of build or training environments, and untrusted artefacts becoming a persistence or staging point for broader system access.

Domain and Governance Relevance

Unsafe model loading sits squarely in AI security and software supply-chain governance because the control failure happens at the boundary between artefact and execution. The term matters operationally because the organisation must decide which model sources are allowed, which formats are permitted, and whether loading behaviour is constrained by design or by convention.

For identity and machine-access governance, the relevant shift is that model artefacts can become a path into systems that hold service credentials, API keys, or orchestration access. That does not make the term primarily an NHI concept, but it does mean the loading path should be treated as a privileged execution surface when machine accounts or automated pipelines are involved. In NHIMG terms, the practical question is not “what model is this?” but “what can this loader execute before trust is established?”

Where teams support external model intake, governance should focus on provenance, allowed formats, and whether the loader is permitted to instantiate custom code at all. That is the point where a convenience feature becomes a control decision.

Related resources from NHI Mgmt Group

• How do organisations stop a model’s safe response from becoming unsafe execution?
• Who is accountable for model outputs that leak unsafe guidance through iterative probing?
• Why can a clean AI model still produce unsafe tool calls?
• How can teams reduce the impact of unsafe model output in MCP workflows?