Malicious Model Deserialisation (Keras and Pickle) is a code-execution risk that occurs when a model file is loaded from an untrusted or tampered source. Pickle-based objects can execute attacker-controlled behaviour during deserialisation, and unsafe model loading can do the same. Only load trusted artefacts, verify integrity and avoid deserialising unknown files in production pipelines.
What Malicious Model Deserialisation Means
malicious model deserialisation is not just a file-format issue, it is a trust-boundary failure. The danger is that the model artefact itself can carry executable behaviour, so loading an unknown or tampered file can trigger code before the model is even used.
With Keras and Pickle, the risk is especially important because deserialisation can become an execution path rather than a passive data import. That means the loader, runtime permissions, and pipeline trust assumptions all matter, not only the model architecture.
Why Keras and Pickle Are Dangerous When Trust Is Weak
Pickle is powerful because it can serialise complex Python objects, but that power is exactly why it is risky with untrusted inputs. A malicious object graph may invoke attacker-controlled behaviour during load time, which turns a model file into a delivery mechanism for code execution.
Keras model loading can be dangerous for the same reason when unsafe loading modes or unknown artefacts are accepted. Even if the file looks like a model, the security question is whether the load step performs hidden actions that an attacker can abuse.
This is why provenance matters. The security property you want is not simply “the file loads successfully”, but “the artefact is known, expected, and integrity-checked before any deserialisation occurs”. For broader supply-chain protection around model artefacts, AI Supply Chain Security and AI-BOM Guide is the right companion reference.
How Organisations Reduce Exposure
Exposure usually comes from one of three places: a model downloaded from an untrusted source, a legitimate artefact that was modified in transit or storage, or a pipeline that treats all model files as safe by default. Any of those conditions can make the deserialisation step the point of compromise.
Teams should treat model artefacts like software inputs that need provenance, integrity validation, and allowlisted handling. That includes knowing where the file came from, how it was produced, and whether it was signed or otherwise verified before the loader sees it. If you need the broader insecure-loading pattern, Insecure Deserialisation Guide explains the underlying mechanism across common runtimes.
In practice, the safest assumption is that unknown model files are hostile until proven otherwise. That is especially true in automated training, evaluation, and deployment pipelines where a single unsafe artefact can affect many environments at once.
Detection, Response, and Post-Compromise Review
If malicious deserialisation is suspected, the question is not only whether the model was loaded, but what happened immediately after load. Look for unexpected processes, outbound connections, file writes, environment inspection, or other behaviour that should not occur during ordinary model import.
Post-patch or post-fix review should include the model artefact, the storage path, the pipeline history, and any systems that executed during loading. A compromised model can be a one-time trigger, but the surrounding environment may still be affected if the attacker used the load event to establish follow-on access.
For threat-detection context around code execution, persistence, and post-compromise activity, MITRE ATT&CK Enterprise Matrix provides a useful lens on the behaviours that can follow initial execution.
Risk and Threat Considerations
Untrusted model deserialisation is risky because it collapses a data-handling step into a code-execution opportunity. The threat is not theoretical, the loader may process attacker-controlled object content before any higher-level validation can stop it, which makes the model file itself a delivery vector.
Failure mechanism: An attacker supplies or alters a model artefact so that deserialisation triggers executable behaviour, then uses that execution to run code, inspect the environment, or pivot into adjacent systems.
Impact: The result can include remote code execution, pipeline compromise, exposure of secrets or credentials available to the runtime, and contamination of downstream training or deployment workflows.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| MITRE ATT&CK | T1059 — Command and Scripting Interpreter | Malicious deserialisation can culminate in attacker-run code execution on the host. |
| Recommendation — Map post-load execution to T1059 and hunt for spawned processes or script activity after model import. | ||
| NIST SP 800-53 Rev 5 | SI-10 — Information Input Validation | Model artefacts are untrusted inputs that must be validated before processing or execution. |
| SI-7 — Software, Firmware, and Information Integrity | Integrity verification is central to preventing tampered model files from executing malicious behaviour. | |
| CM-8 — System Component Inventory | Tracking model artefacts and pipeline components supports trust decisions for loaded files. | |
| Recommendation — Validate model artefacts before deserialising them and reject untrusted or tampered inputs. Verify model integrity and provenance before loading it into any runtime or pipeline. Inventory model artefacts and their sources so unapproved files can be identified and blocked. | ||
Practitioner Guidance
Why practitioners should care: Treat model loading as an execution boundary, not a harmless import step. The loader, the artefact source, and the runtime permissions all need explicit trust decisions because a malicious file can turn the model pipeline into an entry point.
What to watch for: Prefer trusted artefact sources, integrity verification, and safer loading modes wherever they exist. If a pipeline must accept externally produced models, isolate the loader, minimise its privileges, and review any deserialised artefact before it reaches production.
Practitioner takeaway: If the model was not produced and verified inside a trusted process, assume it can behave like code, not just data.
Related resources from NHI Mgmt Group
- Guide: AI Data Poisoning and Backdoors Guide
- How can organisations reduce the risk of malicious model supply chain attacks?
- What does a fully malicious server threat model reveal in a password manager audit?
- How can security teams tell whether Keras model loading is actually safe?
- Why do open AI model ecosystems increase the risk of secrets exposure and malicious code execution?