A Model Serialisation Attack exploits the process of saving or loading a model to trigger code execution, tamper with behaviour, or smuggle malicious objects. It targets unsafe deserialisation paths and untrusted artefacts. Defenders prefer safe formats, avoid loading unverified files, and isolate model conversion workflows.
How model serialisation attacks work
Model serialisation attacks abuse the save, load, or conversion step to make a trusted model artifact do something it should never do. The danger is not the model itself, but the deserialiser, parser, or conversion pipeline that treats attacker-controlled bytes as structured objects.
These attacks matter because serialisation formats often bundle code-like structure, object graphs, or executable hooks. If a pipeline accepts an unverified file, a malicious payload can ride along with a legitimate-looking model and trigger behaviour during load, conversion, or inspection.
Why this attack class is dangerous
Unsafe deserialisation can create direct code execution, but it can also quietly alter model behaviour, introduce backdoors, or plant malicious objects that persist across environments. That makes the issue especially serious in workflows that move models between notebooks, CI jobs, registries, and deployment systems.
A model artifact is often treated as data, yet the loader may implicitly restore methods, references, or embedded metadata. When that trust boundary is weak, serialisation becomes an execution surface rather than a neutral transport layer.
Common exposure points
Exposure usually appears anywhere a team loads third-party models, unreviewed checkpoints, shared research artifacts, or files received from another environment. Risk increases when model conversion happens in a privileged build worker, an automated training job, or a system with access to secrets, internal networks, or deployment credentials.
The same pattern can also affect downstream tooling such as experiment trackers, artifact scanners, and preview services if they deserialize objects automatically. A safe workflow depends on knowing exactly which formats are accepted and which loaders are allowed to interpret them. See also CISA cyber threat advisories for broader exploitation patterns that defenders can correlate with suspicious artifact handling.
Defensive controls and safe handling
Prefer serialization formats and libraries that avoid arbitrary object reconstruction, and separate model data from executable code wherever possible. Treat untrusted model artifacts like any other high-risk input, meaning they should be validated, scanned, and opened only in constrained environments.
Isolation matters because conversion and inspection steps can be the point where a malicious payload activates. Teams should keep these workflows on hardened hosts, with minimal privileges and no unnecessary access to production secrets. For a control baseline, NIST SP 800-53 Rev 5 Security and Privacy Controls provides relevant control families for system integrity, access control, and configuration management. Where identity-aware hardening is important, NIST Cybersecurity Framework 2.0 supports the broader governance and protection lifecycle around these assets.
Risk and Threat Considerations
Model serialisation attacks are especially dangerous because they turn a routine operational step into an exploitation path. A malicious artifact can deliver code execution, persist through resaving, or alter outputs in ways that are hard to distinguish from ordinary model drift.
Failure mechanism: The attacker supplies or tampers with a serialized artifact that a vulnerable loader interprets as trusted structured content, allowing payload execution or behavior manipulation during deserialization.
Impact: Compromise can extend from a single model load to broader environment access, including stolen data, altered predictions, poisoned workflows, and exposure of credentials or internal systems reachable from the loading process.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | SI-7 — Software, Firmware, and Information Integrity | Protects against malicious or altered model artifacts being trusted at load time |
| CM-5 — Access Restrictions for Change | Limits who can modify conversion pipelines and serialized model artifacts | |
| AC-6 — Least Privilege | Minimizes damage if deserialization code executes in a constrained environment | |
| Recommendation — Validate model artifacts before loading and reject untrusted or tampered files. Restrict who can alter serialization workflows and model-handling code. Run model loading and conversion with the minimum privileges needed. | ||
| CIS Controls v8 | CIS-10 — Data Recovery | Supports recovery and integrity checks after malicious artifact exposure |
| CIS-16 — Application Software Security | Covers secure handling of untrusted software components and libraries used to deserialize models | |
| Recommendation — Store clean copies and verify model artifacts before restoring them. Review serialization libraries and remove unsafe deserialization paths. | ||
Practitioner Guidance
What to watch for: Treat every model file as an untrusted artifact until provenance is known, especially when it arrives from outside your controlled build pipeline. Unexpected loading failures, unusual metadata, or a request to use a nonstandard format are all signals to stop and inspect.
Governance implication: Establish an explicit allowlist for model formats, loaders, and conversion tools, and require review for any workflow that can deserialize objects with execution semantics. The safest stance is to make artifact trust a policy decision, not an assumption made by developers or automation.
Related resources from NHI Mgmt Group
- Who is accountable when an AI-orchestrated attack uses a model provider as part of the kill chain?
- Why do agent harnesses create a larger attack surface than the model itself?
- How do teams know whether an AI response shift is a steering attack or normal model variation?
- How should security teams validate AI-driven attack assumptions before relying on model evaluations?