Model serialization attacks are attacks that abuse the saved or transmitted form of a machine learning model. They target model files, checkpoints or deserialisation routines so that loading the model can trigger malicious code, altered behaviour or hidden payloads. Defences include signed artefacts, strict format validation and deserialising only from trusted sources.
What model serialization attacks are
Model serialization attacks abuse the saved or transmitted form of a machine learning model. The risk sits in the model artifact itself, not only in the training pipeline or the inference layer, because loading a maliciously crafted file or checkpoint can execute code, alter behaviour, or activate a hidden payload before the model ever produces an output.
These attacks matter because serialization is often treated as a routine engineering step, yet it becomes a trust boundary when models move between teams, environments, or vendors. Once a loader accepts untrusted content, the model file can function like an execution vector rather than a passive data object.
Why serialization is a security boundary
Serialized models usually contain more than weights in practice: object graphs, metadata, tensors, hooks, and loader-specific structures. If the deserialisation routine interprets that structure unsafely, the attack surface shifts from model quality to parser safety, artifact integrity, and execution control. Anthropic’s first AI-orchestrated cyber espionage campaign report is a reminder that modern attacks increasingly chain legitimate automation and trusted tooling, which is exactly why loading paths deserve the same scrutiny as runtime prompts or API calls.
In security terms, serialization is dangerous because it can hide malicious instructions inside a file that appears to be just a model checkpoint. That makes integrity assurance, provenance, and parser hardening part of the control surface, not optional hygiene.
How organisations assess exposure
Exposure review starts with inventory: where model artifacts are stored, which systems load them, and whether the files came from internal training, a third-party repository, a partner, or an external exchange. Teams should also check whether models are fetched automatically at build time or runtime, because that expands the path by which a poisoned artifact can enter production.
Internet-facing model hubs, shared storage, CI/CD artifact stores, notebook environments, and research sandboxes are common points where trust breaks down. The practical question is not only whether a model is known good, but whether every system that can deserialize it is constrained to trusted sources and validated formats.
Defenders should also review vendor notes, framework advisories, and loader documentation for unsafe deserialisation modes, because the same model format may be safe in one path and dangerous in another. CISA cyber threat advisories remain useful for tracking active exploitation patterns and adjacent supply-chain or software integrity issues that can affect model delivery paths.
Defence and recovery after suspicious loading
The strongest preventive control is to avoid deserialising untrusted artifacts at all. Where that is not possible, organisations should use signed models, strict format validation, allowlisted loaders, and isolated environments that can inspect artifacts before they reach production inference. These measures reduce the chance that a model file can carry executable payloads or tamper with inference behaviour.
If a model has already been loaded from an untrusted source, treat it as a compromise event. Rebuild or replace the artifact from a trusted source, verify downstream systems that consumed it, and review whether any adjacent code, plugins, or callbacks executed during loading. The same discipline applies after a patch, because a malicious artifact may have persisted through caches, registries, or deployment pipelines.
Post-incident review should include what the loader accepted, what integrity checks were missing, and whether any derived artifacts, caches, or exported versions inherited the compromise. If model files are signed or hashed, revalidate those checks after remediation rather than assuming the environment is clean.
Identity and access implications when models are shared
Model serialization attacks often spread through ordinary collaboration workflows, so access control around artifact repositories matters. Teams should limit who can publish, replace, or approve model files, and they should prefer workflows that separate model authorship from deployment permission. NIST AI Risk Management Framework helps frame that broader governance question, while NIST SP 800-53 Rev 5 Security and Privacy Controls supports the underlying control logic for integrity, configuration management, and controlled access.
When model artifacts are exchanged across teams or vendors, trust in the file is really trust in the identity of the source and in the process that produced it. That means provenance, approval, and change control are part of the defensive story even when the primary exploit lands through a technical parser flaw rather than a credential compromise.
Risk and Threat Considerations
Model serialization attacks can turn a normal artifact-loading step into remote code execution, stealthy behavioural manipulation, or long-lived compromise of an AI pipeline. The danger is amplified when organisations reuse checkpoints, accept community models, or load artifacts automatically in build and deployment workflows.
Failure mechanism: The attacker places malicious content in a serialized model object or abuses an unsafe deserialiser so that loading the file executes code, alters weights or triggers hidden behaviour before validation can occur.
Impact: A compromised model can corrupt outputs, plant persistence in surrounding systems, leak data through downstream processing, or undermine trust in every inference made from that artifact until it is replaced and reviewed.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST AI RMF, NIST SP 800-53 Rev 5 and OWASP ASVS set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | Govern | AI model integrity and provenance directly affect AI risk governance. |
| Recommendation — Establish model provenance and loading controls before approving deployment. | ||
| NIST SP 800-53 Rev 5 | SI-7 — Software, Firmware, and Information Integrity | Serialized model integrity depends on validating trusted, untampered artifacts. |
| CM-5 — Access Restrictions for Change | Model publication and replacement need controlled change authority. | |
| AC-6 — Least Privilege | Deserialisation and artifact management should run with minimal permissions. | |
| Recommendation — Verify model artifacts with integrity checks before deserialising them. Restrict who can publish or replace model artifacts in production stores. Limit loader and pipeline permissions to the minimum required. | ||
| OWASP ASVS | V15 — Secure Coding and Architecture | Unsafe object loading and trust-boundary failures are code-architecture concerns. |
| Recommendation — Design model-loading paths to avoid unsafe deserialisation patterns. | ||
| MITRE ATT&CK | T1027 — Obfuscated Files or Information | Malicious model payloads can be hidden inside apparently benign serialized artifacts. |
| Recommendation — Hunt for concealed payloads in suspicious model files and checkpoints. | ||
Practitioner Guidance
What to watch for: Treat any model artifact from an untrusted source, an unexpected format change, or a loader path that permits arbitrary object deserialisation as a security event. The practical test is whether the pipeline can inspect the model without executing arbitrary code.
Governance implication: Assign clear ownership for model provenance, approval, and revocation, especially when artifacts move between data science, platform, and security teams. A model is safe only when its source, format, and loading path are all governed.
Related resources from NHI Mgmt Group
Deepen Your Knowledge
Free weekly newsletter
Subscribe to the NHI & AI Identity Journal
The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.
Bonus 33% off our NHI Course when you subscribe.
Reviewed and updated by the NHIMG editorial team on November 2, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org