What breaks when ML systems unpickle untrusted data?

Why unpickling turns a data load into an execution path

Pickle-style deserialisation is dangerous because the loader does not simply rebuild data structures, it may reconstitute objects with behaviours attached. If an ML system accepts attacker-influenced serialized content, the “read” step can become code execution, state mutation, or command-triggering behaviour instead of a safe parse.

That changes the trust model for model artefacts, caches, and internal data handoffs. A file that looks like a model checkpoint can become a payload delivery vehicle, which is why untrusted pickles are treated as a code-execution risk rather than a normal ingestion problem.

In practice, the break point is the assumption that deserialisation is passive. Once object construction, custom reducers, or environment hooks are allowed, the boundary between “data in” and “actions out” disappears, and the loader inherits the privileges of the process performing the load.

Where ML pipelines become exposed

ML pipelines are especially exposed when serialized objects move through training jobs, artifact stores, notebooks, internal APIs, or shared caches. In those paths, a malicious object may be loaded long before anyone inspects the contents, and the resulting blast radius can include data exfiltration, model tampering, lateral movement, or compromise of downstream jobs.

This is also why provenance matters. If the system cannot prove where a model file, dataset snapshot, or feature cache came from, then any convenience-oriented deserialiser becomes part of the attack surface. The security issue is not only the file format, but the assumption that every upstream producer is trustworthy.

In ML environments, deserialisation risk often hides in automation. Scheduled retraining, CI/CD artifact promotion, and internal model serving can all load objects automatically, so a single untrusted payload may be executed repeatedly across environments rather than once in a manual workflow.

Safer handling patterns for serialized ML artefacts

The defensive answer is to avoid untrusted deserialisation paths wherever possible and use formats that separate data from executable object reconstruction. When pickle or equivalent object serialization is unavoidable, the load step should be constrained to tightly trusted sources, isolated execution contexts, and code paths that do not inherit broad filesystem or network authority.

For the broader control picture, Hugging Face Spaces breach 2024 is a useful reminder that AI platforms often expose secrets and tokens through the surrounding workflow, not just through the model file itself. The same logic applies to serialization: once an artefact can influence runtime behaviour, the surrounding environment becomes part of the trust boundary.

Strong practice is to treat model loading as a privileged operation. Verify source, sign or attest artefacts where possible, run deserialisation in a constrained sandbox, and keep loaders away from long-lived credentials, production APIs, and writable deployment paths.

Risk and Threat Considerations

Untrusted unpickling is a classic deserialisation abuse case because the attacker only needs the system to accept a crafted object once. After that, the payload can execute with the permissions of the loading process, which makes the technique attractive in ML pipelines where artefacts are frequently exchanged automatically.

Failure mechanism: The deserialiser reconstructs attacker-controlled object state or invokes code during object creation, so a supposed data load becomes a remote code execution or command-triggering event.

Impact: Compromise can extend beyond the model file to credentials, caches, training infrastructure, and downstream services that trust the poisoned artefact.

Practitioner Guidance

What to prioritise: Treat every deserialisation boundary as an execution boundary when the source is not cryptographically and operationally trusted. If the artefact can arrive from outside the immediate deployment team, assume the loader must be hardened before the pipeline is trusted.

What to verify: Confirm which code paths can load serialized objects automatically, which of them run with production privileges, and whether the loader has any access to secrets, internal APIs, or writable destinations that would amplify a compromise.

Practitioner takeaway: The most important judgement is to stop thinking about untrusted unpickling as malformed input and start treating it as code execution with a data wrapper.

Related resources from NHI Mgmt Group

• What breaks when ML systems deserialize untrusted model files?
• What breaks when security teams treat untrusted input and sensitive data as separate risk categories in agentic systems?
• What breaks when AI systems can reach too many data sources?
• What breaks when AI systems can access data without context-aware controls?