Model Signing vs Model Scanning: Proving Integrity vs Finding Malice

Teams that adopt model signing often assume a signed model is a safe model, and teams that run a model scanner often assume a clean scan means the model is trustworthy; neither assumption holds. Signing answers "did these exact bytes come from the identity I expected, unchanged?", while scanning answers "do these bytes contain something that will run code or behave badly when I load them?". A model can pass one test and fail the other: a poisoned checkpoint signed by a legitimate but compromised pipeline verifies perfectly, and a clean scan says nothing about who swapped the file last night. This guide compares the two controls on threat model, architecture, failure modes and cost, then gives decision criteria by model source and a combined intake design. It is written for ML platform engineers, application security teams and architects who own model intake, and it builds on the Model Signing and OpenSSF Model Signing Guide and the Model Supply Chain Guide rather than repeating their tooling detail.

Key takeaways

• They answer different questions. Signing proves integrity and origin; scanning looks for malicious or unsafe content. MITRE ATLAS says successful verification "does not establish that the artifact is free of malicious code, poisoned data, backdoors, or harmful instructions" and tells you to combine it with scanning and model validation.
• Scanning is strongest on code paths, weakest on weights. Static scanners read pickle opcodes and flag dangerous imports, but they cannot see a behavioural backdoor trained into the numbers, and parser differences between scanner and loader have produced real bypasses.
• Signing is only as good as the pinned identity. Verifying "any valid signature" lets an attacker sign a malicious model with their own account; verification must pin the expected identity and issuer, or key, per model.
• Source decides the weighting. For models you train, signing at the pipeline carries most of the value; for third-party models, scanning and format policy carry the first gate, and you sign the approved copy yourself.
• Safetensors shrinks the scanning problem. A code-free weight format removes the deserialisation attack path, which lets scanning focus on the remaining code-bearing artefacts and lets signing protect a simpler file set.

The short answer

Use both, but put each where it earns its keep. Model signing belongs at every point where a model crosses a trust boundary you control: training output, registry upload, transformation (quantisation, merging, conversion) and deployment. Model scanning belongs at the point where a model first enters your estate from someone else, and again whenever scanner rules change. If you can only fund one control this quarter, choose by where your models come from: mostly in-house training means signing first; mostly public hub downloads means scanning plus a safetensors-only rule first. Neither control detects a model that was poisoned before it was signed or that carries a trained-in backdoor; that needs evaluation and red teaming, covered in the AI Data Poisoning and Backdoors Guide.

Side by side

Model signing in depth

Threat model

Signing defends against an attacker who can change model files between the producer and the consumer but cannot use the producer's signing identity. That covers a stolen hub write token used to replace weights, a tampered object in a shared bucket, a man-in-the-middle on an internal mirror, an insider who swaps a checkpoint after approval, and plain operational error where the wrong version is deployed. In the Hugging Face API Tokens Exposed 2023 research, Lasso reported 655 exposed tokens with write permission across organisations whose models are widely downloaded; a consumer that verifies signatures against a pinned producer identity turns a silent model swap with such a token into a failed load. ATLAS records the attacker side as AML.T0010.003 (Supply Chain Compromise: Model Repository) and AML.T0115.001 (Publish Poisoned AI Artifacts: Models).

Architecture in words

The OpenSSF model signing project (sigstore/model-transparency) hashes every file in a model directory into a manifest of path and digest pairs, wraps it in an in-toto statement inside a DSSE envelope, and signs it into a Sigstore bundle. Verification checks the signature, then recomputes every file hash and compares. The same format supports Sigstore keyless signing, plain keys, certificates and PKCS #11 hardware keys, and keyless signing events land in a public transparency log that the producer should monitor for signatures it did not make. The OpenSSF launch post for v1.0 (4 April 2025) recommends signing during training and verifying at hub upload, at selection for deployment, and when a model becomes input to another training run. The Model Signing and OpenSSF Model Signing Guide covers identity choice and commands.

Failure modes

• Unpinned verification: checking that a signature is valid without checking whose it is.
• Signing the poison: a compromised training environment or poisoned dataset produces a faithfully signed bad model. Signing at training time is only as trustworthy as the infrastructure holding the signing identity.
• Stolen signer: a leaked CI token or private key produces valid signatures until someone notices; without log monitoring nobody does.
• Coverage gaps: verifying weights but loading a tokeniser, adapter or config fetched separately. ATLAS AML.M0014 tells you to verify the complete set of files, including supporting files that can change behaviour.
• Re-signing without re-checking: an intake job that re-signs an external model with the organisation's identity launders whatever was inside it into "trusted".

Controls that make signing work

• Pinned policy: keep the expected identity and issuer (or key) per model in a version-controlled policy file read by the verifier.
• Protected signing identity: sign only from CI workload identity scoped to one repository and workflow, never from laptops. See the CI/CD Pipeline Identity Security Guide.
• Log monitoring: watch the transparency log or internal signing service for your identities.
• Timestamps: for key and certificate signing, require an RFC 3161 timestamp at verify time where policy needs to prove when a signature was made.

Model scanning in depth

Threat model

Scanning defends against an attacker who publishes or plants a model that runs code when loaded, or that contains operators which read and write files or open connections. The classic case is a pickle-based checkpoint: Hugging Face's pickle documentation explains that the GLOBAL and STACK_GLOBAL opcodes import objects and REDUCE calls them, so a crafted file can execute arbitrary code on load. JFrog reported in February 2024 that it had found around 100 malicious models on Hugging Face, mostly PyTorch with Keras close behind, including one that opened a reverse shell when loaded. ATLAS tracks this as AML.T0011.000 (User Execution: Unsafe AI Artifacts). Note that the attacker in this model usually holds a perfectly valid identity: their own hub account, which is why signing alone does not help.

Architecture in words

Scanners read the serialised file as data without executing it. Hugging Face lists every import in pickled files using pickletools.genops and highlights suspicious ones, and separately runs ClamAV on files under 2 GB at each commit, while stating that antivirus does not cover pickle exploits and that its own pickle scan is "not 100% foolproof". Open source scanners such as ModelScan and picklescan (listed alphabetically) run in your own pipeline: ModelScan reads files byte by byte for unsafe code signatures across pickle, H5 and SavedModel formats and ranks findings CRITICAL to LOW, and both tools return exit codes that a CI job can gate on.

Failure modes

• Parser divergence: the scanner reads the file differently from the loader. JFrog disclosed three picklescan bypasses fixed in version 0.0.31 (CVE-2025-10155 file extension, CVE-2025-10156 ZIP CRC errors, CVE-2025-10157 subclass imports), each exploiting a gap between how the scanner parsed a file and how PyTorch loaded it.
• Deny-list lag: rules recognise known dangerous imports; new import chains pass until rules catch up.
• Fail-open on errors: a scan that crashes or skips an unsupported file is treated as clean. ModelScan distinguishes "error" (exit 2) and "no supported files" (exit 3) from "clean" (exit 0) for exactly this reason.
• Weights-level backdoors: no file scanner detects behaviour trained into parameters. The Sleeper Agents research (arXiv 2401.05566) showed backdoor behaviour can persist through supervised fine-tuning, reinforcement learning and adversarial training.
• Scan-then-swap: a model scanned at intake and stored without a digest can be replaced afterwards; the scan result no longer describes the file you load.

Controls that make scanning work

• Fail closed: treat any non-zero, non-findings result (errors, skipped files) as a block. JFrog's advice is that a model whose scan fails for any reason should be removed and not distributed internally.
• Format policy first: allow safetensors and Keras v3 with safe mode; route pickle formats through isolated conversion. A Trail of Bits audit of safetensors, commissioned by Hugging Face with EleutherAI and Stability AI (May 2023), found no critical flaw leading to arbitrary code execution.
• Loader hardening as a second layer: PyTorch 2.6 and later default torch.load to weights_only=True, and TORCH_FORCE_WEIGHTS_ONLY_LOAD=1 forces it for every call; PyTorch states this does not guard against denial of service.
• Isolated load test: load candidate models in a sandbox with no network and no credentials, watching for process and connection activity.

Is a signed model safe?

No. A signature means the bytes match what a named identity produced. Hugging Face says the same of its signed commits: they guarantee the origin of a file, not that it is safe. Three common cases show the gap.

Decision criteria

• Provenance risk versus content risk: if your realistic worry is someone changing a model you already trust, invest in signing. If it is someone handing you a model you have never seen, invest in scanning and format policy.
• Supply source: in-house training and fine-tuning favour signing at the pipeline; public hubs favour scanning at intake; vendor deliveries need both (verify their signature if they provide one, scan, then re-sign).
• Format mix: an estate already on safetensors gets less from pickle scanning and more from signing; an estate full of legacy .bin, .pt and joblib files needs scanning and conversion before signing adds much.
• Blast radius: models backing agents with tool access, or loaded on GPU hosts holding cloud credentials, justify both controls in enforce mode. Code execution on load in those hosts is a credential theft event.
• Audit and customer evidence: signatures and transparency log entries give verifiable, portable evidence; scan reports show due diligence but are point-in-time.
• Team size: a small team gets most value from two rules: production loads only safetensors from an internal mirror, and the mirror only accepts files with a digest recorded at intake. Large organisations add keyless signing from CI, a hub proxy with scanning and an exception workflow.

A combined intake design

The order matters: scan before you sign, because signing after a scan freezes the scanned state, while signing before a scan just certifies whatever arrived. Run the steps below as one pipeline.

1. Acquire: download by pinned commit hash into an isolated runner with no credentials. If the producer publishes an OpenSSF model signature, verify it against the producer identity you have recorded for that model.
2. Format gate: accept safetensors directly; convert pickle checkpoints in a network-less job; reject remote code unless an exception is approved.
3. Scan: run the scanner on every file, including tokenisers, configs and adapters; fail closed on errors and skipped files.
4. Evaluate: for models backing high-impact use cases, run behavioural tests and red teaming for the intended use, as OWASP LLM03:2025 recommends for third-party models.
5. Record: write digests, source, revision, scan result and approver into the AI-BOM. See the AI Supply Chain and AI-BOM Guide.
6. Sign the approved copy: sign with the intake pipeline's identity and publish to the registry of record.
7. Verify at load: deployment gates and training jobs verify against the intake identity; nothing loads from public hubs directly.
8. Rescan and re-sign on change: rescan when rules update; every transformation gets a new signature from the transforming pipeline.

Roll out in audit mode first: log verification failures and scan findings for a few weeks without blocking, fix process gaps (manual uploads, notebooks), then enforce for internet-facing and agent-backing models before the rest. Rollback is per service, with a named approver able to deploy a previously verified digest rather than turning enforcement off globally.

Worked scenario

A retail bank's data science team wants to deploy an open-weight model from a public hub to summarise customer complaints, plus a fine-tuned variant built in-house. The model repository offers both a pickle .bin and a safetensors copy, and no producer signature.

• Intake: the platform team pins the commit, takes only the safetensors weights and the JSON tokeniser and configs, and blocks the .bin. The scanner runs on all files and reports clean; one config file is flagged as unsupported and the job fails closed until the file type is added to the scanner's settings.
• Evaluation: because the model will see customer data, the team runs a short red-team exercise for data leakage and instruction following before approval.
• Signing: the intake pipeline signs the approved directory with its CI identity and publishes it to the internal registry.
• Fine-tune: the training job verifies the base model signature before starting, and the training pipeline signs its output with a separate identity.
• Deploy: the serving platform's init step verifies the model against the training identity; the pod does not start on failure.
• Outcome: six months later the public repository is updated by a compromised maintainer account. Nothing changes in production, because loads are pinned to the internally signed copy; the change shows up only when someone requests an upgrade and the new commit goes back through intake.

Where identity fits

Both controls rest on non-human identities. The signing identity is a workload identity or key whose compromise produces trusted malicious models, and the hub tokens that publish and pull models are the credentials attackers use to swap files. Scope publishing tokens to one repository with write access only where needed, separate training, intake and release signing identities, and give each an owner and a rotation or revocation plan. See the Hugging Face Security Guide for token scoping on one widely used hub.

Detection and measurement

• Log: every verify result with model, digest, expected and actual identity; every scan result with tool version and rule version; every model load path in serving and training.
• Alert on: verification failures in enforce mode, signatures from your identities you did not expect, scans that error or skip files, child processes or outbound connections from model-loading processes, and loads from outside the registry.

Framework mapping

Common failure patterns

• Treating a green tick as a safety verdict: a hub badge, a signed commit or a valid signature reported as "model approved" without a scan or evaluation.
• Scanning only the weights file: adapters, tokenisers and modelling code fetched separately go unscanned and unsigned.
• Scanner errors read as clean: CI gates that only check for "findings" let crashed or skipped scans through, the exact gap the picklescan CVEs exploited.
• Re-signing external models without intake checks: the organisation's signature turns an unknown file into an apparently trusted one.
• Write tokens left broad: publisher tokens with organisation-wide write access, as in the Lasso research, make replacement easy and push all the defensive burden onto consumers' verification.

Practitioner checklist

• Inventory every model source and every load point, and record the format of each production model.
• Adopt a code-free format policy for production weights, with conversion of legacy pickle files in an isolated job.
• Run a scanner at intake on every file in the model directory and fail closed on errors and skipped files.
• Pin model references to commit hashes and record SHA-256 digests in the AI-BOM.
• Sign the approved copy at intake and every transformed variant, from CI identities scoped per pipeline.
• Verify against a pinned identity and issuer, or key, at registry admission, training start and deployment.
• Monitor the transparency log or signing service for unexpected signatures from your identities.
• Run behavioural evaluation or red teaming for high-impact models, since neither control detects trained-in backdoors.
• Rescan mirrored models when scanner rules update and review format exceptions quarterly.

Standards and references

• MITRE ATLAS: AML.M0014 Verify AI Artifacts
• OWASP: LLM03:2025 Supply Chain
• OpenSSF and Sigstore: model-transparency (model signing)
• OpenSSF: Launch of Model Signing v1.0
• Hugging Face: Pickle Scanning
• PyTorch: Serialization semantics (weights_only)
• JFrog: Three zero-day vulnerabilities in PickleScan
• arXiv: Sleeper Agents, Training Deceptive LLMs that Persist Through Safety Training

Related NHI Mgmt Group resources: Model Signing and OpenSSF Model Signing Guide · Model Supply Chain Guide · AI Model Scanning and Supply Chain · OWASP LLM Top 10 LLM03 Supply Chain Guide · Sigstore Guide