AI model files: what security teams are missing in production

TL;DR: AI model downloads can execute code, hide reverse shells, or carry unsafe companion files, according to Starseer, and scanning alone does not make model files safe. The right control shift is to treat model artefacts like untrusted software supply chain inputs, with provenance, isolation, and format governance.

Editorial analysis by NHI Mgmt Group, based on content published by Starseer: “AI Model Risk 101: The File is the Attack Surface”.

By the numbers:

• JFrog's security research team found over 100 models containing malicious payloads.
• Over 6,000 models on Hugging Face were affected before they were patched.

Key questions

Q: What should security teams do first when they start loading external AI models?

A: Start by restricting model admission to a small set of approved formats, then force every external model through an isolated review pipeline.

Q: Why do AI model files create security risk before inference even begins?

A: Because the load step can be executable.

Q: How can teams tell whether model scanning is actually reducing risk?

A: Look for whether scanning is only a pre-admission signal or a true decision gate.

Practitioner guidance

• Enforce safetensors by default Prefer safetensors for model ingestion and block higher-risk formats in production loading paths unless a risk exception is approved.
• Inspect the full model repository Review weights, configuration files, tokenizer assets, and chat templates together before any model is promoted.
• Treat model artefacts as untrusted code Load external models in isolated environments with restricted network and filesystem access, then promote only after review.

Bottom line: Model files can carry executable behaviour, so the load step itself is part of the attack surface, not just the inference runtime.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →