Open-Weight Model Security Guide: Llama, Mistral, DeepSeek and Qwen

Downloading an open-weight model means importing a large, opaque artefact from the internet and often executing code alongside it, with no vendor standing between you and the risk. Llama, Mistral, DeepSeek and Qwen models are attractive because you control where they run and what data they see, but that control moves the security work onto you: licence terms, file formats that can run code on load, unverified provenance, tampered fine-tunes and adapters, and inference servers that are exposed with weak or partial authentication. This guide covers how to select, verify, load and serve open-weight models safely, and how to measure that the controls work. It is written for platform, ML engineering and security teams running models on their own infrastructure. Poisoning and backdoor detection and AI-BOM practice have their own guides and are linked rather than repeated here.

Key takeaways

• Weights are not the hosted service. Running DeepSeek or Qwen weights locally removes the provider's data handling from the picture, but adds file, runtime and supply chain risks you now own.
• Licences differ more than the word "open" suggests. Qwen open-weight models are Apache 2.0 and DeepSeek-R1 is MIT, while the Llama 4 Community License adds a 700 million monthly active user threshold, naming and attribution rules and an acceptable use policy.
• Pickle files are programs. Prefer safetensors, set use_safetensors=True, keep PyTorch's weights_only=True default, and treat any .bin, .pt or .pkl from an unknown publisher as untrusted code.
• Verify before you trust. Pin repository revisions, check file hashes, scan artefacts, and verify signatures where publishers sign with OpenSSF model signing.
• Inference servers need real authentication. vLLM's --api-key protects only some path prefixes, and llama.cpp advises against exposing its server and RPC backend to untrusted networks.

What changes when you run open weights

With a hosted API, the provider controls the model files, the runtime and much of the abuse monitoring. With open weights, each of those becomes your responsibility. OWASP's LLM03:2025 Supply Chain entry notes that open-access models and fine-tuning methods such as LoRA and PEFT, particularly on platforms like Hugging Face, introduce new supply chain risks, and that models are "binary black boxes" where static inspection offers little assurance.

The distinction matters for DeepSeek in particular. The DeepSeek Database Exposure 2025, where Wiz found an unauthenticated ClickHouse database holding chat logs and API keys, affected DeepSeek's hosted backend, not the downloadable weights. Self-hosting avoids that exposure but not the questions about model behaviour, licence and origin. See the DeepSeek and High-Risk AI Vendor Assessment Guide.

Model licences: Llama, Mistral, DeepSeek and Qwen

Licence terms are a security and governance issue as well as a legal one: OWASP LLM03 lists licensing risk as a supply chain risk, and use-based restrictions travel with derivatives you build. Always read the licence file in the exact repository and revision you deploy, because terms vary between model families and versions.

Record the licence, version and any use restrictions in your AI-BOM so that later fine-tunes inherit the right obligations. See the AI Supply Chain and AI-BOM Guide.

Model file formats: pickle, safetensors and GGUF

The single most important technical control for open weights is refusing formats that execute code. Hugging Face explains that pickle stores a sequence of opcodes and unpickling runs them: the GLOBAL and STACK_GLOBAL opcodes import a callable and REDUCE calls it. PyTorch's own security policy puts it plainly: "PyTorch models are programs".

Loading settings that matter

• PyTorch weights_only: since PyTorch 2.6, torch.load defaults to weights_only=True when no pickle_module is passed. It restricts unpickling to tensors and primitive types and cannot import modules dynamically. PyTorch notes it does not protect against denial of service and that memory corruption may still be possible. Never pass weights_only=False for files from outside your organisation, and alert on TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 in environment configuration.
• Allowlisting: if a checkpoint needs extra classes, use torch.serialization.get_unsafe_globals_in_checkpoint() to see what it references, then allowlist only reviewed items with add_safe_globals or the safe_globals context manager.
• Transformers use_safetensors: setting use_safetensors=True makes loading fail if no safetensors file is present, instead of silently falling back to pickle.
• trust_remote_code: some models ship custom Python modelling files and need trust_remote_code=True, which runs that code. The Transformers security policy says to always verify the modelling files and to set a revision so a later commit cannot change the code you reviewed.

Provenance, scanning and signing

OWASP LLM03 warns that published models have no strong provenance assurances, that model cards offer no guarantee of origin, and that attackers can compromise a supplier account on a model hub or create a look-alike. Its mitigations include using only verifiable sources and integrity checks with signing and file hashes. Layer the following controls.

None of these detect a well-hidden backdoor in safetensors weights; that needs the behavioural testing covered in the AI Data Poisoning and Backdoors Guide. For hub-specific settings, see the Hugging Face Security Guide.

Fine-tunes, LoRA adapters, merges and quantised copies

Most open-weight risk sits not in the base model from Meta, Mistral, DeepSeek or Alibaba's Qwen team, but in the community derivatives built on top. OWASP LLM03 calls out vulnerable LoRA adapters that can be downloaded and applied to deployed models in serving platforms such as vLLM, model merging that can bypass reviews, and conversion services that have been abused to inject malicious code. It also describes attackers publishing fake versions of a removed model under the same name.

• Treat each derivative as a new supplier: a quantised GGUF of Llama from an unknown uploader is a different artefact from Meta's release, with its own provenance questions.
• Control adapter loading: disable runtime adapter downloads in production, and only load adapters from your internal registry after scanning and review.
• Prefer converting yourself: where you need safetensors or GGUF, convert from the official release in your own pipeline rather than trusting a third-party conversion.
• Check the base: a derivative inherits the base model's licence and any use restrictions, and its model card may misstate them.

Securing the inference runtime

Many open-weight compromises will come through the serving layer rather than the weights. The runtime projects' own security documentation is explicit about their assumptions.

• llama.cpp: its security policy says to run untrusted models in an isolated environment such as a container or VM, not to use the RPC backend or llama-server where they must be exposed to untrusted networks, and to confirm hashes of downloaded weights. For multi-tenant hosting it warns about tenant isolation, resource exhaustion and GPU side channels.
• vLLM: its security guide states that --api-key (or VLLM_API_KEY) only protects endpoints under the /v1, /v2, /inference and /cohere prefixes, and that other sensitive endpoints are unauthenticated. Put it behind a reverse proxy or gateway that allowlists routes and enforces authentication.
• Network placement: bind inference servers to private interfaces, restrict ingress to the application tier, and deny outbound internet from model hosts except to your internal artefact registry.
• Isolation for evaluation: load any new or untrusted model first in a sandbox with no credentials, no network egress and a disposable file system. See the Self-Hosted LLM Security Guide.

Where identity fits

Open-weight deployments create non-human identities at both ends. Hub access tokens with write permission can replace a model for everyone who downloads it, and inference API keys let any holder use your GPU capacity and see responses. Scope hub tokens to read-only for consumers, keep write tokens in a vault with short lifetimes, give each calling service its own inference credential, and rotate both. See the API Key Management Guide.

How to implement: a phased approach

1. Inventory (weeks 1 to 2): search code and containers for from_pretrained, torch.load, .gguf and hub URLs to find every model in use, with source, revision, format and licence.
2. Intake gate (weeks 2 to 4): stand up an internal model registry or proxy. Allow only approved publishers, record hashes and licence at intake, run pickle and format scanners, and convert to safetensors where possible.
3. Audit mode (weeks 4 to 6): set use_safetensors=True and block trust_remote_code in a pilot environment, log what fails, and fix or replace those models before enforcing.
4. Enforce (weeks 6 to 8): block direct hub downloads from production networks, require pinned revisions and verified hashes or signatures at deployment, and put inference servers behind an authenticating gateway.
5. Evaluate behaviour: add use-case red teaming before each new model or derivative goes live. See the LLM Security Guide.
6. Roll back: keep the previous approved revision in the registry so a model can be reverted by changing one pinned digest, and keep a documented route to a hosted model if a self-hosted one must be withdrawn.

Decision guidance

• Self-host or hosted API: self-host when data residency, confidentiality or offline operation outweigh the cost of running the controls above; use a hosted API when you cannot staff patching, scanning and runtime security.
• Base release or community derivative: use the official release unless a derivative gives a clear benefit, and then treat it as a new supplier with full intake.
• Small or large organisation: a small team can get most of the benefit by using only official publishers, safetensors, pinned revisions and a private inference endpoint. A large organisation should run a central registry with scanning, signature verification and a model approval board.

Detection and measurement

• Log: model downloads (source, revision, hash), loads that use pickle or trust_remote_code, adapter loads, inference API authentication failures, requests to unauthenticated endpoints, and outbound connections from model hosts.
• Alert: process spawns or network connections from inference or notebook processes immediately after a model load, which is the signature of a pickle payload.

Framework mapping

Common failure patterns

• Loading pickle from anyone: engineers call torch.load with weights_only=False to get an old checkpoint working, turning a model download into code execution.
• Unpinned trust_remote_code: custom modelling code is reviewed once, then the repository changes and the next deployment pulls new code.
• Trusting the uploader name: a popular quantised or merged derivative is used because of its download count, with no check of who produced it or what base it came from.
• Partial authentication on the server: teams set vLLM's --api-key and assume the whole server is protected, leaving other endpoints open.
• Licence drift: a fine-tune based on DeepSeek-V3 or Llama is shipped without the inherited use restrictions, attribution or naming rules.
• Write tokens on developer laptops: long-lived hub tokens with write scope sit in notebooks and repositories where they can leak.

Lessons from incidents

• Hub tokens are model supply chain keys: in the Hugging Face API Tokens Exposed 2023 research, Lasso found 1,681 valid tokens in public repositories, 655 with write permission, and reported write access to Meta's Llama 2 repositories. An attacker with that access could have replaced a widely used model. Hashes and signatures checked at intake are the backstop.
• Malicious models are real: JFrog's February 2024 research found around 100 malicious models on Hugging Face, including a pickle payload that opened a reverse shell on load. Safetensors-only loading would have stopped that class of attack.
• Hosted and open-weight risk are separate: the DeepSeek database exposure shows provider-side risk in a hosted service; self-hosting removes that path but not the need to vet the weights.

Practitioner checklist

• Inventory every open-weight model and derivative with source, revision, format, hash and licence.
• Allow downloads only through an internal registry or proxy from approved publishers.
• Require safetensors (or GGUF for llama.cpp) in production and set use_safetensors=True.
• Keep weights_only=True for torch.load and alert on overrides.
• Block trust_remote_code unless the code is reviewed and the revision pinned.
• Scan artefacts at intake and verify OpenSSF model signatures where publishers provide them.
• Record licence terms and use restrictions in the AI-BOM and carry them to derivatives.
• Put inference servers behind an authenticating gateway on private networks with no internet egress.
• Scope hub tokens to read-only for consumers and vault and rotate write tokens.
• Red team each new model or derivative for your use case before release.

Standards and references

• OWASP: Top 10 for LLM Applications 2025 (LLM03:2025 Supply Chain)
• MITRE ATLAS: AI Supply Chain Compromise
• Hugging Face: Pickle Scanning
• PyTorch: Serialization semantics
• Hugging Face: safetensors
• OpenSSF / Sigstore: Model Transparency (model signing)
• llama.cpp: Security Policy
• Meta: Llama 4 Community License Agreement

Related NHI Mgmt Group resources: AI Supply Chain and AI-BOM Guide · AI Data Poisoning and Backdoors Guide · AI Model Security Guide · Self-Hosted LLM Security Guide · Hugging Face Security Guide