Local deployment shifts control of data and execution to the enterprise, but it also places greater responsibility on the organisation to vet model assets, code paths, and attack surface. Provider-hosted use can simplify operations, but it raises questions about data sharing, service reliability, and infrastructure security. The choice is a trade-off between operational control and dependency risk.
Why This Matters for Security Teams
Trusting an open-weights model locally is not the same as trusting the model itself. It means the enterprise inherits more of the security burden around provenance, artifact integrity, runtime isolation, logging, and update discipline. Using the same model through provider infrastructure shifts some of that burden outward, but it does not remove the need to assess data handling, tenancy boundaries, and the provider’s own control environment. For security teams, the core question is whether the risk is better managed as an internal control problem or an external dependency problem. NIST SP 800-53 Rev 5 Security and Privacy Controls is useful here because it helps teams translate that choice into concrete controls for access, auditing, configuration, and system integrity. Many organisations overestimate the safety of “local” simply because data stays in house, while underestimating the operational exposure created by unmanaged model files, permissive runtimes, and untracked dependency updates. In practice, many security teams encounter model compromise only after a poisoned artifact or exposed inference endpoint has already been deployed, rather than through intentional vetting.
How It Works in Practice
Local deployment gives an organisation direct control over where the model runs, what data it sees, and how outputs are logged or filtered. That is valuable when sensitive prompts, regulated data, or bespoke policies must stay inside the enterprise boundary. But the model package, weights, runtime, and surrounding orchestration become part of the attack surface, which means the security team needs a repeatable process for validation and monitoring. The provider-hosted model shifts those responsibilities toward the vendor, but the enterprise still needs to assess contractual safeguards, data retention, abuse monitoring, and whether prompts or outputs are used for training or telemetry.
A practical review should cover:
- model provenance and whether the weights came from a trusted source;
- hash verification and signed artifact handling before deployment;
- network segmentation and egress controls around inference services;
- prompt, response, and secrets handling in logs and traces;
- provider terms for retention, support access, and incident notification.
For model risk and operational resilience, current guidance suggests treating both patterns as governed production services rather than experiments. The difference is mainly who owns the control plane and who is accountable when something fails. Teams can also use the NIST AI Risk Management Framework to separate governance questions from technical controls, and the EU AI Act becomes relevant when the system is used in regulated decision-making or high-impact contexts. A provider-hosted service can still be risky if the organisation cannot inspect processing paths or constrain sensitive data flow. These controls tend to break down when teams deploy local models into shared infrastructure without hardening the runtime, because the enterprise then loses the very isolation it expected to gain.
Common Variations and Edge Cases
Tighter control often increases operational overhead, requiring organisations to balance security assurance against maintenance burden. That trade-off becomes sharper when open-weights models are fine-tuned internally, because the enterprise is no longer only assessing the base model but also its adapters, embeddings, and retrieval pipeline. Guidance is still evolving on how much assurance is enough for local model vetting, especially in environments that mix internal data with third-party components. There is no universal standard for this yet, so many teams combine supply chain checks, restricted execution, and post-deployment monitoring rather than relying on a single gate.
Two edge cases matter especially:
- Air-gapped or highly regulated environments often prefer local deployment, but only if patching, provenance tracking, and incident response are mature enough to support it.
- Provider-hosted use may be appropriate for lower-risk workloads, but it becomes less attractive when prompts contain secrets, regulated records, or high-value intellectual property.
Where this question intersects with NHI governance, the same logic applies to service accounts, API keys, and agent tooling used to call the model. If those credentials are overprivileged, the security advantage of local hosting is weakened quickly. A local model is not inherently trustworthy just because it is self-hosted, and a hosted model is not inherently unsafe just because it is external. The deciding factor is whether the control set matches the risk profile of the workload, the data, and the surrounding execution environment.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and MITRE ATLAS address the attack surface, NIST AI RMF and NIST CSF 2.0 set the technical controls, and EU AI Act define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | This choice is a model risk and governance decision, not just infrastructure placement. | |
| NIST CSF 2.0 | PR.DS | Both local and hosted use depend on protecting model inputs, outputs, and sensitive data flow. |
| OWASP Agentic AI Top 10 | Agentic or tool-using model setups expand the attack surface through prompts, tools, and execution. | |
| MITRE ATLAS | AML.T0019 | Open-weights models face adversarial ML threats such as poisoning and model tampering. |
| EU AI Act | Hosting choice affects accountability, transparency, and obligations in higher-risk AI use cases. |
Apply data security controls to prompts, outputs, logs, and model artifacts across the deployment path.
Related resources from NHI Mgmt Group
- What is the difference between sandboxing an MCP server and trusting the model to behave safely?
- What is the difference between routing a voice model through an AI gateway and calling it directly from an application?
- What is the difference between a direct model integration and a multi-provider AI gateway?
- What is the difference between routing AI traffic through a gateway and letting each team connect directly to model APIs?
