A self-hosted LLM is a language model run on infrastructure controlled by the organisation rather than a third-party SaaS platform. The security burden shifts to the operator, who must manage exposure, authentication, segmentation, logging, and safe integration with tools and data sources.
Expanded Definition
A self-hosted LLM is not just a model deployed on a private server. It is an operating choice that places model runtime, prompt handling, inference traffic, and surrounding integrations under the organisation’s own control. That control can reduce third-party data exposure, but it also means the organisation inherits responsibility for hardening the environment, governing access, and monitoring model use end to end.
In security terms, the distinction is less about where the model binary sits and more about who controls the trust boundary. A self-hosted deployment may still call external APIs, index internal documents, or expose tool interfaces to an agent. That makes the model part of a broader application security problem, not a standalone AI experiment. Guidance in the NIST AI Risk Management Framework and the NIST AI 600-1 Generative AI Profile is useful here because both emphasise governance, measurement, and operational controls around AI systems.
Definitions vary across vendors on whether a model hosted in a managed private cloud still counts as self-hosted. In practice, the useful question is whether the organisation controls the security configuration, telemetry, and data path. The most common misapplication is treating any internally reachable model as self-hosted when the provider still controls key telemetry, update, or routing functions.
Examples and Use Cases
Implementing self-hosted LLMs rigorously often introduces infrastructure and governance overhead, requiring organisations to weigh tighter data control against greater operational responsibility.
- An internal knowledge assistant is deployed inside a segmented environment so sensitive documents do not leave the organisation’s network boundary.
- A development team runs a model on private infrastructure to support code review, while restricting outbound internet access and logging all prompts for audit.
- A security operations team connects a self-hosted model to ticketing and detection tools, then limits tool permissions to reduce the impact of unsafe agent behaviour, a concern reflected in the OWASP Agentic AI Top 10.
- A regulated business uses a self-hosted model for customer support drafts so that personal data stays under its own retention and access policies.
- An organisation evaluates prompt injection and tool abuse scenarios using the CSA MAESTRO agentic AI threat modeling framework before enabling agent actions.
Self-hosting can also support more deliberate use of retrieval, caching, and model versioning, but each integration increases the chance that a weakness in one layer becomes a model-level incident.
Why It Matters for Security Teams
Security teams need to understand self-hosted LLMs because the risk profile changes the moment the organisation assumes direct operational ownership. Identity and access controls become central, especially when the model can be invoked by employees, service accounts, or agents with tool access. If secrets, API keys, or retrieval connectors are exposed too broadly, the model becomes an amplifier for privilege misuse rather than a productivity control.
This is where NHI governance starts to matter. A self-hosted LLM may depend on non-human identities for orchestration, data retrieval, or downstream actions, and those identities need the same discipline as human accounts. The model should be treated as part of an application estate that requires segmentation, authentication, logging, and continuous review, not as a special case exempt from standard controls. The OWASP Top 10 for Agentic Applications 2026 and MITRE ATLAS adversarial AI threat matrix both reinforce the need to think about misuse paths, not just model accuracy.
Organisations typically encounter the real cost of self-hosting only after a prompt leak, tool misuse, or unexpected data exposure, at which point the term becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | Frames governance and lifecycle risk management for AI systems like self-hosted LLMs. | |
| NIST AI 600-1 | Profiles generative AI risk management practices relevant to hosted model operations. | |
| OWASP Agentic AI Top 10 | Covers agentic app risks that arise when self-hosted LLMs can call tools or act autonomously. | |
| NIST CSF 2.0 | PR.AC | Access control and protective safeguards apply directly to self-hosted model environments. |
| OWASP Non-Human Identity Top 10 | Self-hosted LLMs often rely on non-human identities for orchestration and data access. |
Enforce least privilege, segmentation, and authenticated access around model endpoints and dependencies.
Related resources from NHI Mgmt Group
- How should security teams protect self-hosted AI runtimes from memory disclosure?
- How should security teams choose between managed and self-hosted CIAM?
- How do organisations decide between self-hosted open-weight models and hosted APIs?
- What do teams get wrong about choosing a self-hosted authentication framework?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on August 2, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
