A self-hosted model is an LLM deployed on infrastructure the organisation controls, such as a private cloud, on-premises server, or local GPU machine. It keeps model traffic inside the environment, but it also shifts responsibility for access control, routing, monitoring, and operational resilience to the team running it.
Expanded Definition
A self-hosted model is an LLM that runs on infrastructure controlled by the organisation, such as a private cloud, on-premises system, or local GPU cluster. In NHI security, the term matters because the model is not just a compute workload; it becomes an internal service with its own identity, network path, secrets, and operational trust boundaries.
Compared with managed model APIs, self-hosting gives teams tighter control over data locality, egress, and runtime policy enforcement. That control is useful when prompts, outputs, or embedded credentials must stay inside a regulated boundary. But the shift also means the organisation owns patching, key handling, logging, scaling, and abuse detection. Guidance varies across vendors on how much isolation is enough, so the security standard is defined more by architecture than by the label itself. The relevant control questions are whether the model can be reached only by approved agents, whether its secrets are rotated, and whether its telemetry is sufficient to detect misuse. For broader identity governance context, see the Ultimate Guide to NHIs and the NIST Cybersecurity Framework 2.0.
The most common misapplication is treating a self-hosted model as inherently trusted, which occurs when teams expose it to internal users or agents without authentication, egress limits, or secret isolation.
Examples and Use Cases
Implementing a self-hosted model rigorously often introduces infrastructure and governance overhead, requiring organisations to weigh tighter data control against added operational responsibility.
- A regulated enterprise runs an internal coding assistant in a private cloud so source code never leaves the tenant, while applying network segmentation and access logs for every agent call.
- A security team hosts an LLM on a local GPU server to analyse incident data, but stores API keys in a secrets manager rather than in environment files or container images.
- An R&D group deploys a model behind an internal gateway so only approved service accounts can invoke it, aligning access with least privilege and Zero Trust.
- A product team uses a self-hosted model for document summarisation, then monitors prompt injection attempts and unusual token consumption as part of runtime detection.
- An organisation with strict residency requirements keeps all inference traffic internal, referencing NIST guidance on architectural control while documenting the model as an internal service asset.
These patterns are easier to govern when the organisation can relate model access to NHI controls already described in the Ultimate Guide to NHIs and operationalise that boundary with the NIST Cybersecurity Framework 2.0.
Why It Matters in NHI Security
Self-hosted models change the security burden rather than removing it. Because the organisation owns the runtime, a compromised service account, overprivileged orchestration token, or misconfigured ingress rule can turn the model into a lateral movement point. NHI risk becomes more visible here because the model often needs credentials to reach tools, data stores, or vector databases, and those credentials can be abused if they are not tightly scoped. NHIMG research shows that 97% of NHIs carry excessive privileges, which is especially relevant when a self-hosted model is given broad internal access without explicit boundaries. The same research also notes that 79% of organisations have experienced secrets leaks, with 77% of those incidents causing tangible damage, underscoring why local hosting does not eliminate credential risk.
Practitioners should treat the model, its tooling, and its supporting services as a governed identity surface, not just an application deployment. That includes access reviews, secret rotation, audit logging, and failure-mode testing for the systems the model can reach. Organisations typically encounter this risk only after a prompt-driven workflow, data exposure, or tool misuse has already occurred, at which point self-hosted model governance becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and OWASP Agentic AI Top 10 address the attack and risk surface, while NIST CSF 2.0, NIST Zero Trust (SP 800-207) and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-02 | Self-hosted models depend on secrets and access paths that fit NHI secret-management controls. |
| NIST CSF 2.0 | PR.AC-4 | Internal model access must enforce permissions for users, agents, and service accounts. |
| NIST Zero Trust (SP 800-207) | AC-4 | A self-hosted model is a protected resource that should never be implicitly trusted. |
| NIST AI RMF | AI RMF frames model deployment risks, including operational and governance impacts. | |
| OWASP Agentic AI Top 10 | A1 | Agentic systems using self-hosted models inherit tool-use and authorization risks. |
Inventory model credentials, rotate them, and restrict each model service to least privilege.
Related resources from NHI Mgmt Group
- Who is accountable when a self-hosted model server is left open to the internet?
- When should organisations choose a self-hosted model over a frontier model?
- What breaks when model routing is handled differently for self-hosted and external LLMs?
- When should organisations keep a model fully self-hosted instead of allowing failover?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on August 27, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
