Securing self-hosted LLMs requires AI-serving-layer controls

At a glance

What this is: Self-hosted LLMs reduce data-sovereignty concerns but expose a serving-layer security gap around privileged runtimes, shared GPU memory, and MCP-connected access paths.

Why it matters: IAM, PAM, and cloud security teams need to treat AI serving engines as governed workload boundaries because the risk sits in runtime access, not just in model selection.

Context

Self-hosting large language models changes the security perimeter rather than removing it. When inference moves from a SaaS API into vLLM, Ollama, or Triton, the serving layer becomes the control point between users, the model, GPU memory, and internal systems. That layer can expose privileges, shared memory, and integration paths that conventional application security often does not model well.

This matters for identity governance because the serving engine is effectively a workload identity boundary. If the engine can invoke tools, access local files, or reach enterprise APIs through MCP, then authentication and authorization must extend beyond users to the machine and service identities that mediate model execution. In most enterprises, that governance model is still immature.

Key questions

Q: What fails when self-hosted LLM serving engines run with elevated privileges?

A: The failure is that the serving engine becomes a privileged execution boundary without enough runtime governance. If it can read local files, call tools, or talk to GPU drivers with broad access, compromise of the engine can translate into data leakage or host-level abuse instead of a contained model issue.

Q: Why does GPU multi-tenancy increase risk in self-hosted AI deployments?

A: Because accelerator memory can hold shared inference state that is not governed like ordinary host memory. When multiple workloads or users share the same GPU pool, a weakness in isolation can expose KV cache fragments, prompt history, or other residual data across tenants.

Q: What do security teams get wrong about MCP-based AI integrations?

A: They often focus on whether a tool is connected and miss the more important question of which tool paths are possible. A safe read action can become unsafe when it feeds an unsafe write action. Teams should model the entire chain, because the attack surface is created by transitions as much as by individual tools.

Q: How should organisations govern self-hosted LLMs alongside IAM and PAM?

A: They should treat the serving engine like a privileged service identity with explicit ownership, narrow permissions, and runtime enforcement. That means governing who can deploy it, what it can access, and which downstream systems it may call on behalf of the model.

Technical breakdown

Why the AI serving layer becomes the real trust boundary

The serving layer is the software stack that receives prompts, runs inference, manages memory, and mediates access to GPUs and downstream tools. In self-hosted deployments, vLLM, Ollama, and Triton often run with elevated privileges so they can talk to drivers, schedule workloads, and support high-throughput inference. That makes the serving engine more than an application container. It becomes the runtime boundary where data exposure, privilege abuse, and host interaction can all intersect. If teams only secure the model artifact, they miss the layer that actually executes the workload.

Practical implication: Treat the serving engine as a privileged workload and apply runtime policy to its process, file, and network behaviour.

GPU multi-tenancy and memory isolation

GPU multi-tenancy improves utilisation, but it also creates a shared memory risk that differs from ordinary CPU or RAM controls. Shared GPU high-bandwidth memory can hold KV cache fragments and other inference-state data that may outlive a single prompt or user session. Traditional cloud workload tools often inspect container and host memory without understanding GPU-specific state, so they can miss leakage paths inside accelerator memory. The governance issue is not only whether a container is compromised, but whether the accelerator pool itself allows one tenant to observe another tenant’s inference state.

Practical implication: Map GPU-sharing patterns to isolation requirements and verify that runtime controls cover accelerator memory, not only host memory.

MCP changes how serving-layer compromise reaches enterprise systems

Model Context Protocol connects AI systems to tools and data sources, which means a serving engine can become a proxy for actions beyond chat completion. If that engine is compromised, the attacker is not limited to model outputs. They may be able to trigger queries, reach internal APIs, or abuse delegated access paths that were intended for the model but not for an adversary. This is where AI governance and identity governance meet. The issue is delegated authority at runtime, not just prompt quality or content filtering.

Practical implication: Constrain tool and API delegation so the serving layer can only use explicitly approved identities, scopes, and destinations.

Threat narrative

Attacker objective: The attacker wants to use the AI serving layer as a privileged foothold for data exposure, model theft, or broader environment access.

1. Entry occurs through a vulnerable or weakly governed self-hosted serving engine such as vLLM, Ollama, or Triton, often via an exposed API or unsafe inference endpoint.
2. Credential or privilege abuse follows when the serving process uses elevated access to reach GPU drivers, local files, or internal tools through MCP-connected integrations.
3. Impact can include cross-tenant data leakage, model-weight theft, or lateral movement from the AI workload into the underlying host or enterprise systems.

Breaches seen in the wild

• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.
• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Self-hosted LLM serving layers are now workload identity boundaries, not just model runtimes. The practical security decision is no longer whether the model is open-weight or SaaS-hosted. It is whether the engine that serves the model has a governed identity, scoped privileges, and runtime enforcement that matches the sensitivity of the data it can reach. Practitioners should treat serving-layer governance as part of IAM and PAM, not as an afterthought to application deployment.

GPU memory isolation is the missing control plane in many AI security programmes. Traditional cloud and container controls are built around CPU-centric assumptions, while accelerator memory can hold inference state that may be exposed across tenants or sessions. That gap creates a distinct AI infrastructure risk that is easy to overlook until the serving layer is treated as an asset class in its own right. Practitioners should extend control design to the accelerator boundary, not only the container boundary.

MCP expands the blast radius of a compromised serving engine. Once a model-serving process can act as a proxy to tools and enterprise data, identity governance has to follow delegated action paths, not just user authentication paths. This is where the named concept of serving-layer delegation risk matters: the weak point is the runtime authority granted to the engine itself. Practitioners should bound that authority as tightly as any privileged service account.

Code-to-cloud thinking is insufficient for AI infrastructure. The article’s real lesson is that AI needs governance over behavioural execution, not just image scanning and Kubernetes posture checks. This aligns with broader runtime control thinking in NIST-CSF and OWASP-NHI where privilege, identity scope, and execution context matter as much as deployment hygiene. Practitioners should shift from model deployment assurance to runtime containment and auditability.

Self-hosting AI shifts responsibility, it does not eliminate the control problem. Enterprises gain sovereignty, but they also assume direct accountability for the security of serving engines, GPU access, and downstream integrations. The category will increasingly reward teams that can prove least privilege, isolation, and traceable access at the serving layer. Practitioners should expect AI governance to converge with machine identity governance over time.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

Serving-layer delegation risk: the important governance shift is that AI systems now act through privileged runtime paths, not only through model outputs. That means security teams need to manage tool access, service identity scope, and runtime containment together, rather than treating them as separate programmes.

Self-hosted AI is likely to widen the gap between deployment speed and governance maturity unless organisations can prove that the serving layer, GPU memory, and MCP connections are all under policy control. AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That trend reinforces why runtime authority, not just model provenance, is becoming the control point.

For practitioners

• Define the serving layer as a governed asset Inventory vLLM, Ollama, Triton, and similar inference engines as privileged workloads with named owners, approved use cases, and explicit trust boundaries.
• Constrain GPU and container privileges Remove unnecessary root access, isolate GPU access paths, and verify that runtime policy can block unexpected file, process, and network behaviour.
• Limit MCP and tool delegation Authorize only the specific enterprise APIs, databases, and tool endpoints that the serving engine must reach, then deny all other delegated paths by default.
• Test for GPU-memory leakage paths Validate whether shared accelerator memory, KV cache state, and inference artifacts can be observed across tenants or sessions in your deployment model.
• Instrument audit trails for AI runtime actions Log model access, GPU allocation, privileged process behaviour, and outbound calls so security teams can investigate serving-layer abuse without relying on generic container logs.

Key takeaways

• Self-hosted LLMs move the security problem from model hosting to the serving layer, where privileged execution, shared GPU memory, and delegated tool access create new exposure points.
• AI infrastructure is already leaking faster than core model providers, which means runtime governance is becoming a first-order issue for identity, access, and workload security teams.
• The strongest control response is to govern the serving engine as a privileged workload with tight runtime policy, bounded delegation, and auditability across GPU, process, and API access.

Key terms

• AI Serving Layer: The AI serving layer is the runtime service that accepts prompts, routes requests, and returns model output in production. It matters because this layer often holds network reachability, authentication logic, and access to downstream data or compute, making it a privileged non-human identity boundary rather than a simple application wrapper.
• GPU Multi-Tenancy: GPU multi-tenancy is the practice of sharing a single physical GPU across multiple workloads or users. It improves efficiency, but it also creates isolation challenges because memory, cache, and driver interactions can expose data between tenants if controls are weak or visibility is incomplete.
• Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.
• Serving-Layer Delegation Risk: Serving-layer delegation risk is the possibility that a model-serving process uses its authorized access to perform actions beyond what the organisation intended. It arises when runtime identities, tool permissions, and network reach are broader than the business task requires.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, and secrets management in the context of modern infrastructure risk. It helps security and identity teams connect machine access decisions to the controls their programmes must enforce.