Securing self-hosted LLMs requires AI-serving-layer controls

At a glance

What this is: This analysis argues that self-hosted LLMs move the security burden from SaaS providers into the enterprise serving layer, where GPU sharing, privileged containers, and AI-specific APIs create new exposure paths.

Why it matters: It matters because IAM, PAM, and NHI teams increasingly need to govern AI serving engines as sensitive runtime identities, not just secure the application and cloud stack around them.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, sharing sensitive data, and revealing access credentials.

👉 Read AccuKnox's analysis of securing self-hosted LLM serving layers

Context

Self-hosted LLMs change the threat model because the enterprise, not the model provider, now owns the serving layer, GPU access, and runtime security decisions. That matters for AI governance and for identity teams because the systems that execute inference increasingly behave like privileged non-human workloads with access to data, tools, and internal APIs.

vLLM, Ollama, and NVIDIA Triton are not just deployment choices. They become part of the trust boundary for AI workloads, which means shared GPU memory, privileged containers, and unmanaged API endpoints can create the same governance gaps that IAM and PAM teams see in other high-value machine identities.

The article's starting position is typical for organisations moving from SaaS AI to self-hosted AI: they regain control over data location, but they often underinvest in runtime controls and workload identity governance.

Key questions

Q: How should security teams govern self-hosted LLM serving layers?

A: Treat the serving engine as a privileged workload identity, not a generic app container. Give it only the network paths, data sources, and GPU permissions it genuinely needs, then enforce continuous runtime monitoring so unexpected API calls or file access are blocked before the session can be abused.

Q: Why do self-hosted LLMs increase data leakage risk?

A: They move inference, memory handling, and access control into the enterprise environment, where shared GPUs and privileged runtime paths can expose cached prompts, conversation fragments, or model artefacts. If the platform cannot isolate those paths, confidentiality depends on assumptions rather than enforced controls.

Q: What breaks when AI serving engines run with excessive privileges?

A: Excess privilege lets a compromised inference service reach host resources, GPU drivers, local files, or internal networks that should be off limits. That turns a model-serving issue into a broader workload compromise and makes lateral movement much easier.

Q: How should teams decide whether MCP access is safe enough to allow?

A: Teams should allow MCP access only when the agent or server can be bounded with explicit scopes, revocable credentials, and traceable client registration. If the integration depends on static secrets, shared keys, or opaque delegation, the access model is too durable for reliable governance and should be redesigned before production use.

Technical breakdown

Why self-hosted LLM serving layers become a new trust boundary

In a self-hosted LLM stack, the serving engine mediates between application traffic, model memory, GPU drivers, and internal services. That makes it more than an inference process, because it can hold conversation context, process privileged requests, and initiate outbound connections. If the engine is exposed through an unauthenticated API or overly broad runtime permissions, the model host becomes a control plane for data access rather than a simple application component.

Practical implication: Treat the serving engine as a privileged workload and bound its permissions, network paths, and API exposure accordingly.

GPU multi-tenancy and memory leakage in shared inference environments

GPUs are often shared to improve utilisation, but shared High Bandwidth Memory and cached inference state introduce cross-tenant exposure risks. KV cache contents can contain fragments of previous conversations, while memory reuse and weak isolation can create side channels between workloads. Traditional endpoint and cloud workload tools usually see CPU, container, or node activity, but not the GPU memory state that actually matters for AI inference confidentiality.

Practical implication: Use GPU-aware isolation and monitoring so shared inference capacity does not become a cross-tenant data leakage path.

MCP expands the blast radius of an exposed serving engine

When a serving engine can speak Model Context Protocol, it is no longer just returning text. It can also trigger tool use, query internal services, and act as a proxy into enterprise data sources. That shifts the risk from model misuse alone to delegated access abuse, where a compromised inference layer can become a route into databases, APIs, and workflows that were never meant to be directly reachable from the model runtime.

Practical implication: Constrain MCP-connected tools with explicit identity and policy controls, not just network segmentation.

Threat narrative

Attacker objective: The attacker wants to turn the AI serving layer into a trusted relay for data theft, model extraction, or pivoting into internal systems.

1. Entry occurs through an exposed or weakly protected AI serving endpoint, such as an unauthenticated completions API or a vulnerable inference service.
2. Escalation follows when the serving engine's privileged access to GPU drivers, host resources, or internal tools is abused to access memory, files, or adjacent systems.
3. Impact occurs when attackers exfiltrate conversation fragments, model weights, or internal data, or use the service as a proxy for broader compromise.

NHI Mgmt Group analysis

Self-hosted AI creates a workload identity problem, not just a hosting problem. Once inference shifts in-house, the serving engine becomes a privileged non-human workload with permissions, trust assumptions, and lifecycle obligations. That changes how IAM and PAM teams should think about AI infrastructure, because the risk is no longer limited to model quality or container hardening. Practitioners need to treat the serving layer as an identity-governed asset with explicit ownership and revocation paths.

GPU memory is now part of the sensitive data perimeter. Traditional data controls rarely account for inference-state exposure, yet KV cache and shared memory can carry meaningful fragments of user interaction. This is a GPU memory leakage boundary, and it is the specific concept practitioners should use when evaluating self-hosted LLM risk. If the platform cannot see or isolate that boundary, it cannot credibly claim confidentiality control.

MCP turns AI serving into delegated access, which raises the bar for governance. The moment a model can reach tools and enterprise data sources, the question changes from whether the model is accurate to whether the model runtime is authorised to act. That makes identity governance, least privilege, and session-bound policy enforcement central to agentic AI and self-hosted LLM security.

Runtime controls matter more than perimeter assumptions in this class of deployments. The article correctly points to visibility gaps in generic cloud and container tooling, but the deeper lesson is that AI infrastructure needs continuous control enforcement, not just pre-deployment scanning. Security teams should expect the market to move toward AI-specific runtime governance because static controls do not model inference-time behaviour.

What this signals

AI infrastructure is converging with identity governance. As self-hosted LLMs absorb more data access and tool invocation, the control plane starts to resemble an NHI problem with GPU semantics. That means programmes should expect workload identity, policy enforcement, and runtime auditability to become baseline requirements for any serious AI deployment.

Runtime blind spots will matter more than model selection. Choosing between vLLM, Ollama, or Triton is less important than proving that the deployment can enforce least privilege, observe GPU interactions, and stop unexpected behaviour. The operational signal to watch is whether your security stack can explain what the serving engine touched, not just where the container ran.

Delegated access is the next governance frontier. Once self-hosted AI uses MCP or similar tool protocols, the model runtime becomes a broker for sensitive enterprise actions. That is why teams should align self-hosted AI governance with [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) guidance and agentic AI control patterns, not just cloud hardening checklists.

For practitioners

• Define the AI serving layer as a governed trust boundary Assign ownership for vLLM, Ollama, and Triton deployments, then document which data, APIs, and GPU resources they are allowed to touch. Keep that boundary visible in IAM, PAM, and asset inventories.
• Restrict privileged GPU runtime paths Reduce container and host privileges needed for GPU driver interaction, and block unexpected process calls from the serving engine to local files or outbound destinations.
• Add GPU-aware monitoring and isolation Monitor memory, process, and driver interactions that standard CWPP tools miss, especially where multiple inference workloads share the same hardware.
• Control MCP-connected tools with explicit policy Require identity, authorization, and task scoping for every tool the model can invoke, especially when the serving engine can query databases or internal APIs.
• Test for unauthenticated inference exposure Check whether completions or inference endpoints can be reached without strong authentication, because exposed model APIs often become the first path to memory and data abuse.

Key takeaways

• Self-hosted LLMs reduce SaaS dependence, but they shift confidentiality and access risk into the AI serving layer.
• Shared GPU resources, privileged runtime paths, and MCP-enabled tooling create exposure that generic cloud controls do not reliably see.
• Identity governance for AI workloads now means continuous control of the serving engine, not just policy around the model or application.

Key terms

• AI Serving Layer: The AI serving layer is the runtime service that accepts prompts, routes requests, and returns model output in production. It matters because this layer often holds network reachability, authentication logic, and access to downstream data or compute, making it a privileged non-human identity boundary rather than a simple application wrapper.
• GPU Multi-Tenancy: GPU multi-tenancy is the practice of sharing a single physical GPU across multiple workloads or users. It improves efficiency, but it also creates isolation challenges because memory, cache, and driver interactions can expose data between tenants if controls are weak or visibility is incomplete.
• Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.
• AI Security Posture Management: A governance approach for discovering and tracking AI assets such as models, agents, datasets, vector stores, and related infrastructure. It becomes useful only when inventory is connected to runtime exposure and the identity that can actually reach the data.

What's in the full article

AccuKnox's full article covers the operational detail this post intentionally leaves for the source:

• Deployment-oriented guidance for securing vLLM, Ollama, and NVIDIA Triton in production inference environments
• Specific runtime visibility and enforcement concepts tied to GPU drivers, eBPF, and privileged containers
• A deeper explanation of how the Model Context Protocol changes the AI serving layer trust model
• Practical examples of inline remediation for anomalous AI-serving behaviour

👉 The full AccuKnox article covers vLLM, Ollama, Triton, GPU visibility, and runtime enforcement detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, workload identity, secrets management, and agentic AI identity. It helps practitioners connect identity controls to the runtime systems that increasingly power enterprise AI.