TL;DR: AI workloads running on premises inherit container, Kubernetes, and supply chain risks while adding prompt injection, data leakage, and GPU abuse paths, according to Aqua Security. The security gap is not the model alone but the identity, policy, and runtime controls around it, especially where AI components touch sensitive data and business transactions.
At a glance
What this is: NHI Mgmt Group's analysis, drawing on Aqua Security's blog post: On-premises AI infrastructure needs identity, policy, and runtime guardrails because the model is only one part of the risk surface.
Why it matters: IAM, NHI, and security teams need to govern the surrounding workload, gateway, and runtime layers because AI systems can expose data, overuse compute, and bypass intended boundaries if those controls are weak.
Context
On-premises AI infrastructure is the combined stack that lets a model, its context services, and its runtime controls execute business tasks inside an enterprise environment. The security problem is that this stack inherits container and Kubernetes risk while adding AI-specific failure modes such as prompt injection, data leakage, and GPU abuse.
For IAM and NHI programmes, the central issue is not model quality but control of who and what can invoke the AI path, what data the system can access, and how much compute or tool reach a workload can consume. When those boundaries are weak, AI becomes an access and runtime governance problem, not just an application feature.
Aqua Security frames this as a production issue rather than an experiment issue. That is the correct lens, because the risk appears when AI is embedded in real business applications that already handle sensitive data and transactions.
Key questions
Q: How should security teams govern in-house AI inference workloads?
A: Security teams should govern in-house AI inference workloads as non-human identities with scoped permissions, named ownership, and lifecycle controls. That means inventorying service accounts, separating deployment rights from infrastructure administration, and reviewing credentials whenever models move between environments or vendors. If the workload can call other systems, it needs a defined trust boundary.
Q: Why do prompt injection attacks create governance risk for AI agents?
A: Prompt injection creates governance risk because the model often sits in the control path between text input and tool execution. If attackers can change what the model treats as authoritative, they can influence access decisions, data exposure, or downstream actions without compromising a traditional account. That makes prompt provenance and instruction hierarchy part of AI identity governance.
Q: What signals show that AI runtime controls are failing?
A: Warning signs include unexplained tool usage, access to data outside the expected workflow, repeated policy overrides, and behavioural drift across sessions. If teams cannot trace why an action happened, who authorised it, and what context the system used, runtime controls are too weak to trust.
Q: How do on-premises AI controls differ from ordinary application security controls?
A: On-premises AI controls differ because they must govern a system that both processes data and can influence actions in real time. Traditional app controls focus on access and code, while AI controls must also bound prompts, context, responses, and compute use. That makes runtime governance essential.
Technical breakdown
Why on-premises AI changes the trust boundary
On-premises AI systems are not a single model call. They usually include a client or gateway, a context service, an inference service, and compute resources that may reach into internal data or tools. Each layer expands the trust boundary because the request is no longer just asking for a prediction. It is asking for an action with business context attached. That means authentication, request scoping, and data-minimisation controls must operate across the full path, not only at the model edge. In practice, this is where classic application security and identity controls meet AI runtime behaviour.
Practical implication: map the full AI request path and apply policy at each layer that can read, enrich, or act on the request.
Prompt injection and data leakage in business AI workflows
Prompt injection works because the system treats untrusted input as if it were part of the operating instructions. In a business workflow, that can lead the model to reveal data, ignore policy, or perform actions outside the intended transaction scope. The risk is amplified when the AI system has access to internal context services or downstream tools. Data leakage is therefore not just a model issue. It is a governance failure in how prompts, responses, and attached context are exposed to the runtime. Visibility into prompts and responses becomes a control requirement, not an observability nice-to-have.
Practical implication: treat prompts, responses, and attached context as governed data flows and inspect them for jailbreak and leakage patterns.
Why GPU abuse and container misconfiguration belong in the same control discussion
AI workloads on premises often run inside containers and Kubernetes, so the familiar cloud-native failure modes still apply. A misconfigured cluster, an unscanned image, or an unchecked workload can create blast radius beyond the AI component itself. GPU abuse is especially important because compute is both expensive and shared, so one unconstrained workload can starve others while hiding in normal resource demand. AI security therefore needs runtime guardrails that cover identity, resource limits, workload isolation, and supply chain hygiene together. The stack is only as safe as the weakest operational boundary.
Practical implication: enforce image scanning, workload isolation, and GPU usage limits as part of the same AI security policy set.
Threat narrative
Attacker objective: The attacker aims to make the AI system reveal sensitive data, execute unintended actions, or consume expensive compute at scale.
- Entry occurs when a malicious prompt, document, or application input reaches the AI workflow and is treated as trusted context.
- Escalation follows when the workload can expose internal records, call supporting tools, or expand its scope beyond the intended transaction.
- Impact is the disclosure of sensitive records or the exhaustion of shared GPU resources, turning a business AI feature into an operational incident.
Breaches seen in the wild
- LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
- ShadowRay 2024: Attackers took over internet-exposed Ray AI clusters via the disputed CVE-2023-48022, exposing cloud keys, AI API tokens and GPU compute.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Identity and runtime controls are now inseparable in on-premises AI. The article shows that the model is not the security boundary. The boundary is the chain of client, context, inference, and compute services that turn a prompt into business action. That means IAM teams must treat AI requests as governed transactions, not as isolated model calls. Practitioners should design for control across the full runtime path.
Runtime guardrails matter because AI systems can be driven outside intent by untrusted input. Prompt injection is not just a content issue, it is a policy failure when the system accepts malicious instructions as operational context. The practical lesson is that AI workflows need explicit trust boundaries around prompts, attached data, and tool invocation. Organisations that only secure the model miss the place where misuse actually happens.
On-premises AI security is a cloud-native security problem with an AI-specific consequence layer. Containers, Kubernetes, image hygiene, and supply chain scanning still matter, but they now shape the blast radius of AI misuse. The named concept here is AI runtime blast radius: the amount of data, compute, and downstream action an AI workload can affect before controls intervene. Practitioners should measure that blast radius, not just model performance.
Shared compute turns poor AI governance into a cost and availability problem as well as a security one. GPU abuse is a direct sign that AI workloads need resource governance, not just content filtering. If one workload can monopolise expensive acceleration resources, the incident becomes operational immediately. Security teams should therefore include compute exhaustion in the AI threat model and not treat it as a separate ops concern.
Business context is what makes AI security different from ordinary application hardening. The same system that powers a claims workflow or banking action can be pushed into exposing records if permissions and context are not tightly bounded. That is why identity for AI must include request origin, context scope, and action boundaries. Practitioners should govern AI as a transaction path, not as a static service.
From our research library:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
- Read next: AI Infrastructure Workload Identity Guide
What this signals
AI runtime blast radius: on-premises AI expands the security boundary from the model to the full transaction path, so identity controls must extend to context services, tool use, and compute consumption. Organisations that still govern only the endpoint will miss where misuse actually becomes impact.
A useful operating model is to treat prompts, responses, and attached context as governed data flows. That shifts the programme from model-only review to runtime policy enforcement, which is where AI-specific misuse is most likely to surface.
For practitioners
- Map the full AI transaction path Document the client, gateway, context service, inference layer, and compute resources that participate in each AI request. Assign control owners to every step where data can be added, read, or acted on.
- Restrict context attachment to least necessary data Limit what internal account data, business rules, or records can be attached to an AI request, and review those scopes as tightly as other privileged application permissions.
- Inspect prompts and responses as governed data flows Capture prompt and response content for security review, strip sensitive material where possible, and alert on jailbreak language, policy bypass attempts, or unexpected disclosure patterns.
- Enforce image scanning and workload isolation Scan every AI image and SDK before deployment, and isolate workloads so an untrusted or misconfigured component cannot extend its reach across the environment.
- Set GPU consumption limits and anomaly thresholds Apply resource quotas and alerting for abnormal GPU consumption so one workload cannot exhaust shared capacity or hide abuse behind legitimate model demand.
Key takeaways
- On-premises AI security is really a governance problem across identity, context, runtime, and compute boundaries, not just a model-protection problem.
- The article ties AI risk to familiar cloud-native weaknesses such as container misconfiguration, unscanned images, and uncontrolled GPU usage.
- Practitioners should focus on runtime visibility, least-necessary context, and workload isolation before AI moves deeper into production workflows.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 and MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-04 — Insecure Authentication | The article centers on authenticated AI request paths and control of who can invoke workload actions. |
| NHI-05 — Overprivileged NHI | AI workflows can access internal context and compute beyond what the business task requires. | |
| NHI-08 — Environment Isolation | The article repeatedly ties AI risk to container, Kubernetes, and shared runtime boundaries. | |
| Recommendation — Apply NHI-04 to authenticate AI workload requests before they can read context or trigger tools. Reduce AI workload privilege to the minimum data, tools, and compute scope needed for each transaction. Isolate AI workloads so one compromised or misconfigured environment cannot affect adjacent services. | ||
| MITRE ATT&CK | TA0006;TA0040 — Credential Access; Impact | The breach pattern involves key theft and destructive or resource-consuming impact across AI infrastructure. |
| Recommendation — Map AI infrastructure abuse to credential access and impact tactics when hunting for compromised workloads. | ||
| NIST CSF 2.0 | PR.AA-05 — Access Permissions, Entitlements and Authorizations | The article is fundamentally about who and what may access context, tools, and compute in AI systems. |
| Recommendation — Use PR.AA-05 to govern AI request entitlements, tool access, and runtime authorization boundaries. | ||
Key terms
- AI Runtime Blast Radius: The amount of data, compute, and downstream action an AI workload can affect before controls stop it. In practice, it is the combined impact of permissions, context scope, tool reach, and resource access around the model, not the model’s accuracy or size alone.
- Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
- Context service: A service that gathers internal information, such as account data or business rules, and attaches it to an AI request. It improves usefulness, but it also expands the trust boundary because the model receives more sensitive context than a public assistant would.
- Runtime Guardrail: A control applied while an AI agent is operating, not just during configuration or review. Guardrails can block dangerous tool calls, require approval for sensitive actions, or stop data leakage before it reaches systems or users.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.
Published by the NHIMG editorial team on June 24, 2026.
Updated on October 11, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org