Shadow AI is exposing company data to LLMs through ungoverned access

At a glance

What this is: Pomerium argues that Shadow AI is creating ungoverned LLM data leakage as employees send sensitive company context to external models outside approved controls.

Why it matters: IAM, NHI and platform teams need to treat LLM access as a governed identity path because once data leaves through unmanaged AI usage, auditability and least privilege break down.

Context

Shadow AI is the uncontrolled use of external LLMs and AI assistants with company data, and it creates an identity governance problem as much as a data protection problem. The core failure is not simply that employees use AI tools, but that they do so outside approved authentication, authorisation and logging paths.

In practice, the article describes a familiar pattern: users move faster than policy, and the security team only notices after sensitive context is already inside a model. That makes the relevant control plane an identity-aware gateway, not a prohibition notice, because the governance gap is the absence of a bound, auditable access path for AI context loading.

Key questions

Q: How should security teams govern employee use of public LLMs to reduce confidentiality risk without blocking useful AI work?

A: Security teams should treat public LLM use as a governed data handling problem, not just a technology choice. The practical baseline is clear policy, explicit approval paths for sensitive data, and training that explains what can never be shared. The survey shows many organisations already have protocols, yet employees still enter sensitive information, so enforcement and awareness must both be in place.

Q: Why do shadow AI tools create risk for IAM teams?

A: Shadow AI complicates IAM because the real subject is often not just the employee, but the AI service, token, or connector acting on the employee's behalf. That expands the identity surface without formal provisioning, certification, or offboarding, which means access governance becomes incomplete even when human login controls are strong.

Q: What breaks when employees use LLMs through personal accounts and devices?

A: The enterprise loses the audit trail and the enforcement point. Personal access paths let users send code, documents or customer data into external models without central policy checks, so security cannot reliably constrain scope, inspect content or reconstruct exposure after the fact.

Q: Should organisations prioritise logging or content filtering first for AI governance?

A: Start with logging and identity binding, then add filtering. If you cannot prove who accessed what, the organisation cannot investigate misuse or demonstrate control. Content filtering is important, but without identity-aware logging it only reduces risk at the margins and still leaves the path ungoverned.

Technical breakdown

Why ungoverned LLM access creates a new identity path

When an employee pastes source code, customer records or internal documents into an external LLM, the data no longer moves through the company’s sanctioned application path. Instead, the LLM session becomes a shadow access channel that bypasses existing identity, logging and data controls. The problem is not just exfiltration. It is that the access decision happens in an unmanaged context, often from a personal device or account, with no enterprise policy boundary around what may be shared or retrieved. Practical implication: treat LLM usage as a governed access surface, not as informal collaboration.

Practical implication: Route all approved AI usage through a controlled access path that can enforce identity, scope and logging before context leaves the enterprise.

What an identity-aware gateway changes for LLM context loading

An identity-aware gateway sits between the user or agent and the model provider, enforcing strong authentication, scoped authorisation, audit logging and optional content filtering. That makes the gateway functionally similar to an identity-aware proxy for internal applications, but tuned for prompts, files and retrieved context. The key architectural point is that the gateway governs both requests and responses, so it can inspect what data is being sent to the model and what comes back. Practical implication: integrate the gateway with the existing identity provider and policy engine instead of creating a separate AI trust stack.

Practical implication: Bind LLM access to the same identity source and policy logic used for other enterprise applications, then log each context load end to end.

Why prompt injection turns uncontrolled AI use into an abuse surface

Once an LLM receives internal context, the security boundary no longer ends at disclosure. The model can be manipulated through prompt injection, where hostile or crafted input changes what the assistant reveals or does. That is why every uncontrolled integration point becomes an attack surface, especially when the model can reach internal tools or downstream workflows. In governance terms, the risk is not only leakage, but delegated misuse of the same access that the human or service account originally held. Practical implication: apply content controls and rate limits at the point where AI requests touch enterprise data.

Practical implication: Add scanning, filtering and rate limiting at the access point so malicious prompts cannot turn approved context into unauthorized action.

Threat narrative

Attacker objective: Obtain sensitive company context, or abuse the model’s access path, to expose data and bypass enterprise controls.

1. Entry occurs when employees move company data into public LLMs through chat interfaces, IDE plugins or personal accounts outside managed enterprise pathways.
2. Credential or context exposure happens when sensitive code, customer data, tokens or internal documents are pasted into the model, giving the external service access to material it should not hold.
3. Escalation follows when unmanaged integrations, copied context or prompt injection cause the model to reveal more data or act on privileged information beyond the original user intent.
4. Impact is data leakage, loss of auditability and the possibility that downstream systems or workflows consume tainted AI output without visible governance.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Shadow AI is now an identity governance problem, not just a data loss problem. The article’s central point is that employees are already creating unsanctioned AI access paths that sit outside standard IAM, logging and policy controls. Once context leaves through those channels, traditional governance has no clean way to recover visibility or enforce scope. Practitioners should treat AI usage as a governed access domain, not an exception to policy.

Ungoverned AI usage creates a context-leakage model that conventional DLP alone cannot absorb. DLP can inspect content, but it does not by itself establish who is authorised to send which context to which model under which conditions. The article points toward a different control pattern: bound the path, bind it to identity, and make every retrieval auditable. The practical conclusion is that AI governance must start at access issuance, not after the prompt is already in flight.

Identity-aware gateways are becoming the runtime control plane for Shadow AI. The article effectively describes a zero-trust pattern for LLM context loading, where authentication, authorisation, logging and filtering happen before data reaches the model. That shifts the centre of gravity from user education to enforceable policy. Practitioners should assume that if the secure path is optional, it will be bypassed.

Prompt injection turns ordinary AI adoption into delegated abuse unless the model’s scope is constrained. The security issue is not only what users paste into LLMs, but what the model can be induced to reveal or trigger once it has access. This is where AI agent governance, NHI controls and data protection intersect. The field should now treat model context as a controlled privilege boundary, not a convenience feature.

Ephemeral trust debt is the right way to think about Shadow AI exposure. Every unmanaged AI interaction accumulates a debt of missing logs, missing approvals and missing scoping that cannot be reconstructed after the fact. That makes retroactive review weak by design. The practitioner implication is simple: govern the access path before the organisation scales usage further.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Read next: AI Agent Observability, Audit and Incident Response Guide

What this signals

Shadow AI is forcing IAM teams to move governance upstream. Policies that only address approved applications will miss the fastest-growing path, which is employee-led access to external LLMs outside enterprise controls. The operational response is to govern context loading itself, not just the applications around it.

Identity-aware gateways are becoming the control point for AI adoption. They let organisations decide who may send what data to which model, under which conditions, while keeping logs and policy enforcement attached to the request path. That is the difference between manageable AI usage and a permanently invisible shadow channel.

The surrounding AI infrastructure is leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026. That pattern tells practitioners the exposure is expanding around the model layer itself, so governance has to follow the data path rather than the brand of the model.

For practitioners

• Enforce an identity-aware AI gateway Require all approved LLM traffic to pass through a gateway that validates user identity, applies policy and records each context load before any data reaches an external model.
• Bind model access to enterprise authorisation Map each AI request to the same permissions that govern the originating user or service account, so the model only receives data already within that subject’s approved scope.
• Log every prompt and retrieval event Capture who requested the context, what data was retrieved, which model received it and whether the request came from a user, tool or agent.
• Strip sensitive material before submission Filter API keys, customer identifiers and other sensitive fields at the gateway so accidental paste events do not push protected data into public models.
• Make the secure path the default path Block direct access to unmanaged models once the governed gateway is stable, because optional controls are the first thing users bypass when they need speed.

Key takeaways

• Shadow AI is an access-governance failure as much as a data-protection issue, because unmanaged LLM use bypasses authentication, authorisation and auditability.
• The scale of the problem is visible in the day-to-day behaviour of employees who paste sensitive context into external models to get work done faster.
• The practical control shift is to make approved AI use flow through an identity-aware gateway with logging, scoping and content checks.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Identity-aware gateway: An identity-aware gateway is a control layer that verifies identity, applies authorization policy, and records audit data before forwarding a request. For AI systems, it becomes the enforcement point between the orchestrator and internal services, preventing the model from becoming the authority for access decisions.
• Context loading: Context loading is the act of supplying documents, code, or records to an LLM so it can answer with relevant internal information. It matters because the transfer itself can expose secrets, personal data, or intellectual property if it is not governed and audited.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.