Shadow AI is exposing company data to LLMs through ungoverned access

At a glance

What this is: Pomerium argues that employees are already sharing sensitive company data with public LLMs and that most organisations lack visibility or control over those flows.

Why it matters: This matters because IAM, NHI, and human identity teams need to govern who can send which data to which model, with auditability and least privilege, before shadow usage becomes normalised.

👉 Read Pomerium's analysis of how employees are already dumping company data into LLMs

Context

Shadow AI is the uncontrolled use of public or semi-public LLMs with internal data, often outside approved IT workflows. In identity terms, the gap is not just data loss. It is that the organisation cannot reliably answer which human, service, or workload identity sent which information to which model, under what permissions, or with what logging.

Pomerium's central claim is that bans do not solve the problem because users route around them. That creates a governance problem across human IAM, NHI context loading, and emerging agentic workflows, where the same access assumptions that protect internal apps no longer hold once data is copied into an external model.

The practical question for practitioners is no longer whether employees will use LLMs, but whether the organisation can place those interactions behind identity-aware policy, audit logging, and data filtering. The article's starting position is typical, not exceptional, and it reflects a pattern many security teams are already seeing.

Key questions

Q: How should security teams govern employee use of public LLMs?

A: Security teams should treat prompt submission as a governed access event. The practical control set is identity-aware routing, policy enforcement, logging, and pre-send filtering so sensitive data never leaves the enterprise boundary without oversight. If users can bypass the approved path, the governance model has already failed.

Q: Why do bans on public AI tools usually fail?

A: Bans fail because employees route around them when the sanctioned path is slower or less useful. That pushes usage onto personal devices and unmanaged accounts, which removes audit trails and weakens data protection. The real control objective is not prohibition. It is making the secure workflow easier than the insecure one.

Q: What breaks when secrets are pasted into LLM prompts?

A: What breaks is the organisation's ability to control where the secret goes next. Even if the user had legitimate access to the source system, the prompt can expose credentials to retention, logging, or reuse conditions the enterprise does not control. That makes prompt filtering and DLP mandatory for high-risk data.

Q: Who is accountable for LLM data leakage in an organisation?

A: Accountability should sit with the teams that own identity policy, data classification, and approved AI access paths. In practice, that means IAM, security, and data governance must share responsibility for the control plane. If no team owns the path from user identity to model submission, leakage will remain invisible.

Technical breakdown

Why unsanctioned LLM use creates an identity and data boundary problem

The core technical issue is context exfiltration. Users do not merely query an LLM, they often paste code, tickets, documents, and credentials into the prompt, which moves sensitive material outside the organisation's enforcement boundary. Once that data is in an external model, the enterprise loses control over retention, reuse, and downstream exposure. In identity terms, the request may begin with a human user, but the risk is amplified by the absence of policy enforcement at the point of transfer. Practical implication: treat prompt submission as a governed data movement event, not a harmless productivity action.

Practical implication: classify LLM prompt flows as data transfers and enforce controls before content leaves the trust boundary.

Identity-aware gateways for LLM context loading

Pomerium's architecture describes an access proxy between the user or agent and the model. That gateway authenticates the caller, checks authorisation, logs the request, and can apply filtering or rate limits before context reaches the LLM. This is the same control pattern as an identity-aware proxy, but adapted to model inputs and outputs rather than internal web apps. It matters because the model is not the trust anchor. The trust anchor is the identity behind the request and the policy attached to it. Practical implication: put policy enforcement in front of the model, not after the data has already been submitted.

Practical implication: insert an identity-aware proxy or gateway in front of approved LLM usage and make it the default path.

Audit logging, DLP, and least-privilege scopes for AI access

The article makes clear that the minimal viable control set includes strong authentication, scoped authorisation, comprehensive audit logging, content safety checks, and throttling. In practice, that means the organisation must know which user or service requested access, which data was retrieved, which model received it, and whether secrets or PII were stripped. This is important because generic access controls do not solve model-specific leakage. A user may be entitled to the source document yet still be unfit to send it to an external model. Practical implication: align LLM access scopes with the sensitivity of the data being forwarded, not just the user's baseline permissions.

Practical implication: log every model-bound context transfer and apply DLP before data reaches any external LLM.

NHI Mgmt Group analysis

Shadow AI is now an identity governance problem, not only a data governance problem. The article shows employees bypassing IT controls because the productivity benefit is immediate and the approved path is too slow or too restrictive. That means governance must track the identity behind every model interaction, not just the destination service. Practitioners should treat unmanaged LLM use as a control-plane failure across human access, logging, and policy enforcement.

Context loading is the new blast-radius multiplier for human identities. The risky action is not abstract AI usage, but the deliberate transfer of sensitive context into an external model session. Once code, customer records, or credentials are pasted into that session, the exposure scope expands beyond what traditional application access reviews were designed to see. Security teams should recognise that the blast radius is defined by the data moved, not by the application allowed.

Least privilege for LLMs fails when access is evaluated only at the source system. A user can be entitled to the underlying document and still create risk by forwarding it into a model with broader retention or training implications. That is a policy mismatch, not a simple permissions issue. IAM and data teams need a shared control model that evaluates both source access and destination risk before the transfer happens.

Identity-aware gateways are becoming the enforcement point for AI usage at scale. The article's recommended pattern aligns with the wider zero trust view that access must be authenticated, scoped, logged, and continuously constrained. For NHIs and emerging agents, the same pattern will be required for model calls, tool calls, and retrieval flows. Practitioners should expect the gateway layer to become the operational boundary for AI governance.

Risk ownership will shift from policy statements to observable enforcement. The article is clear that bans drive usage underground and remove audit trails. That means the field has to stop measuring policy intent and start measuring whether the approved path is actually the easiest path. The practical conclusion is that identity governance for AI will be judged by control adoption, not by policy publication.

From our research:

• 38% of secrets incidents in collaboration and project management tools like Slack, Jira, and Confluence are classified as highly critical or urgent, according to The State of Secrets Sprawl 2025.
• Around 100,000 valid secrets were found in public Docker images, with ENV instructions alone accounting for 65% of all secret leaks in containers.
• For the operational response, see OWASP NHI Top 10 for the adjacent agentic risk model that often sits beside Shadow AI leakage.

What this signals

Shadow AI will force identity teams to move from policy ownership to path ownership. The organisation that only writes usage guidance but does not govern the route from user to model will keep losing visibility. With 15% of commit authors having leaked at least one secret in their contribution history, per The State of Secrets Sprawl 2025, the pattern is already established: people disclose sensitive data when the workflow is easier than the control.

Context transfer is becoming a first-class control object. That means logging, filtering, and approval need to sit on the exact payload going into the model, not on the application environment in general. Practitioners should expect AI gateways to be evaluated alongside zero trust patterns such as the NIST AI Risk Management Framework and identity-aware access layers.

For programmes that already manage workload identity, the next step is to extend those controls to AI-assisted workflows and retrieval paths. The same discipline that governs tokens, service accounts, and secrets now needs to govern prompt-bound data movement, because the failure mode is no longer credential theft alone but credential disclosure by ordinary users.

For practitioners

• Map LLM data flows by identity Inventory which human, service, and workload identities are sending data to public or approved models. Record the data classes involved, the source systems, and whether the flow is authenticated and logged end to end.
• Force model traffic through a governed gateway Route all approved LLM usage through an identity-aware proxy that can enforce authorisation, apply policy, and block direct access paths that bypass visibility.
• Apply pre-send filtering to prompts and context Strip secrets, customer identifiers, and regulated personal data before content reaches any external model. Use DLP rules that operate on the exact payload being submitted, not just on stored data.
• Log every context transfer for auditability Capture who requested the context, what was included, which model received it, and whether the request was blocked, modified, or allowed. Keep these records in the same telemetry stack used for identity governance review.
• Make the secure path the easiest path Reduce latency, remove duplicate approvals, and publish a supported workflow for approved LLM use so employees do not default to personal devices or unmanaged accounts.

Key takeaways

• Shadow AI turns routine employee productivity into a governed identity and data movement problem that standard policy bans do not solve.
• The key failure is not just tool misuse. It is the lack of visibility, logging, and pre-send control over what leaves the enterprise and reaches an external model.
• Identity-aware gateways, prompt filtering, and auditable model access are the controls that determine whether AI adoption stays manageable or becomes a leakage channel.

Key terms

• Shadow AI: Shadow AI is the use of AI tools or models without organisational visibility, approval, or governance. In practice, it often means employees move sensitive data into external systems through personal accounts, unmanaged devices, or unreviewed workflows that bypass policy and logging.
• Identity-aware gateway: An identity-aware gateway is a policy enforcement point that authenticates the caller, checks authorisation, and mediates access before a request reaches an application or model. For AI use, it becomes the control surface for context loading, logging, and data filtering.
• Context loading: Context loading is the act of supplying documents, code, or records to an LLM so it can answer with relevant internal information. It matters because the transfer itself can expose secrets, personal data, or intellectual property if it is not governed and audited.
• Prompt filtering: Prompt filtering is the inspection and modification of model input before it is sent to an AI system. It is used to strip secrets, personal data, or other sensitive material from requests so the organisation can reduce leakage without blocking all AI use.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by Pomerium: Your Employees Are Already Dumping Company Data to LLMs (Here’s What To Do About It). Read the original.