Shadow AI exposes sensitive code and data in public chatbots

At a glance

What this is: This is an analysis of Shadow AI, where employees use public LLMs and other unapproved AI tools to expose sensitive code and data.

Why it matters: It matters because IAM, NHI, and human access controls all fail if organisations cannot see where data is going, who is sending it, and which approved tools have quietly added AI features.

👉 Read JumpCloud's analysis of Shadow AI, public LLMs, and data exposure

Context

Shadow AI is the unauthorised use of generative AI tools in ways that move sensitive data outside approved governance paths. The core problem is not software sprawl alone, but the data transaction created when employees paste code, customer information, or regulated content into public models.

That changes the identity and access question for IAM teams. The issue now spans human users, approved applications that have acquired AI features, and unmanaged browser-based access paths that bypass corporate controls entirely.

Key questions

Q: How should security teams prevent sensitive data from being pasted into public AI tools?

A: Security teams should combine policy, discovery, and enforcement. That means identifying AI destinations on managed devices, binding approved AI use to corporate identity and device trust, and blocking or flagging submissions that include regulated or proprietary data. User guidance alone is not enough when the tool is instantly reachable in a browser.

Q: Why do approved SaaS applications still create Shadow AI risk?

A: Approved SaaS can become Shadow AI when vendors add generative features that change where data is processed or sent. The app may still be sanctioned, but the AI-enabled workflow may not be. Teams need to review feature changes as trust-boundary changes, not just as product updates.

Q: What breaks when employees use personal accounts for AI tools?

A: Corporate SSO, access policy, and usage logging lose effectiveness when employees use personal AI accounts. The organisation no longer has the same identity signal or entitlement control, so it cannot reliably govern what content is shared or where it is retained. That makes personal account use a governance gap, not just a convenience issue.

Q: How do Zero Trust controls help with Shadow AI?

A: Zero Trust helps by making AI use conditional on identity verification, device trust, and approved destinations. It does not eliminate all risk, but it forces each session to pass a policy check before sensitive data can move. That is the practical control model for AI use in enterprise environments.

Technical breakdown

How Shadow AI creates a data exfiltration path

Shadow AI is dangerous because the user interaction itself becomes the leakage event. When an employee pastes source code, customer data, or internal context into a public LLM, the transfer may occur outside traditional file controls and network logging, especially if the tool is accessed through a browser or personal account. In effect, the model becomes a new destination for sensitive content, and the organisation loses visibility into what was submitted, by whom, and under which policy boundary. This is not classic malware-driven exfiltration. It is voluntary, workflow-driven disclosure through an ungoverned identity path.

Practical implication: monitor AI destinations and browser-mediated submissions, not just installed software.

Why approved apps can still become Shadow AI

Shadow AI is not limited to obvious third-party chatbots. Approved SaaS platforms can add generative features later, shifting data flows without a fresh procurement or security review. That means an application once considered safe may begin sending prompts, documents, or metadata to a third-party model that was never included in the original trust decision. The risk is compounded when employees use personal accounts, because corporate SSO and entitlement controls no longer govern the session. In practical terms, the control boundary moves from the app name to the actual AI-enabled workflow.

Practical implication: re-evaluate trusted applications whenever AI features are added or account context changes.

Zero Trust for AI use is really trust-boundary enforcement

Applying Zero Trust to Shadow AI means treating every AI interaction as untrusted until the identity, device, and destination are verified. In this context, Zero Trust is not about blocking all AI use. It is about ensuring that access to approved AI resources is conditional on user identity, device trust, and policy enforcement, while unapproved endpoints are denied or flagged. That framing matters because the real failure mode is uncontrolled data movement, not simply unapproved software. If the organisation cannot constrain where sensitive content goes, it cannot claim meaningful AI governance.

Practical implication: bind AI access to identity, device trust, and approved destinations before allowing use.

NHI Mgmt Group analysis

Shadow AI is a governance failure of data movement, not just application sprawl. Traditional Shadow IT controls were built to find unauthorised software and reduce procurement risk. Shadow AI shifts the centre of gravity to what users submit into models, which means the leak path is created by the interaction itself. The implication is that identity governance must extend to prompt and content boundaries, not just application approval.

Browser-mediated AI use creates a control blind spot that classic endpoint and network logging often miss. Many AI tools now operate through browser extensions or web sessions layered over approved apps, which means sensitive data can leave through the same user path that looks legitimate to existing monitoring. That undermines assumptions behind file transfer controls and application allowlisting. Practitioners should treat browser sessions as governance surfaces, not just user interfaces.

Approved software can become unapproved AI infrastructure overnight. The article’s most important warning is that feature creep turns a trusted SaaS application into a data-sharing route when generative functionality is added later. That means the original risk decision ages quickly unless organisations track AI feature activation as a change event. Security teams should reclassify trust whenever the data path changes, even if the application name does not.

Shadow AI exposes the gap between human intent and enforceable policy. Employees often use public or personal AI tools to work faster, but policy language alone does not stop them when the experience is frictionless. The real challenge is aligning acceptable-use rules with technical controls that prevent sensitive content from leaving the enterprise boundary. Programmes that cannot enforce the rule will not contain the behaviour.

Shadow AI should be governed as part of broader identity and access lifecycle control. Unapproved AI use is not a separate security category. It intersects with user identity, SaaS discovery, access certification, and data governance, which is why siloed controls fail. The practitioner takeaway is clear: map AI use to the same lifecycle governance discipline applied to human access and sanctioned machine identities.

From our research:

• 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job, according to The 2026 Infrastructure Identity Survey.
• Only 44% of organisations have implemented any policies to manage their AI agents, despite 92% agreeing that governing AI agents is critical to enterprise security.
• That gap points to the next control question in OWASP NHI Top 10: how to constrain identity and data exposure before AI workflows are normalised.

What this signals

Shadow AI will increasingly sit inside sanctioned tools, not outside them. The operational challenge for security teams is no longer only discovery of rogue applications. It is continuous reclassification of SaaS workflows as vendors add AI features, because the trust boundary shifts even when the application name does not. Teams that do not track those feature changes will miss the real data path.

With 67% of organisations still relying heavily on static credentials despite the risks they pose to agentic AI deployments, per The 2026 Infrastructure Identity Survey, identity controls are already too brittle for the pace of AI adoption. That brittleness becomes visible first in Shadow AI, where access, device context, and data handling all diverge at once.

For practitioners

• Discover AI access paths across managed devices Scan browser activity, network requests, and extension usage to identify public chatbot access and other AI endpoints that bypass software installation checks.
• Reclassify trusted SaaS when AI features appear Treat newly enabled generative features as a change in data handling. Re-run security review, privacy assessment, and approval checks whenever an existing application begins sending content to a model.
• Bind AI use to identity and device trust Allow sanctioned AI resources only when the session is tied to corporate identity, managed device posture, and explicit policy controls. Block or flag access from personal accounts and unmanaged endpoints.
• Prevent sensitive data from reaching public models Use DLP, conditional access, and usage policy to stop code, customer data, and regulated content from being entered into public LLMs or browser plugins that lack enterprise data controls.

Key takeaways

• Shadow AI is a data-governance problem disguised as software misuse, because the risk is what employees submit to public models.
• The evidence points to a structural control gap, with AI access often exceeding the privileges organisations would grant for the same human task.
• Practical governance requires discovery of AI endpoints, identity-bound access rules, and policy controls that stop sensitive content from leaving the enterprise boundary.

Key terms

• Shadow AI: Shadow AI is the use of generative AI tools outside approved governance channels. In practice, it often means employees using public chatbots, browser plugins, or personal accounts in ways that move sensitive data beyond enterprise control and make identity-based enforcement difficult.
• Data Exfiltration Path: A data exfiltration path is the route sensitive information takes when it leaves an organisation’s controlled environment. In Shadow AI cases, the path may be a prompt field, browser extension, or personal account rather than a file transfer or network event.
• Trust Boundary: A trust boundary is the line where an organisation’s security assumptions stop applying. For Shadow AI, the boundary shifts when a sanctioned SaaS app adds AI features or when users move from corporate identity to personal accounts.
• Conditional Access: Conditional access is a policy control that allows or blocks a session based on context such as identity, device posture, or location. In Shadow AI governance, it becomes a way to restrict approved AI resources while denying unsanctioned or unmanaged access.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by JumpCloud: Shadow AI and the risks of employees feeding sensitive data into public chatbots. Read the original.