Sensitive enterprise data is flowing into AI tools at scale

At a glance

What this is: Cyberhaven says sensitive enterprise data is now flowing into AI tools at scale, driven by personal-account usage and limited security visibility.

Why it matters: This matters because AI usage is becoming a data exfiltration and governance problem, especially where enterprise identity, retention, and access controls do not extend to the tools employees actually use.

By the numbers:

• 39.7 percent of all AI interactions involve sensitive data.
• 32.3 percent of ChatGPT usage happens through personal accounts.
• 58.2 percent of Claude usage happens through personal accounts.
• 60.9 percent of Perplexity usage happens through personal accounts.

Context

AI usage has moved from a specialist function into ordinary enterprise work, which means data governance now has to follow employees into the tools they actually use. The problem is not just whether AI is allowed, but whether sensitive data is being copied, pasted, uploaded, or retained outside enterprise controls.

Where personal accounts dominate, SSO, centralized logging, retention policy, and model-training restrictions become ineffective. That creates an identity and data governance gap at the same time: the organisation loses visibility into who used the tool, what data entered it, and how long the information may persist.

The article’s starting position is increasingly typical. Most enterprises are not dealing with a fringe use case anymore; they are dealing with everyday AI usage that has outpaced control design.

Key questions

Q: What breaks when employees use personal and corporate AI accounts interchangeably?

A: Interchangeable account use breaks attribution, policy enforcement, and data handling assumptions. Security teams can no longer tell whether a prompt came from a governed enterprise identity or a personal account with different terms and controls. That weakens both auditability and the organisation's ability to enforce acceptable use.

Q: Why does AI data leakage become worse when tools sit outside enterprise identity controls?

A: Because the organisation can no longer prove who used the tool, what data entered it, or how the service handled the content afterwards. Once the interaction sits outside managed identity, policy enforcement becomes partial at best and the exposure window grows with every prompt, paste, or upload.

Q: What are the signs that AI tool usage is outside governance?

A: Look for repeated calls to AI endpoints from unmanaged devices, local processes tied to MCP servers, newly active OAuth tokens, and browser or desktop extensions that have not been reviewed. Those signals show the tool is not just installed, but actively moving data through a path security teams may not know exists.

Q: Should organisations block AI tools or enable them safely?

A: Organisations should enable AI safely rather than rely on blanket blocking. Bans often push employees toward personal accounts and unmonitored tools, which reduces visibility and increases risk. A safer model combines approved AI paths, data classification, monitoring, and clear enforcement for prohibited content.

Technical breakdown

Why personal AI accounts break enterprise visibility

Personal-account AI use bypasses the control plane that security teams usually rely on. When employees access GenAI tools outside SSO, the organisation loses unified identity context, audit trails, retention enforcement, and policy-based restrictions on data handling. That matters because the security question is no longer only whether the tool is approved, but whether the interaction is attributable, reviewable, and governable. In practice, the enterprise can have data flowing into the same external service through two completely different trust paths, one managed and one unmanaged. The unmanaged path is where visibility collapses.

Practical implication: extend governance to the identities employees actually use, not just the applications IT has approved.

How AI prompts and uploads become a data leakage channel

AI leakage is not limited to obvious file uploads. Prompt text, copied snippets, pasted incident details, and embedded context from browser or endpoint workflows can all carry sensitive information into a model interaction. Once that data leaves the enterprise boundary, downstream handling depends on the provider’s storage, reuse, and training settings, not the organisation’s own policy. This is why data classification alone is insufficient if enforcement stops at email and storage systems. AI sessions turn everyday knowledge work into a live data-transfer path.

Practical implication: classify AI interactions as a data-loss surface and monitor prompts, paste activity, and file transfers together.

Why agentic AI raises the exposure level further

AI agents increase risk because they do more than answer questions. They can retain context, access local files, read clipboards, and synchronise information into other systems without the same oversight applied to traditional SaaS use. That creates a persistent, high-value exposure zone on endpoints and inside workflows, especially when agents operate with broad permissions. The security problem shifts from a single user query to a semi-continuous interaction layer that can accumulate and propagate sensitive material. For identity teams, these agents behave like non-human actors that need scoped governance.

Practical implication: treat AI agents as governed non-human identities with explicit access boundaries and telemetry requirements.

Threat narrative

Attacker objective: The practical objective is not always a direct external intrusion but durable exposure of confidential enterprise data outside governed controls.

1. Entry occurs when employees use personal AI accounts or endpoint-based agents outside enterprise identity controls, creating an unmanaged path for sensitive information.
2. Credential or policy controls fail at the boundary because the interaction is not covered by SSO, centralized logging, or enterprise retention rules.
3. Sensitive data is then exposed through prompts, copy and paste actions, uploads, or agent context persistence, expanding the organisation’s data footprint beyond oversight.
4. The impact is loss of control over where data persists, who can access it, and whether it may be reused or retrained by the AI service.

Breaches seen in the wild

• Samsung ChatGPT leak 2023: Samsung staff pasted chip source code and meeting notes into ChatGPT weeks after it was allowed, leading Samsung to restrict generative AI tools.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Unmanaged AI use is becoming a data-governance failure, not a simple accept-or-block policy issue. The article shows that employees are already using AI as part of ordinary work, often through personal accounts that evade enterprise controls. The real gap is not policy intent but control reach, because SSO and logging do not govern what they cannot see. Practitioners should treat AI usage as an enterprise data path, not a side channel.

Personal-account AI usage creates a visibility break that identity teams should recognise immediately. When a worker shifts from managed identity to personal login, the organisation loses attribution, enforcement, and retention control in one step. That is a classic governance boundary failure, and it is especially serious when the content flowing through the session includes regulated or source-controlled data. The practitioner conclusion is to govern the session path, not just the user’s stated intent.

AI agents introduce a new non-human data-handling problem that conventional DLP was not built to absorb. Persistent context windows, clipboard access, file-system reach, and endpoint-level execution make these tools behave like privileged automated actors that can accumulate sensitive material over time. AI data leakage through personal accounts: this is the named concept the article exposes, and it describes the widening gap between enterprise data policy and the accounts through which data actually leaves the organisation. Security leaders should classify agents as governed NHI-like systems with explicit scope and telemetry.

Blocking AI outright is usually a loss-making control strategy. The article suggests that hard restrictions often push use into shadow channels rather than eliminating it. That means the control objective should be visibility and governed enablement, not blanket prohibition. The practitioner conclusion is to make sanctioned AI paths safer than unsanctioned ones, then measure whether usage actually returns to managed identities.

The security question is now who can move sensitive data into AI systems, not only which AI tools are allowed. That shifts the centre of gravity from tool approval to identity assurance, endpoint monitoring, and data flow governance. The article’s evidence supports a broader conclusion for the field: AI risk is emerging where identity controls stop at login and do not extend to the session, the data, or the agent. Practitioners should govern the interaction layer as a first-class control surface.

From our research library:

• 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks, according to the State of Secrets Sprawl 2026.
• 43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to the State of Secrets in AppSec.
• Read next: AI Agent Identity Security Buyer's Guide

What this signals

AI data leakage through personal accounts: this is the operational pattern security teams need to name, because it explains why enterprise controls fail even when policy exists. The boundary has shifted from application approval to account provenance and session visibility, which means IAM, DLP, and endpoint telemetry now have to work together.

Personal-account GenAI use also creates a governance asymmetry: employees can move faster than security teams can inventory the tools they are using. That gap is especially important for agents and copilots that persist context locally, because the organisation may not even know which interaction created the exposure.

43% of security professionals are concerned about AI systems learning and reproducing sensitive information patterns from codebases, according to the State of Secrets in AppSec. That concern is warranted, but the deeper issue is still control placement: sensitive data must be governed at the point of interaction, not after it has already entered the model.

For practitioners

• Map AI usage by identity path Inventory which GenAI tools are accessed through enterprise SSO and which are accessed through personal accounts, then separate managed from unmanaged usage in reporting.
• Extend monitoring to prompts and paste events Correlate clipboard activity, browser use, file uploads, and prompt content so you can see where sensitive data is entering AI tools.
• Treat AI agents as governed non-human identities Assign scoped access, telemetry requirements, and ownership for agents that can read files, retain context, or synchronise data.
• Review retention and training settings for approved AI services Confirm that enterprise AI agreements prevent unintended reuse or training on prompts and uploads, especially for regulated or source-controlled material.
• Create safer sanctioned alternatives to shadow AI Reduce the incentive to bypass controls by providing approved AI workflows that meet the same productivity needs with visible identity and data controls.

Key takeaways

• Sensitive data is already moving into AI tools through ordinary work patterns, and personal-account usage is the main reason visibility breaks down.
• The control problem is broader than application approval because SSO, logging, retention, and training limits all fail when the session is unmanaged.
• AI agents raise the stakes further by persisting context and handling data on endpoints, so practitioners need identity and telemetry controls that extend beyond the browser.

Key terms

• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Personal Account Usage: Personal account usage means employees access work-related AI tools through non-corporate accounts. This reduces organisational control over logging, retention, policy enforcement, and user identity, especially when the same device or network is involved. It creates a major blind spot because corporate monitoring may not capture the activity at all.
• AI data leakage: AI data leakage occurs when sensitive business information is exposed through prompts, outputs, or copied content in AI-assisted workflows. In browser-driven work, the risk is often accidental rather than malicious, so governance depends on data rules, usage policy, and session controls.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions, including calling APIs, writing code, and orchestrating other agents, with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps practitioners align identity controls with the way modern AI and automation actually move data.