Generative AI security risks are exposing data governance gaps

At a glance

What this is: Cyberhaven says generative AI is exposing a governance gap as sensitive data flows into AI tools that many organisations cannot see or control.

Why it matters: It matters because DLP, DSPM, and compliance programmes need control over conversational data egress, personal accounts, and AI-assisted workflows, not just traditional file movement.

By the numbers:

• There has been an 80% year-over-year increase in data movement events into and out of GenAI SaaS, according to Cyberhaven.

Context

Generative AI security risk is not primarily a model vulnerability problem. It is a data governance problem created when sensitive information moves through conversational interfaces that legacy DLP and DSPM programmes were not built to observe.

The identity and access issue is that many of those interactions happen through personal accounts, unmanaged sessions, or tools outside central policy enforcement. That weakens visibility, auditability, and control over where regulated or confidential data ends up.

Key questions

Q: What breaks when employees use generative AI tools outside DLP coverage?

A: Legacy DLP breaks because it is tuned for file movement, not conversational disclosure. When employees paste or upload sensitive content into GenAI tools, the organisation can lose visibility before any policy engine sees a transfer event. The practical result is blind spots in monitoring, alerting, and evidence collection.

Q: Why do personal AI accounts create governance risk in software teams?

A: Personal accounts break visibility, ownership, and policy enforcement. They make it harder to prove which model or tool generated code, whether corporate data was exposed, and whether approved safeguards were applied. From a governance perspective, the account becomes part of the control boundary, so unmanaged accounts weaken accountability across the development lifecycle.

Q: How can security teams tell whether AI lifecycle controls are working?

A: They should look for evidence that access requests, policy enforcement, and usage visibility are centrally recorded and current. If those signals are fragmented across platforms, the programme may be documenting governance rather than enforcing it. Continuous traceability is the practical test.

Q: What should teams do first when sensitive data is moving through AI tools?

A: First, identify which AI tools, browser flows, and MCP-connected systems can touch sensitive data. Next, define what content is allowed, what must be masked, and what must be blocked. Finally, tie those rules to enforcement so policy applies in real time across the same workflows users rely on.

Technical breakdown

Why conversational AI breaks legacy DLP assumptions

Traditional DLP was designed around predictable egress paths such as email, USB, cloud storage, and sanctioned SaaS. Conversational AI changes the control surface because the sensitive content is pasted, uploaded, or generated inside a browser session rather than transferred as a file. That matters because file-centric monitoring can miss the exact moment data leaves governance boundaries. In practice, the control failure is not that DLP stopped working, but that the policy logic no longer matches how users move information.

Practical implication: redefine monitored egress around prompts, uploads, and copy-paste activity, not just file transfer events.

Why personal AI accounts create a governance blind spot

A personal account severs the link between the user, the device, and enterprise governance. Once an employee uses a consumer AI account, central logging, retention policy, and enforcement often disappear, even if the work content is highly sensitive. That creates a split-control problem: the organisation may know the employee used AI, but not which account type carried the data or whether the session was governed. This is where visibility-first controls become essential before policy can be credible.

Practical implication: distinguish corporate from personal AI sessions at the endpoint and tie usage to account type before data leaves the device.

How data lineage becomes the control layer for AI workflows

Data lineage tracks where content came from, how it changed, and where it went after entering an AI workflow. For GenAI security, that matters because the security question is often not whether a file was moved, but whether a sensitive paragraph, record, or code fragment was exposed through a prompt or chat session. Lineage gives security teams the context to separate routine AI use from risky disclosure patterns and to prove what touched which system when compliance questions arise.

Practical implication: extend lineage tracking into AI interactions so investigators can trace prompt-level exposure and not just document movement.

Threat narrative

Attacker objective: The practical objective is to extract or expose sensitive enterprise data through AI workflows that bypass normal governance controls.

1. Entry occurs when employees paste, upload, or generate sensitive material inside GenAI interfaces that sit outside traditional DLP coverage.
2. Credential or account context is weakened when those interactions happen through personal AI accounts, which bypass enterprise logging and policy enforcement.
3. Impact follows as confidential, regulated, or proprietary data leaves the governance perimeter without reliable detection, auditability, or remediation evidence.

Breaches seen in the wild

• Samsung ChatGPT leak 2023: Samsung staff pasted chip source code and meeting notes into ChatGPT weeks after it was allowed, leading Samsung to restrict generative AI tools.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Generative AI has turned data egress into a conversational control problem. The most important shift is not that AI creates new categories of sensitive data, but that it changes where and how exfiltration-like behaviour happens. Security teams that still anchor policy to file movement are governing yesterday's workflow, not today's. Practitioners should treat prompt-level disclosure as a first-class governance event.

Shadow AI is a visibility failure before it is a policy failure. If security teams cannot see which AI tools are used, through which account types, and with what content, then enforcement is already downstream of the risk. This is why blocking alone does not solve the problem. The control gap sits in discovery, classification, and session attribution, which means governance must start at the endpoint and identity layer.

Personal accounts are now an access-control issue for data security. When work data moves through non-corporate AI sessions, SSO, retention, and logging controls can disappear even though the user believes they are simply being productive. That creates a governance trust gap between user intent and policy enforcement. Organisations need to treat account context as part of the data protection boundary, not just as an authentication detail.

Data lineage is becoming the named concept that links DLP and AI governance. Legacy DLP tells you that a file moved, but AI-era governance needs to know what content was exposed, where it went, and whether it touched an unmanaged model session. That aligns most closely with NIST AI RMF governance thinking and with the broader need to classify and monitor AI-bound data flows. Practitioners should build controls around lineage, not just alerts.

From our research library:

• 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks, according to the State of Secrets Sprawl 2026.
• Read next: Enterprise AI Copilot Security Guide

What this signals

Data lineage is becoming the governing concept for AI-era DLP. Organisations that only monitor files will miss the real exposure path, because prompts and pasted text now carry the sensitive content. The reader's programme should start by mapping where GenAI interactions occur and where those interactions escape policy boundaries.

Visibility into AI usage now needs to extend beyond tool discovery into account context and content sensitivity. That is the operational difference between a policy that exists on paper and a programme that can actually prove control.

The governance challenge is no longer whether AI should be allowed, but whether the organisation can evidence control over how data enters, moves through, and exits GenAI workflows.

For practitioners

• Define AI egress as a monitored data path Extend DLP policies to cover prompts, uploads, copy-paste actions, and browser-based AI sessions so conversational disclosure is treated like other egress channels.
• Separate corporate and personal AI usage Use endpoint controls to identify whether AI activity occurs in enterprise-managed sessions or personal accounts, then apply different policy and logging rules to each.
• Add data lineage to AI monitoring Track where sensitive content originated, which AI tools handled it, and where the resulting data went so incident response and compliance evidence remain defensible.
• Risk-rank AI interactions by content sensitivity Allow low-risk use, alert on medium-risk content, and block or coach on high-risk prompts that include regulated, confidential, or strategic material.
• Bring AI usage into governance reporting Report on AI tool adoption, account type distribution, and sensitive-data interaction patterns so IAM, security, and compliance teams share the same view of exposure.

Key takeaways

• Generative AI is exposing a control gap where sensitive information moves through conversational interfaces that legacy DLP programmes do not reliably monitor.
• The article's evidence points to routine exposure, with employees inputting sensitive information into AI tools every three days on average and GenAI data movement rising sharply year over year.
• The practical response is to govern prompts, account context, and data lineage together so AI usage remains visible, auditable, and policy-driven.

Key terms

• Generative AI: AI designed to create text, code, images, or other content in response to a prompt. It is usually reactive rather than autonomous, which means the main security concern is output quality, leakage, and misuse of generated content rather than independent action.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Data Lineage: The record of how data moves across systems, applications, and workflows. In security operations, lineage shows where sensitive data propagates, which identities touch it, and how a compromise could spread across connected environments.
• Conversational DLP: Conversational DLP is the application of data loss prevention controls to prompts, chats, and other AI interactions rather than just files and email. It extends inspection and policy enforcement to the way people actually disclose information in GenAI workflows.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps identity and security practitioners connect governance controls to the operational realities of modern access and data movement.