Generative AI security risks are exposing data governance gaps

At a glance

What this is: This analysis says generative AI has become a new enterprise data egress channel, with prompt-based sharing, shadow AI, and personal accounts undermining legacy DLP and governance models.

Why it matters: It matters because IAM, NHI, and data security teams must now govern AI use as part of access control, visibility, and auditability rather than treating it as an isolated productivity risk.

By the numbers:

• Cyberhaven's 2026 AI Adoption & Risk Report found that employees input sensitive information into AI tools on average once every three days.
• There’s been an 80% year-over-year increase in data movement events into and out of GenAI SaaS.
• 32.3% of ChatGPT usage, 58.2% of Claude usage, and 60.9% of Perplexity usage in the enterprise occurs through personal rather than corporate accounts.

👉 Read Cyberhaven's analysis of the top generative AI security risks in the enterprise

Context

Generative AI security is now a data governance problem as much as a technology problem. The primary issue is not model quality or prompt engineering, but the fact that sensitive information is moving into interfaces that legacy DLP, DSPM, and IAM controls were never designed to see. For teams responsible for data protection and identity governance, the core question is whether policy and enforcement still match how employees actually work.

Cyberhaven's article shows a familiar control boundary breaking down: corporate users are moving into personal AI accounts, conversational interfaces are replacing file transfers, and sanctioned controls are losing visibility at the point of use. That creates a real intersection with identity governance because account type, session context, and access enforcement now determine whether AI use is observable or effectively shadowed.

Key questions

Q: How should security teams govern personal AI assistants that act on behalf of employees?

A: Treat each assistant as a distinct non-human actor with its own identity, policy scope, and audit trail. Human delegation alone is not enough when the assistant can move across email, documents, calendars, and internal systems. Governance should bind the sponsor, the executor, and the target resource so access reviews and investigations can separate request from action.

Q: Why do generative AI tools increase data security risk?

A: Generative AI tools increase risk because they expand the number of places where sensitive content can be ingested, copied, surfaced, or misused. They also consume unstructured data that legacy classification tools often misread, which weakens policy enforcement. The result is a larger blast radius when access is over-permissioned or data visibility is incomplete.

Q: What breaks when DLP only monitors file transfers instead of AI prompts?

A: It misses the dominant leakage channel. Sensitive data can leave through pasted text, uploads, and conversational summaries without triggering traditional file-based alerts. That creates a blind spot where employees can move confidential information into AI tools while appearing to use them normally.

Q: Who is accountable when an AI agent accesses regulated data improperly?

A: Accountability sits with the teams that govern the agent's identity, the data classification, and the policy that allowed the access path. If those controls are disconnected, no single owner can explain why the access existed or why it was not removed sooner. Shared context is what makes accountability traceable.

Technical breakdown

Why GenAI behaves like a new data egress channel

Large language model tools are not just another SaaS category. They accept prompts, pasted text, uploads, and API calls that can carry sensitive data out of the organisation without creating the file movement events traditional DLP expects. Because the interaction is conversational, the control point shifts from static transfer scanning to runtime context, including who is using the tool, under which account, and what data class is being entered. That is why data-aware controls and lineage are becoming central to AI-era governance.

Practical implication: extend detection to prompt content and account context, not just file events.

How shadow AI creates an identity and governance blind spot

Shadow AI is the enterprise pattern where employees use unmanaged tools, often via personal accounts, outside corporate logging and policy enforcement. The problem is not only the tool itself, but the loss of identity assurance and administrative control when the session is detached from SSO, central retention, and enterprise audit trails. In that condition, security teams may know AI is in use, but they cannot reliably associate the activity with a managed identity or a governed policy set.

Practical implication: classify AI access by account type and tie it back to identity enforcement before allowing broader use.

Why insider risk now includes AI-assisted exfiltration

AI changes insider threat because the same interface can support accidental leakage, deliberate exfiltration, and legitimate work. A user pasting code into a coding assistant may be trying to be productive, while the same pattern can also be used to move data out with less forensic evidence than email or file transfer. Traditional insider risk tools that watch for bulk downloads or abnormal file access miss this conversational path, so data lineage and behavioural context matter more than single-event alerts.

Practical implication: correlate AI interactions with source data sensitivity and user behaviour over time.

Threat narrative

Attacker objective: The attacker objective is to extract sensitive enterprise data through AI workflows while minimising visibility and forensic evidence.

1. Entry occurs when employees paste sensitive data, upload files, or submit API payloads to AI tools through browser sessions or unmanaged accounts.
2. Credential or account abuse follows when personal accounts bypass SSO, logging, and retention controls, removing the activity from enterprise governance.
3. Impact occurs when confidential source code, customer data, regulated records, or strategic material leaves the organisation without a durable audit trail.

NHI Mgmt Group analysis

AI security is becoming an identity governance problem. When employees use personal AI accounts, the organisation loses the policy enforcement, logging, and accountability that make access governable. That means AI usage cannot be treated as a standalone productivity issue; it must be tied to identity assurance, session context, and data classification. For practitioners, the real control question is whether the account behind the AI interaction is managed enough to be governed.

Shadow AI is a visibility failure before it is a policy failure. Security teams often try to block tools first, but blocking without managed alternatives usually shifts activity into less observable channels. The more useful framing is visibility to control, then policy to enforcement. This aligns with NIST Cybersecurity Framework 2.0 and identity-centric governance because the organisation needs to know which identities are touching AI systems before deciding what is permitted.

Data lineage is the missing control plane for GenAI governance. Legacy DLP watches endpoints and file movement, but GenAI risk is distributed across prompts, uploads, summaries, and copy-paste interactions. That creates a distinct governance gap we can call conversational egress blindness: the organisation cannot see how sensitive data enters, travels through, and exits AI workflows. Practitioners should treat that as a design flaw, not a tuning problem.

AI-assisted insider risk changes the boundary between malicious and routine behaviour. Employees are often using AI to work faster, but the same channels can also support low-and-slow exfiltration with fewer alerts than conventional channels. That means governance models must distinguish intent, sensitivity, and account provenance rather than assuming all exfiltration looks anomalous. The practitioner conclusion is clear: behavioural context and data context now need to operate together.

What this signals

Conversational egress blindness should now be treated as a control gap in its own right. The problem is not only that AI tools are popular, but that their use often escapes the accountability model that traditional DLP and IAM rely on. Security teams should expect pressure to prove which identities touched which AI systems, on what data, and under what policy conditions.

The next governance step is to connect endpoint telemetry, identity context, and data classification into a single decision layer. That means aligning AI monitoring with established frameworks such as NIST Cybersecurity Framework 2.0 and treating unmanaged AI access as a measurable programme risk rather than an isolated acceptable-use issue.

For practitioners

• Inventory AI usage by identity type Classify which tools are used through corporate SSO, which are accessed through personal accounts, and which sessions lack enterprise logging or retention. Prioritise the unmanaged flows first because they are the weakest governance point.
• Extend DLP to conversational egress Add controls that inspect prompts, pasted text, and file uploads in browser-based AI sessions, then map those interactions to data sensitivity and user identity. File-transfer monitoring alone will miss the dominant leakage path.
• Bind AI access to managed identity and audit trails Require SSO-backed access for sanctioned tools, preserve session records, and make account provenance visible to compliance and insider-risk teams. If a session cannot be attributed, it cannot be governed.
• Use data lineage for AI risk triage Track where content originated, which AI interfaces touched it, and where it was reused or exported. This gives security and compliance teams a defensible basis for distinguishing normal productivity use from high-risk exposure.
• Set risk-based AI policy thresholds Allow low-risk use, alert on sensitive data in prompts, and block or coach on high-risk interactions involving regulated or proprietary material. Blanket bans usually displace use rather than reduce it.

Key takeaways

• Generative AI turns prompts and uploads into a new egress path that legacy DLP often cannot see.
• Personal AI accounts weaken auditability, identity assurance, and enforcement at the exact point where sensitive data leaves the organisation.
• Security teams need conversational controls, data lineage, and managed identity enforcement if they want AI use to remain governable.

Key terms

• Conversational Egress: The movement of sensitive data out of an organisation through chat prompts, pasted text, uploads, or AI-generated interactions rather than traditional file transfer channels. It is a governance problem because the data may leave without the telemetry that standard DLP and logging were built to capture.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Data Lineage: The record of how data moves across systems, applications, and workflows. In security operations, lineage shows where sensitive data propagates, which identities touch it, and how a compromise could spread across connected environments.
• Conversational Egress Blindness: A governance failure in which organisations can monitor file movement but cannot see sensitive data leaving through AI prompts, summaries, or uploads. The term describes a control gap, not a product category, and it highlights why modern data security must inspect interaction context as well as transport.

What's in the full article

Cyberhaven's full blog covers the operational detail this post intentionally leaves for the source:

• Examples of the data categories most commonly entering AI prompts, including source code, customer records, legal documents, and regulated data.
• Detailed explanation of how personal accounts bypass SSO enforcement, centralized logging, retention policies, and governance controls.
• Step-by-step guidance on visibility-first AI policy design, including risk-based monitoring and blocking thresholds.
• Discussion of how endpoint controls can distinguish corporate from personal AI sessions in practice.

👉 Cyberhaven's full post covers the DLP gaps, shadow AI patterns, and governance measures in more implementation detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in a way that helps practitioners connect identity controls to real operational risk. It is designed for security teams that need to govern access, accountability, and lifecycle control across modern identity programmes.