13% of GenAI prompts expose sensitive data to enterprise risk

At a glance

What this is: Lasso Security’s study shows that a material share of employee GenAI prompts contain sensitive data, with code, network, and personal information all surfacing in day-to-day use.

Why it matters: For IAM teams, this is a governance problem as much as a data-loss problem because employee prompt behaviour can bypass traditional access controls, privacy controls, and acceptable-use policies.

By the numbers:

• 13% of employee-submitted prompts to GenAI chatbots contained security or compliance risks.
• 30% of prompts in the code and token sharing category included exposed credentials, secrets, or proprietary code.
• 38% of network information exposure prompts posed direct risks by enabling network reconnaissance and unauthorized access.

👉 Read Lasso Security's research on sensitive data exposure in GenAI prompts

Context

Generative AI prompt leakage is a data governance problem that emerges when employees paste secrets, infrastructure details, or personal data into chat tools that were never designed for unrestricted business disclosure. In identity terms, the issue sits at the boundary between human behaviour, access policy, and sensitive-data control, where existing programme ownership is often split across IAM, security, legal, and privacy teams.

The core weakness is not that GenAI is inherently unsafe. It is that organisations often deploy it faster than they define which data classes are allowed, which users are allowed to interact with which tools, and how those interactions are monitored or blocked. That makes prompt handling a control plane problem, not just a user-awareness problem.

Key questions

Q: How should security teams prevent employees from sharing sensitive data in GenAI prompts?

A: Combine policy, browser-level enforcement, and user education. The control needs to block secrets, credentials, personal data, and internal infrastructure details before they are submitted, not after. Teams should also define which roles can use which tools and which data classes are prohibited in each environment.

Q: Why do GenAI chat tools create data leakage risk for IAM and security teams?

A: Because authentication does not control disclosure. Once a user is signed in, the prompt box becomes an unmanaged disclosure point unless policy and inspection are added around it. That makes GenAI use a governance issue across human identity, privacy, and data loss prevention programmes.

Q: What do organisations get wrong about prompt injection and jailbreak risk?

A: They often treat the problem as model behaviour alone. In practice, the risk also comes from untrusted content, hidden instructions, and connected tools that the model can act on. If those inputs are not checked, the model can be steered into unsafe disclosures or actions.

Q: How can organisations measure whether GenAI guardrails are actually working?

A: Track blocked prompt attempts, the share of prompts containing sensitive data, the number of unapproved tools in use, and whether security, legal, and compliance teams have the same visibility. If those metrics do not improve together, the guardrails are not covering actual employee behaviour.

Technical breakdown

Prompt leakage as a data exfiltration path

A GenAI prompt can function like a lightweight exfiltration channel when users paste in source code, configuration snippets, customer records, or internal URLs. Unlike traditional file transfer, the disclosure often feels conversational and low-friction, which reduces user suspicion and increases volume. If the platform logs, retains, or trains on that input, the organisation may lose control over where the data persists and who can later retrieve it. The technical issue is not only prompt content but the lifecycle of the content after submission.

Practical implication: classify prompt inputs by data sensitivity and block submission of secrets, credentials, and regulated data at the browser or gateway layer.

Why network details and code fragments expand attack surface

Internal URLs, IP addresses, MAC addresses, and code samples are not harmless context when they leave the enterprise boundary. Together they help an attacker map systems, identify technology stacks, and infer where access controls are weak. In practice, these fragments can support reconnaissance, credential targeting, and lateral movement planning. The important technical point is that low-value disclosures become high-value when combined across prompts, sessions, or users, especially in tools that are not centrally governed.

Practical implication: treat prompt inspection as reconnaissance prevention, not only as privacy filtering.

Jailbreaks and prompt injection in enterprise GenAI use

Prompt injection and jailbreak attempts aim to override model constraints by placing malicious instructions directly in the user prompt or indirectly in content the model ingests. The risk rises when the model can call tools, retrieve documents, or act on hidden instructions without a strong policy layer. OWASP’s LLM risk guidance puts prompt injection at the centre of current GenAI threats because it can turn a benign workflow into an unsafe one. In an enterprise setting, the control question is whether the model can be manipulated into revealing, transforming, or routing data outside policy.

Practical implication: validate prompts and retrieved content before execution, especially where models can access internal systems or embedded documents.

Breaches seen in the wild

• Shai Hulud npm malware campaign — Shai Hulud campaign: npm malware exposed secrets on GitHub.
• Reviewdog GitHub Action supply chain attack — reviewdog/action-setup GitHub Action supply chain attack exposed secrets.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

GenAI prompt leakage is becoming a human identity governance problem, not just a data security problem. The study shows employees are already using chat tools as informal workspaces for code, network details, and personal data. That puts policy, training, and enforcement into the same operating model, because the risk starts with a person and ends with data exposure. Practitioners should treat prompt behaviour as governed identity activity, not casual experimentation.

Prompt controls are now part of the identity boundary. Traditional IAM proves who can sign in, but it does not prove what a user may disclose once authenticated. Lasso’s findings show that 13% of prompts still carry security or compliance risk, which means policy enforcement must move closer to the interaction point. The implication is straightforward: the boundary of identity governance now includes the text box, not just the login screen.

Code and token sharing create a persistent trust debt inside GenAI programmes. Once developers and employees normalise pasting secrets into chat systems, the organisation inherits a disclosure pattern that recurs across teams and tools. The 30% exposure rate in code and token sharing shows this is not an edge case. Security teams should assume that any model handling operational work will eventually see credentials unless policy blocks the behaviour.

Shadow LLM is the hidden policy gap many programmes are still not measuring. If employees can reach multiple unapproved tools, governance has already lost the first decision point. The study’s discovery across more than 12,000 tools and services points to a control problem that extends beyond a single chatbot. Practitioners should focus on tool discovery, not just prompt moderation, because unmanaged tool choice is where leakage becomes untraceable.

Prompt injection exposes the failure of trust-by-conversation in GenAI workflows. Enterprises often assume that a text interaction is safer than an API call because it feels less programmable. The article shows that assumption is wrong when malicious instructions can be smuggled through prompts or documents. The practitioner takeaway is that GenAI governance must evaluate content provenance, not only user identity and access rights.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%), according to AI Agents: The New Attack Surface report.
• Only 44% of organisations have implemented policies to govern AI agents, even though 92% agree that governing them is critical to enterprise security.
• Ultimate Guide to NHIs shows why rapid identity growth turns unmanaged access into a structural risk.

What this signals

Prompt governance will increasingly converge with human identity policy. Security teams should expect GenAI controls to sit alongside acceptable-use, DLP, and data classification programmes rather than in a separate AI-only lane. The organisations that succeed will define approved data classes, tool access, and enforcement points together, then operationalise them through browser controls and policy engines.

Shadow AI discovery needs to become a standing control, not a one-time survey. When employees can reach thousands of tools across browsers and personal accounts, the real problem is not one chatbot but ungoverned tool choice. As a result, identity teams will need visibility into where AI is used, what data reaches it, and which users are bypassing approved channels.

GenAI security is now a control-stack issue across human identity and data protection. NIST Cybersecurity Framework 2.0 remains a useful organising model for identify, protect, detect, respond, and recover, but GenAI programmes need those functions applied at the point of interaction. The most durable programmes will treat prompt controls as part of access governance, not a bolt-on review step.

For practitioners

• Block sensitive data at the point of prompt submission Inspect prompts before they leave the browser or enterprise gateway and stop secrets, credentials, customer data, and regulated content from being submitted to GenAI tools.
• Define allowed-data rules by role and tool Publish clear controls for which employee groups can use which GenAI tools, and map permitted data classes to each approved use case.
• Discover and catalogue shadow LLM usage Inventory every GenAI platform and assistant in use across the organisation, including personal accounts and browser-based tools that bypass central procurement.
• Add prompt inspection to data-loss prevention workflows Extend existing DLP and privacy processes so they analyse prompt content, attachments, and retrieved context before model submission or tool execution.

Key takeaways

• Lasso’s research shows that employee GenAI use is already exposing sensitive data at a rate high enough to matter operationally, not just anecdotally.
• The underlying failure is governance at the prompt boundary, where access rights do not prevent disclosure once a user starts interacting with a chatbot.
• Organisations need prompt-level policy enforcement, shadow AI discovery, and data-class rules if they want GenAI adoption without normalising data leakage.

Key terms

• Prompt Leakage: Prompt leakage is the unintentional disclosure of sensitive data through text submitted to a GenAI system. It matters because users often paste secrets, internal URLs, or personal data into a chat interface that may log, store, or otherwise process the content outside the organisation’s normal control plane.
• Shadow LLM: Shadow LLM refers to undiscovered or unmanaged GenAI tools used inside an organisation. It creates governance blind spots because security teams cannot apply policy, logging, or data controls to tools they have not inventoried, especially when employees use personal accounts or browser-based access paths.
• Prompt Injection: Prompt injection is a technique that embeds malicious instructions into user input or retrieved content so a GenAI system follows them instead of intended policy. In enterprise settings, it can cause data exposure, unsafe actions, or tool misuse when the model lacks robust content and execution controls.

Deepen your knowledge

GenAI prompt governance and sensitive-data controls are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are working on a similar human identity and data-protection problem, it is worth exploring.

This post draws on content published by Lasso Security: Lasso Research Reveals 13% of Generative AI Prompts Contain Sensitive Organizational Data. Read the original.