How should security teams reduce identity risk when employees use large language models with sensitive enterprise data?

Security teams should combine access control, data-loss prevention, and user governance before broad LLM rollout. Limit who can reach high-risk models, restrict what data can be submitted, and monitor for prompt injection or exfiltration paths. Role-based access control is useful, but it must be paired with inline inspection and clear acceptable-use policy to reduce data leakage.

Why This Matters for Security Teams

Large language models change the identity risk profile because employees are no longer only accessing data through familiar applications. They are pasting sensitive content into systems that may retain prompts, route data through external services, or expose information through plugins and connected tools. That makes the question less about whether the model is “safe” and more about how identity, data, and usage controls are enforced at the point of interaction.

Security teams should treat LLM use as an access path, not just a productivity feature. The right control set depends on who can use the model, what data they can submit, and whether the model can call tools or systems on their behalf. NIST guidance is helpful here because it anchors the program in risk management rather than one-off technical blocks, as reflected in the NIST Cybersecurity Framework 2.0. NHIMG research also shows why this matters operationally: in the Ultimate Guide to NHIs, identity security failures are consistently tied to weak visibility and overexposure, which is the same pattern emerging around employee LLM use.

In practice, many security teams encounter prompt leakage only after sensitive data has already been submitted to a model or copied into a third-party workflow.

How It Works in Practice

The most effective approach is layered control at the identity, data, and workflow levels. Start by deciding which employee groups may use which models, then narrow allowed data classes and set clear rules for what can never be entered. For higher-risk use cases, inline DLP or content inspection should sit in the request path so that confidential data is blocked before it reaches the model, not discovered after the fact.

Identity control matters because model access often expands through single sign-on, browser extensions, copilots, and connected SaaS tools. Use strong authentication, role-based entitlements, and conditional access so that model use reflects business need. Where the model can trigger actions through connectors or plugins, treat those tool paths like privileged access. The NIST SP 800-53 Rev. 5 Security and Privacy Controls is useful for mapping logging, access enforcement, and media protection to concrete safeguards.

Security teams should also define acceptable-use policy in operational terms, not legal language. That means specifying whether source code, regulated records, customer data, or secrets may be used in prompts, and whether outputs may be copied into downstream systems without review. If the environment supports it, separate approved enterprise models from public consumer tools and route sensitive workloads only through managed tenancy. The Top 10 NHI Issues page highlights how over-privilege, poor visibility, and weak monitoring repeatedly turn identity convenience into exposure.

• Classify prompt content by sensitivity before submission.
• Restrict model access by user role, device posture, and location.
• Block secrets, regulated data, and high-risk identifiers inline.
• Log prompt, response, and connector activity for investigation.
• Review third-party plugins and outbound tool calls as privileged pathways.

These controls tend to break down when users can freely move between managed and unmanaged models because policy enforcement and audit logging no longer follow the data.

Common Variations and Edge Cases

Tighter LLM governance often increases friction for employees, so organisations have to balance usability against leakage risk. Best practice is evolving, but current guidance suggests that one-size-fits-all restrictions are usually too blunt for knowledge work. A finance team, a software engineer, and a customer support agent do not carry the same data exposure, so policy should vary by role, dataset, and model trust level.

One common edge case is shadow AI, where employees use public chat tools outside sanctioned channels. Another is enterprise copilots that appear safe because they are internally branded, even though they still connect to external model services or plugins. The 2024 ESG Report: Managing Non-Human Identities shows how broadly exposed identity problems can become once controls are inconsistent, and that pattern applies directly when model access is fragmented across teams.

Security teams should also be cautious with retention settings, training opt-outs, and connector scopes. There is no universal standard for whether prompts should be retained for detection, and that decision should be tied to legal, privacy, and incident response requirements. The safest operating model is to keep sensitive data out of general-purpose models unless the environment provides enterprise-grade isolation, strong logging, and explicit contractual limits on data use.

When these controls are absent, the risk is not just leakage from the prompt itself but downstream exposure through cached outputs, copied summaries, and over-permissioned integrations.

Related resources from NHI Mgmt Group

• How should security teams use sensitive data discovery to reduce AI risk?
• How should security teams control shadow AI use when employees paste sensitive data into public models?
• Why do large language models create risk when organisations use them with sensitive data or operational knowledge?
• How should security teams reduce cloud identity risk in customer data environments?