What should security teams do when employees are using public LLMs with sensitive data?

Why This Matters for Security Teams

When employees paste sensitive material into public LLMs, the risk is not limited to the immediate prompt. Data can be retained, echoed back in outputs, exposed through account compromise, or reused in ways the organisation cannot fully control. That makes this a data handling problem, a governance problem, and often an identity problem when employees use personal accounts outside managed authentication.

Security teams should treat public LLM use as an approved-or-prohibited decision, not a vague productivity preference. The strongest responses align policy, data classification, monitoring, and user guidance so that the organisation can reduce exposure without blocking legitimate work. Guidance from the NIST AI Risk Management Framework is useful here because it frames AI use as a managed risk lifecycle rather than a one-time policy statement.

In practice, many security teams encounter the real impact only after source code, customer data, or internal incident details have already entered a public model through a convenience-driven workflow.

How It Works in Practice

The practical response starts with classification. Not all data should be treated the same, and public LLM usage should be restricted first where the input includes regulated, confidential, or security-sensitive material. Teams typically combine policy controls with technical guardrails such as browser restrictions, DLP integration, secure enterprise AI alternatives, and logging of sanctioned AI usage.

Operationally, the goal is to reduce both intentional misuse and accidental disclosure. That means employees need clear examples of prohibited inputs, not just abstract policy language. It also means security teams should define how prompts and outputs are reviewed, how exceptions are approved, and how incidents involving AI tools are escalated. The NIST AI 600-1 Generative AI Profile is helpful for translating AI governance into control expectations around mapping, measurement, and monitoring.

• Block or warn on prompts containing secrets, personal data, or regulated records.
• Route approved use cases to enterprise-controlled AI services where retention and access are governed.
• Train users on what counts as sensitive input, including code, logs, tickets, and incident notes.
• Review logs for repeated high-risk usage patterns and shadow AI adoption.
• Document exception handling so business teams do not create informal workarounds.

Where agentic workflows are involved, the risk expands because a model may not only see data but also act on it. That is why the OWASP Agentic AI Top 10 matters for environment design: tool access, data flow, and human approval boundaries all need explicit control points. These controls tend to break down in highly decentralised organisations with unmanaged personal accounts and no DLP coverage, because security teams cannot reliably see where sensitive prompts are going.

Common Variations and Edge Cases

Tighter restrictions on public LLM use often increase friction for knowledge workers, so organisations need to balance productivity gains against confidentiality risk. The right answer is not always a total ban, but best practice is evolving toward tiered use based on data sensitivity, business function, and whether the model is public or enterprise-managed.

One edge case is non-sensitive text that becomes sensitive in context. A harmless-looking meeting summary may reveal strategy when combined with project names, customer references, or operational timelines. Another is prompt data that includes embedded secrets in code snippets, logs, or configuration files. Security teams should assume that employees will not reliably self-classify edge cases under time pressure.

Identity controls also matter. If staff use public LLMs through unmanaged personal accounts, the organisation loses visibility into who accessed what, under which terms, and whether the account is protected by strong authentication. That is where NHI and identity governance intersect: the issue is not only the data, but the lack of enterprise control over the identity that handled it. MITRE ATLAS adversarial AI threat matrix can help teams think beyond leakage to broader abuse patterns, especially where prompt manipulation or output exploitation is plausible.

Where regulation is in play, current guidance suggests aligning policies with internal data handling rules first, then mapping them to sector or privacy obligations. The exact control set will vary, but the operating principle is consistent: if the organisation cannot inspect, govern, or revoke the exchange, it should not allow sensitive data into the public model.

Related resources from NHI Mgmt Group

• How should security teams control shadow AI use when employees paste sensitive data into public models?
• How should security teams stop employees pasting sensitive data into AI prompts?
• How should security teams stop sensitive data from being uploaded into public AI tools?
• How should security teams prevent sensitive data from being exposed when employees use Gemini at work?