Accountability remains with the organisation that decides how the tool is used and what data employees may submit. For regulated data, legal and security teams must ensure lawful basis, minimum necessary disclosure, and approved retention terms. Vendor controls help, but they do not replace internal governance, training, and enforcement.
Why This Matters for Security Teams
When regulated data enters a generative AI tool without the right controls, the risk is not just accidental disclosure. It can trigger privacy, contractual, and sector-specific compliance failures, especially if the organisation has not defined approved use, retention limits, or review requirements. Accountability sits with the organisation because it chose the workflow, allowed the access, and failed to put controls around the data path. The practical question is not whether the model accepted the input, but whether governance existed before that input was permitted. The NIST Cybersecurity Framework 2.0 is useful here because it frames AI use as part of broader governance, risk, and control management rather than a standalone technology decision.
Security teams often get this wrong by treating chatbot usage as a personal productivity issue instead of an enterprise data handling issue. That creates a gap between policy and actual behavior, especially when employees paste sensitive case notes, customer records, or financial data into public or unmanaged tools. In practice, many security teams encounter the exposure only after the data has already been submitted, rather than through intentional approval and control design.
How It Works in Practice
Accountability should be assigned through normal governance lines, not left to the vendor interface or the individual employee alone. Legal, security, privacy, and data owners each have a role: legal determines whether the use is lawful, privacy validates data handling, security defines technical safeguards, and business owners decide whether the workflow is permitted at all. Current guidance suggests that organisations should treat ChatGPT and similar tools as data-processing environments when regulated information may be entered, even if the tool is externally hosted.
In practice, the control set usually includes approved-use policy, data classification, user training, logging, access restrictions, and escalation paths for prohibited disclosures. The NIST SP 800-53 Rev 5 Security and Privacy Controls maps well to this problem because it helps organisations translate policy into concrete safeguards such as access enforcement, auditability, configuration management, and incident response. For many teams, the critical issue is not just whether the data is sensitive, but whether the organisation can prove it controlled who could submit it, what could be submitted, and how outputs were reviewed.
- Define which regulated data classes are prohibited, restricted, or allowed only in approved environments.
- Require user guidance that is specific enough to prevent vague “do not enter sensitive data” policy language.
- Align retention, logging, and monitoring with legal and privacy obligations before rollout.
- Use technical controls where possible, including CASB, DLP, SSO, and app allowlisting.
- Document who approves exceptions and who investigates violations.
Where this guidance breaks down is in bring-your-own-device environments with shadow AI use, because the organisation may have policy but no effective technical visibility or enforcement.
Common Variations and Edge Cases
Tighter control often increases friction, requiring organisations to balance speed of adoption against the risk of uncontrolled disclosure. That tradeoff is especially visible when teams want the productivity benefits of GenAI but also handle health, financial, legal, or customer-identifiable data. Best practice is evolving, and there is no universal standard for whether all regulated data must be blocked or whether some categories can be used under strict safeguards and documented approvals.
One common edge case is employee prompt use for redacted or partially anonymised data. That can reduce risk, but it does not eliminate it if re-identification is possible or if the prompt still reveals context that matters under law or contract. Another edge case is enterprise-tuned or privately hosted AI tools: accountability still remains with the organisation, because hosting location does not remove the need for policy, monitoring, and retention governance. In some regulated environments, the strongest control is to prohibit direct input entirely and route users to vetted internal workflows instead. That approach is not always the most convenient, but it is often the only defensible option when data sensitivity is high and the blast radius of disclosure is unacceptable.
Where organisations struggle most is when procurement, security, and legal have not agreed in advance on acceptable use terms and the first review happens only after a sensitive prompt has already been submitted.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0, NIST SP 800-63 and NIST AI RMF set the technical controls, while PCI DSS v4.0 define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.OV-01 | Governance oversight is central when employees can enter regulated data into AI tools. |
| NIST SP 800-63 | Identity assurance matters when access to AI tools and sensitive workflows must be controlled. | |
| PCI DSS v4.0 | Financial data in prompts can trigger cardholder-data handling obligations and logging requirements. | |
| NIST AI RMF | AI risk management supports accountability for data handling, misuse, and downstream harm. |
Prevent cardholder data from being entered into public AI tools unless explicitly approved and controlled.
