What breaks when organisations rely only on encryption and SOC 2 for ChatGPT security?

Why This Matters for Security Teams

Encryption and SOC 2 both matter, but they answer different questions than ChatGPT security does. Encryption protects data in transit and at rest, while SOC 2 assesses whether controls are designed and operating consistently. Neither control stops a person from pasting sensitive material into a prompt, nor does it automatically govern how prompts, outputs, plugins, or connected tools are handled. For that reason, organisations can remain technically compliant while still creating a direct data exposure path through everyday use.

This is where AI security needs a control model that covers input handling, output review, retention, and access boundaries. Current guidance suggests that organisations should treat prompt content as a data loss surface, not just a convenience feature. That means classifying what users may submit, deciding whether logs are stored, and defining when human review or automated redaction is required. The ENISA Threat Landscape is useful here because it frames modern compromise paths around abuse of normal digital workflows, not only classical perimeter failure. In practice, many security teams encounter the ChatGPT risk only after sensitive text has already been submitted, rather than through intentional control design.

How It Works in Practice

Operationally, the gap appears when organisations assume a secure transport layer and a certified service posture are enough to manage user behaviour. That assumption fails because ChatGPT security is partly a governance problem and partly a data handling problem. If staff are allowed to paste customer records, source code, secrets, or internal incident notes into a model interface, encryption does not change the fact that the content was disclosed to a third party service or internal logging layer.

A stronger approach usually combines policy, technical guardrails, and monitoring. Practical measures include:

• Defining what data classes may never be entered into prompts, especially secrets, regulated data, and internal confidential content.
• Using DLP or content filtering at the browser, proxy, or gateway layer to detect risky submissions before they leave the environment.
• Restricting tool and plugin access so a model cannot reach systems that hold sensitive records without explicit approval.
• Reviewing retention settings, audit logs, and vendor data-use terms to understand where prompts and outputs are stored.
• Training users on safe prompt hygiene, because policy alone does not prevent accidental disclosure.

The governance lens in NIST AI Risk Management Framework is especially relevant because it treats AI risk as a lifecycle issue involving mapping, measuring, managing, and governing. For prompt-heavy environments, organisations should also validate output handling, because sensitive information can reappear in generated text even when the original prompt seemed harmless. These controls tend to break down when the organisation uses unmanaged browser access, shadow AI accounts, or loosely governed integrations that bypass inspection.

Common Variations and Edge Cases

Tighter prompt controls often increase friction for employees, requiring organisations to balance usability against leakage reduction. That tradeoff is real, especially in teams that rely on AI for drafting, coding, support, or analysis. Best practice is evolving, and there is no universal standard for how aggressively prompt content should be inspected across every business unit.

Some environments also face special constraints. Legal, healthcare, finance, and critical infrastructure teams may need stricter handling because prompt content can include regulated records or operationally sensitive material. In those cases, SOC 2 can still be valuable as evidence that a provider has process discipline, but it is not a substitute for use-case-specific controls. If a deployment connects ChatGPT to internal systems through APIs, the risk expands from simple disclosure to action-taking and data retrieval, which makes identity, authorization, and logging more important than encryption alone.

For organisations comparing control sets, the practical question is not whether encryption is enabled, but whether the model can see data it should never receive in the first place. That is where AI governance overlaps with identity governance and data security. For broader threat context on abuse patterns and evolving attacker behaviour, ENISA Threat Landscape remains a useful reference point, but the control design still has to be local to the organisation’s AI use cases.

Related resources from NHI Mgmt Group

• What breaks when organisations rely on manual data classification for AI security?
• What breaks when organisations rely on EDR alone for browser security?
• What breaks when organisations rely only on USB blocking for device security?
• What breaks when organisations rely only on posture checks for NHI security?