How should healthcare teams stop PHI from reaching AI tools in the first place?

Why This Matters for Security Teams

Stopping PHI from reaching AI tools is a data-loss problem first and a compliance problem second. Once a prompt contains patient details, the organisation has already crossed the line that policy, training, or a downstream BAA cannot reliably undo. Current guidance suggests teams should treat prompts like any other outbound data channel and enforce policy before submission, using identity-aware controls and approved workflows. That approach aligns with the NIST Cybersecurity Framework 2.0 and with privacy-by-design expectations in healthcare.

The practical issue is that clinicians, administrators, and analysts will often use AI tools in time-sensitive workflows, especially when they believe the request is low risk. If the guardrail sits only in policy documents or annual training, PHI still leaves the organisation in the first draft of the prompt. NHI Management Group has also documented how exposed credentials and sensitive context are rapidly abused once they are accessible, including in the DeepSeek breach analysis. In practice, many security teams discover prompt leakage only after data has already been submitted to an unapproved AI tool.

How It Works in Practice

The strongest pattern is to stop PHI at the prompt layer before it can be sent to a model endpoint. That means routing AI use through approved applications, proxying requests, and checking content in real time against policy, classification, and identity context. The control objective is simple: only the minimum necessary information should be allowed to leave the organisation, and only for an approved use case. This is consistent with the NIST Cybersecurity Framework 2.0 identify-protect-monitor cycle, where prevention and monitoring are both required.

Operationally, teams usually need four pieces working together:

• Prompt inspection or DLP controls that detect PHI before submission.
• Identity-controlled access so only approved roles can reach approved AI tools.
• Tier controls that block consumer or no-BAA products from receiving regulated data.
• Logging and review that show which prompts were blocked, allowed, or redacted.

For healthcare environments, the design should also reflect workload identity and policy enforcement at runtime, not just user training. The DeepSeek breach is a reminder that once sensitive material enters an AI environment, it can be copied, retained, or exposed in ways the original sender did not intend. Current best practice is to pair preventive controls with explicit approval paths, so users have a safe alternative instead of a shadow workflow. These controls tend to break down in BYOD settings and unmanaged browser access, because the organisation cannot reliably inspect or intercept the prompt before submission.

Common Variations and Edge Cases

Tighter prompt-layer controls often increase friction, requiring organisations to balance patient-data protection against clinical speed and user adoption. That tradeoff is real, and there is no universal standard for exactly how much redaction is enough. For low-risk summarisation or drafting, many teams allow limited text transformation, while high-risk workflows such as patient notes, referral letters, or case reviews need stricter blocking and stronger approval. Guidance is still evolving on how aggressive automated redaction should be when the same prompt may contain both PHI and harmless operational context.

Edge cases usually appear when an AI tool is embedded inside another product, when a vendor model is accessed through an API, or when staff paste PHI into personal accounts to bypass friction. In those situations, the main failure is not the model itself but the lack of a controlled submission path. The DeepSeek breach shows how quickly sensitive material can spread once it enters an uncontrolled environment, while the NIST framework supports treating these flows as governed data channels rather than informal productivity shortcuts. Best practice is evolving, but the direction is clear: if an AI use case cannot be mediated, it should not receive PHI at all.

Related resources from NHI Mgmt Group

• How should security teams govern Cursor-like AI coding tools in the enterprise?
• How should security teams handle risks from AI browser extensions?
• How should security teams govern API keys used for generative AI access?
• How should teams stop secrets from entering code in the first place?