A BAA does not control what employees enter into prompts, which tier they use, or whether PHI is exposed before the model ever receives it. That leaves the covered entity responsible for the hardest part of compliance: controlling data at the point of use. Without runtime policy, the contract documents responsibility but does not reduce exposure.
Why a BAA does not govern what happens at the prompt
A hipaa BAA sets contractual expectations between the covered entity and the service provider, but it does not control the user’s input moment. If a clinician pastes PHI into ChatGPT, the exposure has already happened before any downstream processing or contractual remedy can help. The real control point is the interface where people decide what to share and which workflow they use.
That distinction matters because compliance failure is often created by ordinary user behaviour, not by a broken vendor contract. A BAA can support lawful use of a service, but it cannot prevent employees from entering sensitive data into the wrong tier, using the wrong account, or relying on a workflow that was never approved for PHI.
What actually breaks in the operational model
The main break is that responsibility stays with the organisation at the point where data is disclosed. A contract can allocate obligations, but it cannot enforce prompt discipline, classify data in real time, or stop a user from exposing PHI to a model session that sits outside the intended control path. That is why runtime controls, approved use cases, and data-entry guardrails matter more than paper assurances.
This is also where healthcare teams often confuse procurement comfort with security coverage. If the only control is a signed BAA, there is no operational check on whether the workflow enforces minimum necessary use, whether the user is authorised to share the data, or whether the application prevents PHI from being pasted into an unsafe context.
For healthcare environments, that gap is especially visible when identity and access rules are already strict elsewhere in the stack. NHIMG’s Healthcare Identity Security Guide shows why clinician workflows, shared workstations, and access governance become the practical control plane when sensitive data is in play.
Why ChatGPT use needs runtime policy, not just vendor terms
Using ChatGPT safely requires a policy that acts before disclosure, not after it. That usually means defining which data classes may be used, which tools or tiers are approved, what redaction is required, and when a use case must be blocked entirely. Without those decisions embedded in workflow, the organisation is asking users to make security judgments in the middle of active work, which is unreliable at scale.
In practice, the strongest control is to make the safe path the easy path. Teams should use approved enterprise configurations, enforce data-loss protections where available, and remove ambiguity about whether PHI may be entered at all. If that cannot be done, the right answer is often not better wording in the BAA, but a narrower scope of permitted use.
The broader compliance lesson is that contractual coverage and operational control are different layers. NHIMG’s Identity Security Regulatory Map is useful here because it helps teams map legal and regulatory obligations to the actual control mechanisms they need to operate, rather than stopping at vendor paperwork.
Where exposure shows up in practice
Exposure usually appears in three places: the prompt itself, the session history, and the downstream handling of the output. Even if the model provider has appropriate contractual commitments, the original prompt may already contain PHI, the conversation may be retained longer than expected, or the generated answer may be copied into an unapproved system. Each step creates a separate chance for disclosure, retention, or misuse.
That is why healthcare teams should think in terms of workflow containment, not just service selection. A BAA may help with vendor-side assurance, but it does not solve user behaviour, does not define acceptable prompts, and does not verify that a clinician’s interaction is limited to a safe, approved environment.
For a practical healthcare-specific reference point, Ultimate Guide to NHIs, Regulatory and Audit Perspectives is relevant because it highlights how auditability, governance, and access review become central when sensitive systems are used in regulated environments.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 provides the primary governance reference for this topic.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AC-6 — Least Privilege | Limits who can use AI workflows that may touch PHI. |
| IA-2 — Identification and Authentication (Organizational Users) | Supports controlled access to approved enterprise AI workflows. | |
| AU-2 — Event Logging | Supports auditability of prompt-use and disclosure-relevant activity. | |
| Recommendation — Restrict PHI-capable ChatGPT access to approved roles and need-to-know use cases. Require strong user authentication before any approved ChatGPT workflow can process sensitive data. Log approved AI usage events needed to investigate PHI handling and policy violations. | ||
Practitioner Guidance
What to prioritise: Start by classifying which ChatGPT use cases are allowed to touch PHI, then define the exact data types, workflows, and user roles that are permitted. If you cannot describe that boundary in operational terms, the BAA is not giving you meaningful protection.
What to verify: Confirm whether prompts are being logged, retained, or routed through approved enterprise controls, and verify that staff understand the difference between a vendor contract and a use-policy control. A signed agreement is not evidence of safe handling at the point of use.
Common mistake: Treating the BAA as a substitute for prompt governance. The organisation still owns disclosure decisions, and the first failure is usually human workflow, not vendor noncompliance.
Practitioner takeaway: If the control does not shape what users can enter before the model sees it, it does not materially reduce PHI exposure.
Related resources from NHI Mgmt Group
- What breaks when healthcare teams rely on business as usual instead of enforcing HIPAA controls consistently?
- How should security teams use IAST and RASP in NHI governance?
- What breaks when teams rely on periodic access certification alone?
- What breaks when healthcare organisations rely on RBAC alone?