Who is accountable when PHI reaches a non-covered Gemini surface?

Why This Matters for Security Teams

Accountability does not disappear when PHI lands in the wrong Gemini surface. If a workforce member enters regulated data into a non-covered interface, the organisation still owns the privacy, access, and oversight failure. A BAA can define vendor obligations for specific services, but it does not automatically extend to consumer tools, experimental AI surfaces, or unmanaged integrations. That distinction matters because the security team is often judged on control design, not vendor intent.

The practical issue is surface sprawl. Users rarely think in terms of contract boundaries; they think in terms of convenience and available tools. That means policy has to be translated into approved workflows, enforced technical controls, and visible guardrails. NIST SP 800-53 Rev. 5 Security and Privacy Controls is useful here because it anchors accountability in access control, auditability, and data protection expectations rather than assumptions about how people will behave.

In practice, many security teams encounter PHI exposure only after a user has already copied it into an unapproved AI surface, rather than through intentional data governance.

How It Works in Practice

Operational accountability starts with scoping. Teams need a clear register of which Gemini surfaces are covered by contractual and security review, which are prohibited for PHI, and which require additional approval before use. That scope should be reflected in policy, user training, DLP rules, and data classification standards. The covered entity remains the accountable party, so internal controls must prevent employees, contractors, and connected systems from treating all AI endpoints as equivalent.

At a minimum, practitioners should align on four control layers:

• Data classification and routing rules that identify PHI before it reaches an AI prompt or attachment.
• Approved-surface governance that distinguishes covered environments from consumer or trial services.
• Logging and audit evidence for access, submissions, and administrative changes.
• Exception handling for research, testing, or integration use cases that may need separate legal and security review.

This is also an identity governance problem. If a user can access both a covered surface and a non-covered one with the same identity, the organisation still needs policy-based segmentation, conditional access, and monitoring that reflect the sensitivity of the data rather than the convenience of the account. The NIST SP 800-53 Rev 5 Security and Privacy Controls catalogue remains relevant because it supports enforceable patterns for access restriction, audit logging, and system integrity.

Where AI is used to process clinical text, current guidance suggests treating prompt content, file uploads, and generated outputs as regulated data paths until proven otherwise. Organisations should also validate whether any connected extensions, connectors, or agentic workflows expand the data path beyond the original Gemini surface. These controls tend to break down when shadow AI usage is allowed on unmanaged devices because policy cannot see or stop PHI movement in time.

Common Variations and Edge Cases

Tighter control often increases friction for clinicians, analysts, and support teams, requiring organisations to balance privacy protection against workflow speed. That tradeoff becomes sharper when teams want to use AI for drafting, summarisation, or search without creating a blanket ban that drives users to unapproved tools.

There is no universal standard for this yet across all AI surfaces, so organisations should separate settled obligations from emerging practice. For example, if a Gemini deployment is covered in one environment but not another, the accountable entity still needs environment-specific governance, because contract coverage does not travel with the user’s intent. The same is true for delegated administration, plugins, and API-based workflows: a small configuration change can move PHI from a controlled system into a surface with different retention, training, or logging behaviour.

For cross-border or multi-entity programmes, legal accountability may be shared, but operational accountability should still be explicit. Someone must own the approved-surface list, someone must own detection and escalation, and someone must own user access reviews. That ownership should be documented before an incident, not argued after one. Where PHI is involved, governance should also be tested against NIST SP 800-53 Rev 5 Security and Privacy Controls and the organisation’s internal data handling policy so exceptions are not silently normalised.

Related resources from NHI Mgmt Group

• Why do non-human identities increase attack surface in cloud environments?
• Why do non-human identities increase attack surface risk?
• Who should be accountable when risky non-human access is found?
• Who is accountable when an AI scribe exposes PHI?