How should banks assess AI risk when customer data can enter public models?

What banks should measure before letting customer data reach public AI models

The right assessment starts with data flow, not vendor branding. Banks need to know exactly which customer records, notes, prompts, attachments, retrieval snippets, and inferred outputs can reach a public model, even indirectly through copilots, support tooling, or embedded AI features. The practical question is whether sensitive information can cross an approved boundary in a form that is recoverable, reusable, or visible to a third party.

That means the assessment has to follow the data at each transformation step. Raw input, redaction, summarisation, retrieval, caching, logging, and export each create different exposure points. If a bank cannot explain where customer data enters, where it is processed, and where it is stored or retained, it does not yet have a defensible AI risk view.

Approval should depend on the kind of data, the path it takes, and the controls around it. A public model used only on heavily sanitised text is a different risk from one that can see payment details, account history, complaint narratives, or internal policy context. Banks should treat the boundary question as a control test, not as an IT procurement formality.

Why chatbots, copilots, vendor AI and retrieval layers must all be in scope

Public model exposure rarely comes only from a direct user prompt. A chatbot may be the visible interface, but the real path can run through a vendor copilot, a document connector, an embedded SaaS assistant, or retrieval augmented generation that silently injects customer data into prompts. Human workarounds matter too, because staff will paste data into whatever tool gets the job done if the approved path is slow or incomplete.

That broader scope is important because the same data can be exposed at several layers at once. A retrieval layer may pull sensitive snippets from an internal repository, the prompt may add more context, and the model provider may retain telemetry or logs depending on the service terms. Even when direct saving is blocked, the bank still has to know whether data was disclosed during processing, because disclosure itself can be the control failure.

The assessment should therefore cover the whole chain: front-end experience, orchestration layer, retrieval source, model endpoint, vendor contract, and user behaviour. Vercel Context.ai OAuth Supply Chain Breach is a useful reminder that shadow AI integrations can expose customer data through unmanaged third-party access, not just through the model prompt itself.

What a defensible bank assessment should conclude

A useful AI risk assessment should end with a clear boundary decision for each use case, not a generic “AI allowed” or “AI blocked” label. The bank should classify what data may enter, what must be transformed first, what may never enter a public model, and which exceptions need compensating controls such as masking, tokenisation, approval gates, or private deployment.

The assessment should also test whether controls are observable. If the bank only relies on policy text or user training, it will not know when staff use public AI through email add-ins, browser extensions, or copy-and-paste into external tools. Evidence of control means the bank can demonstrate monitoring, logging, enforcement, and review across the actual channels where data moves.

Where the risk is customer-impacting, the bar should be high. Banks should assume that model access can multiply exposure across assistants, plugins, and vendor ecosystems, especially when the same customer record is reused in multiple workflows. Palo Alto Networks Salesforce data theft 2025 shows how third-party tokens and connected apps can widen the blast radius around customer data, and Microsoft SAS token exposure 2023 reinforces how over-permissive access paths can turn an integration into a broad data exposure event.

Risk and Threat Considerations

Public AI models create two linked risks for banks: accidental disclosure and uncontrolled reuse. Sensitive customer data can be exposed through prompts, retrieval content, telemetry, cached context, or staff workarounds, and once that data crosses a boundary the bank may lose practical control over where it goes next. The problem is not only the model response, but the entire processing path around it.

Failure mechanism: Weak boundary control, excessive connector access, or poor redaction allows raw or semi-raw customer data to be passed into tools that are outside the bank’s approved trust zone. Vendor retention, logging, and downstream reuse can then preserve or propagate the exposure even when the original user intent was benign.

Impact: Customer confidentiality can be lost, regulatory posture can weaken, and the bank may be unable to prove that its AI controls actually prevented disclosure. At scale, the same failure can affect many customer records across multiple copilots, support tools, and business units.

Practitioner Guidance

What to verify: Verify the exact point where customer data enters any AI workflow, and confirm whether the path is direct, retrieved, transformed, or manually pasted. If you cannot trace the data path from source to model boundary, the use case is not ready for approval.

Decision rule: If the use case can function only when staff paste raw customer information into a public model, treat that as a control gap, not an acceptable operating model. If the same outcome can be achieved with masking, summarisation, or a private deployment, prefer the less exposed option.

What practitioners underestimate: Blocking one interface does not stop data from reaching public models through adjacent tools, browser add-ons, or shadow workflows. The control objective is not to ban AI use, but to make every permitted data path explainable, bounded, and auditable.

Practitioner takeaway: Banks should judge AI risk by whether customer data can be observed, transformed, and contained end to end, because the weakest disclosure path, not the intended one, is what determines exposure.

Related resources from NHI Mgmt Group

• How should security and governance teams reduce the risk of AI outputs becoming inaccurate or harmful when models are trained on broad public data?
• How should organisations evaluate vendor AI risk when third-party products use generative models on customer data?
• Why does AI data accountability matter once models enter core workflows?
• Why do traditional data in motion and data at rest models fail for AI risk?