Third-Party AI Risk Guide: Assessing AI Vendors

Most of the AI in a typical organisation was not built there: it arrives inside SaaS products, through model APIs and as vendor agents connected to business systems, and ordinary vendor assessments miss most of what makes it risky. A SOC 2 report says little about whether your prompts train someone else's model, how long a provider keeps them, which model sits behind a feature this quarter, or what a vendor's agent can do with the OAuth token you granted it. This guide covers scoping, AI-specific risks, questionnaires and assurance schemes, contract clauses, vendor agents as identities, monitoring and deployer duties. It is for third-party risk, security, privacy and procurement teams extending an existing vendor programme to AI.

Key takeaways

AI arrives through four doors: features switched on inside existing SaaS, standalone AI tools, model and API providers, and vendor agents or integrations that act inside your systems. Each needs a different depth of review.
The questions that matter are about data and change: training use, retention, sub-processors and model providers behind the vendor, and how model or feature changes are notified.
Contracts carry the controls you cannot test: no training on your data without consent, defined retention and deletion, sub-processor approval, advance notice of new AI features and model versions, incident notification and exit rights.
Vendor agents and AI integrations are non-human identities: their OAuth grants, API keys and service accounts need owners, least-privilege scopes, monitoring and revocation, exactly like your own.
Responsibility stays with you: the EU AI Act gives deployers their own duties, DORA keeps financial entities "fully responsible" for ICT third parties, and NIST AI RMF GOVERN 6 expects third-party AI policies and contingency plans.

What counts as third-party AI

AI risk in a vendor portfolio is spread across contracts never reviewed as "AI purchases". Split it by how the AI reaches you, since that decides who controls the data flow and the model.

Route	Examples	Who controls the model	Typical gap
AI features in existing SaaS	Summarisation in a CRM, AI search in a ticketing tool, meeting notes in collaboration software	The SaaS vendor, often using a model provider as a sub-processor	Feature enabled by a release, not a purchase, so no review happens
Standalone AI SaaS	Writing assistants, AI note takers, research and coding tools	The AI vendor and its model provider	Bought on a card or through a free tier; often discovered after data has been shared
Model and API providers	Foundation model APIs, hosted open-weight models, embedding and speech APIs	The provider; you control prompts, system design and keys	Treated as infrastructure; retention, regional processing and model deprecation are not tracked
Vendor agents and AI integrations	A vendor chatbot connected to your CRM, an AI assistant with mailbox access, an agent platform calling your APIs	The vendor, acting with credentials you issued	OAuth grants and tokens outlive the project and are broader than the use case
AI used by service providers	Outsourced support, claims handling or development teams using AI on your data	Your supplier's own vendors (fourth parties)	Not disclosed unless the contract requires it

Inventory comes first: combine procurement, SSO and OAuth consent logs, expenses and CASB telemetry, and ask critical vendors whether they have added AI features or AI sub-processors. See the Shadow AI Discovery Guide.

Risks that ordinary vendor reviews miss

Training on customer data: some services use inputs and outputs to improve models unless you opt out, and terms can change. OWASP lists unclear terms leading to "sensitive data being used for model training" as a supply chain risk (LLM03:2025), including a scenario where an operator changes its terms to require an explicit opt-out.
Retention you did not expect: even where training is excluded, providers often keep prompts and outputs for abuse monitoring. One major model provider's enterprise terms, for example, state that API inputs and outputs may be retained "for up to 30 days", with zero data retention only for eligible endpoints on request. Check each provider per product tier.
Hidden sub-processors and model providers: a SaaS vendor's AI feature usually sends data to a model provider, sometimes in another region. Your due diligence on the vendor does not cover the provider unless you ask.
Silent model changes: a model version update can change accuracy, refusal behaviour, output format and cost with no change on your side. NIST AI RMF MANAGE 3.1 expects third-party AI risks to be "regularly monitored".
Output reliability and liability: generated answers can be wrong and your organisation, not the vendor, faces the customer. In Moffatt v. Air Canada (2024), a Canadian tribunal held the airline responsible for incorrect information given by the chatbot on its website. Check how the contract allocates responsibility for outputs and intellectual property claims.
Prompt injection through vendor agents: an agent that reads content from outside your organisation (web forms, emails, tickets) and can act or reply on your behalf can be steered by that content. The ForcedLeak 2025 research showed a web form submission turning a CRM agent into an exfiltration channel. See the Prompt Injection Guide.
Agent and integration access: vendor agents need credentials to your systems, and stolen integration tokens give attackers the same reach. The Salesloft Drift breach showed one chatbot integration's OAuth tokens opening hundreds of Salesforce tenants.
Concentration and lock-in: many AI features depend on a small number of model providers and hyperscalers, so a single outage or policy change can affect several of your vendors at once.

Tiering AI vendors

Not every AI feature needs a deep review. Tier on four factors and let the highest one set the depth of assessment.

Factor	Lower tier	Higher tier
Data sent to the AI	Public or internal, no personal data	Personal, special category, client confidential, regulated, source code, credentials
Autonomy and access	Read-only suggestions a person reviews	Agent that writes, sends, deletes or calls APIs with granted credentials
Decision impact	Internal productivity, easily reversed	Decisions about people (hiring, credit, access to services), customer-facing answers, safety
Business criticality	Convenience feature with a manual fallback	Supports a critical or important function, no quick alternative

Low tier: terms check (training, retention, sub-processors), admin settings verified, inventory entry.
Medium tier: AI questionnaire, data flow and model provider review, contract addendum, named owner.
High tier: plus independent assurance (such as ISO/IEC 42001), prompt injection testing evidence, agent permission review, exit plan and legal review of regulatory roles.

Small vs large organisations: a small organisation can run the lower tiers with a one-page intake form and a standard addendum, saving outside help for the few high-tier vendors. Large organisations should build AI questions into the TPRM platform so an AI feature added to an approved vendor triggers reassessment before renewal.

Due diligence: AI questionnaires and assurance

Use an established question set rather than writing one from scratch, then add questions specific to your use case. The main options differ in depth and in whether evidence is independently checked. See the Security Questionnaires Guide for running questionnaires at scale.

Scheme	What it is	Use it for
CSA AI Controls Matrix (AICM) and AI-CAIQ	AICM v1.1 has 247 control objectives across 18 domains, with guidance split by Model Provider, Application Provider, Orchestrated Service Provider, AI Customer and Cloud Service Provider; the AI-CAIQ v1.1 has 320 questions	Deep assessment of AI vendors and model providers; mapping responsibilities across the AI stack
CSA STAR for AI	Level 1: published AI-CAIQ self-assessment. Level 2 (from 20 November 2025): a Valid-AI-ted AI-CAIQ combined with ISO/IEC 42001 certification	Checking the public registry before sending your own questionnaire
ISO/IEC 42001 certification	Certified AI management system; Annex A includes controls on third-party and customer relationships	Evidence that the vendor governs AI as a system, not proof that a given feature is safe
Shared Assessments SIG (2026)	Standardised vendor questionnaire that now references ISO/IEC 42001 to assess third parties' AI practices	Organisations already using the SIG for TPRM
HECVAT 4 (current version 4.1.6)	Higher education vendor assessment toolkit; version 4 added questions on privacy and AI	Universities and their vendors
SOC 2 Type 2	Attestation on controls over a period	Baseline security; check whether the AI feature and model provider are in the system description

Questions that separate good answers from vague ones

Data use: Are our inputs, outputs, files or feedback used to train, fine-tune or evaluate any model, yours or a provider's? Is that off by default and contractually excluded?
Retention: How long are prompts, outputs, embeddings and logs kept, and where? What does any zero retention option exclude?
Model supply chain: Which model providers and versions power each feature, where are they hosted, and can you switch provider without notice?
Change management: How are model changes and new AI features tested and announced? Can we keep a feature disabled?
Security testing: Do you red team for prompt injection, cross-tenant leakage and unsafe tool use, and will you share a summary?
Agents and access: What can the AI do in our environment, with which credentials and scopes, and are its actions logged separately from user actions?
Isolation: Are our retrieval indexes and fine-tuned models separated from other customers, and does retrieval respect user permissions?

Ask for evidence, not just answers: a data flow diagram, the sub-processor list with AI providers marked, system cards and screenshots of the admin settings for training and retention.

Contract clauses for AI vendors

Most AI controls live outside your environment, so the contract is where they become enforceable. Put these in an AI addendum next to the data processing agreement; if you cannot negotiate, check the standard terms and record gaps as accepted risk.

Clause	What it should say	Anchor
No training on customer data	Inputs and outputs are not used to train or improve any model without explicit written consent, including by sub-processors	US OMB M-25-22 requires agency contracts to "permanently prohibit" training on nonpublic agency data absent consent
Retention and deletion	Maximum retention per data type, deletion on termination with written confirmation, scope of any abuse-monitoring copy	GDPR Art. 28(3); DORA Art. 30(2)(d) access, recovery and return
Location of processing	Regions where model inference and storage occur; advance notice of changes	DORA Art. 30(2)(b)
Sub-processors and model providers	Named list, notice and objection right before adding or changing AI providers	GDPR Art. 28(2) authorisation of sub-processors
New features and model changes	Advance notice before new AI features are enabled; new features off by default for your tenant; notice and testing before model version changes	M-25-22 asks for notice "prior to the integration of new AI enhancements" and performance standards before new versions
Incident notification and assistance	Defined notice period for security and AI incidents affecting your data, and assistance at no or pre-agreed cost	DORA Art. 30(2)(f)
Testing and audit	Right to receive testing results, to test within agreed rules, and to audit for critical services	DORA Art. 30(3)(e); M-25-22 ability "to regularly monitor and evaluate"
Regulatory cooperation	Vendor supplies the information you need to meet deployer duties (instructions for use, logs, documentation)	EU AI Act Art. 25(4) written agreement for high-risk systems; Art. 26
Outputs and IP	Ownership of outputs, any IP indemnity and its exclusions, responsibility for vendor-caused errors	NIST AI RMF GOVERN 6.1 names third-party IP infringement risk
Exit and portability	Data and configuration export in usable formats, transition period, deletion of fine-tuned models and indexes built on your data	DORA Art. 28(8) and Art. 30(3)(f); M-25-22 "data and model portability"

Where identity fits: vendor agents and AI integrations

The fastest route from a vendor compromise to your data is usually not the model but the OAuth grant, API key or service account you gave the vendor's AI to read your CRM, mailbox or ticket queue. Those credentials are non-human identities and need the same governance as your own.

Inventory every grant: list the OAuth apps, connected apps, API keys and service accounts held by AI vendors, with owner, scopes and last use. See the SaaS and OAuth App Governance Guide.
Scope to the use case: a meeting summariser does not need mailbox send rights; a support chatbot does not need full CRM export. Reject vendors that require tenant-wide admin consent for a narrow feature.
Give each vendor agent its own identity: separate service principals or integration users per vendor and use case, so logs show which agent acted and revocation does not break other integrations.
Restrict and monitor token use: where the platform allows, restrict connected apps to the vendor's published IP ranges, set token lifetimes, and alert on bulk queries or exports by integration identities.
Revoke on exit and on incident: offboarding a vendor means revoking grants and rotating keys the same day. The Vercel Context.ai breach showed a stolen token from a forgotten AI app exposing customer secrets.
Treat vendor agents like vendor staff: approval, time limits and review. See the Third-Party Access Guide.

Rolling out a third-party AI risk process

Build the inventory: record each AI tool, feature and integration with owner, data types, model providers and credentials held.
Set the intake gate: add an AI question to procurement, SaaS onboarding and OAuth consent workflows; block user consent to unverified apps for high-risk scopes.
Tier and assess: send the AI-CAIQ or your question set to medium and high tiers, after checking the STAR registry and certificate scope.
Contract: attach the AI addendum at purchase, and at renewal for vendors that have added AI; record accepted gaps with an owner and review date.
Configure: set training opt-out, retention, region, feature toggles limited to a pilot group, and least-privilege integration scopes.
Pilot: enable for a small ring, test for leakage across permission boundaries and prompt injection through external content, and confirm the logs you will need.
Expand with monitoring: widen access in stages while watching usage, integration activity and vendor change notices.
Keep a rollback ready: document how to disable the feature tenant-wide, revoke the vendor's tokens, request deletion and fall back to the manual process; test it once before full rollout.

Ongoing monitoring and metrics

AI vendors change faster than annual reviews. Monitor the signals that indicate your risk has changed between assessments.

Terms and policy changes: watch terms, privacy policies, sub-processor lists and trust centres of high-tier vendors for changes to training or retention language.
Feature and model releases: release notes for AI features enabled by default, model version changes and deprecations.
Integration activity: SaaS and identity provider logs for AI integration identities: new consents, scope changes, unusual data volumes.
Data flow: DLP or CASB events for sensitive data sent to AI services.
Vendor incidents: breach disclosures from AI vendors and their model providers; the DeepSeek database exposure showed chat logs and secrets held by an AI provider leaking through an open database.

Metric	What good looks like
Share of AI vendors and AI features in the inventory with a completed tiered assessment	All high tier, rising coverage of medium tier
High-tier AI vendors with signed no-training, retention and change-notice clauses	Close to 100%, with gaps formally accepted
AI integration grants with a named owner and scopes reviewed in the last 90 days	Rising; unowned grants revoked
Time from vendor AI change notice (feature, model, sub-processor) to review decision	Days, not the next annual review
Unapproved AI tools discovered per month and time to decision	Falling discovery count, short decision time
Vendor offboardings with tokens revoked and deletion confirmed	Every exit, same day revocation

Regulatory angles and framework mapping

Regulators place AI vendor risk on the organisation using the AI. The table summarises where the duties sit; it is not legal advice. See the AI Governance Guide and DORA Guide for the wider programmes.

Framework	Reference	What it means for AI vendors
EU AI Act	Art. 26 (deployers of high-risk systems)	Use per the provider's instructions, assign competent oversight, monitor and inform the provider of risks, keep logs under your control for at least six months. Applies from 2 December 2027 (Annex III) and 2 August 2028 (Annex I)
EU AI Act	Art. 25	You become the provider if you rebrand, substantially modify or repurpose a system so it becomes high-risk; high-risk providers need written agreements with AI component suppliers
EU AI Act	Art. 53(1)(b)	General-purpose model providers must give downstream providers information on "capabilities and limitations": ask your vendor for it
DORA	Art. 28 and 30	Financial entities remain fully responsible; register of information; pre-contract assessment including concentration risk; mandatory contract content and exit strategies
GDPR	Art. 28, Art. 35	Processor terms and sub-processor authorisation; DPIA for high-risk processing
NIST AI RMF 1.0	GOVERN 6.1, 6.2; MAP 4.1, 4.2; MANAGE 3.1, 3.2	Policies for third-party AI risks; "Contingency processes ... to handle failures or incidents in third-party data or AI systems deemed to be high-risk"; third-party risks monitored
ISO/IEC 42001:2023	Annex A, A.10	Third-party and customer relationships in the AI management system
General security frameworks	NIST CSF 2.0 GV.SC; SP 800-53 SA-9, SR-6; CIS Controls v8.1 Control 15; ISO/IEC 27001:2022 A.5.19 to A.5.23	Extend existing supplier controls to AI features, model providers and AI integrations

The NCSC's secure AI development guidelines add a practical test: "require suppliers to adhere to the same standards your own organisation applies to other software".

Common failure patterns

Reviewing the vendor, not the feature: an approved SaaS vendor adds an AI assistant that sends data to a new model provider, and nobody reassesses because the contract has not changed.
Relying on the default setting: training opt-out and retention are configured per tenant or per product tier, and the setting the questionnaire describes is not the one in your tenant.
Grants that outlive the pilot: a trial AI integration keeps tenant-wide scopes after the project ends, with no owner watching its activity.
No exit plan: fine-tuned models, embeddings and retrieval indexes built from your data stay with the vendor after termination because deletion was never specified.
Assuming a certification covers the use case: an ISO/IEC 42001 or SOC 2 certificate whose scope excludes the AI product you are buying.

Practitioner checklist

Keep an AI vendor inventory covering embedded features, AI SaaS, model APIs and vendor agents, with model providers and credentials.
Route new AI use and new AI features in approved tools to review through procurement, SaaS onboarding and OAuth consent.
Tier AI vendors on data, autonomy, decision impact and criticality, and scale assessment depth accordingly.
Use the AI-CAIQ or an equivalent AI question set for medium and high tiers, and check STAR for AI and ISO/IEC 42001 certificate scope first.
Sign an AI addendum: no training without consent, retention and deletion, sub-processor notice, new-feature and model-change notice, incident notice, audit and exit.
Verify tenant settings for training, retention, region and feature enablement against what the contract says.
Govern vendor agents and AI integrations as NHIs: owner, least-privilege scopes, separate identity, monitoring and same-day revocation.
Monitor terms, sub-processor lists, release notes and integration logs between annual reviews.
Document and test a rollback: disable the feature, revoke credentials, confirm deletion, fall back to manual.
Map duties to the EU AI Act deployer role, DORA, GDPR and NIST AI RMF GOVERN 6 where they apply.