AI Procurement Checklist: Questions Before Buying AI

Most AI risk enters an organisation through a purchase order, not through a model the organisation built itself. A chatbot added to a SaaS contract, a copilot switched on in an existing licence or an AI API bought by a product team can each start sending business data to a model provider on terms nobody in security or legal has read. This template is a checklist of the questions to answer before signing, grouped into eight numbered sections that procurement, security, privacy, legal and the business owner can work through together. It draws on the EU AI Act, the European model contractual clauses for AI procurement, US OMB memo M-25-22, UK government AI procurement guidance and the NIST AI RMF. It is written for buyers in private and public organisations and is vendor neutral.

How to use this template

Replace text in square brackets, such as [Organisation], [AI governance lead] and [risk tier], with your own terms and thresholds.
Run the checklist when a new AI product or AI feature is proposed, when an existing supplier adds AI to a product you already use, and at each contract renewal.
Scale the depth to the risk tier of the use case (see Implementation guidance). Low-risk tools need sections 1, 3 and 4; high-risk or customer-facing uses need all eight.
Record each answer and the evidence behind it (document, test result, contract clause number) in the [procurement record] and link it to the entry in the AI register. See the AI Register and AI Inventory Template.
Treat a "no" or "unknown" as a finding to be accepted, mitigated or negotiated, not as an automatic rejection. Findings that need contract terms go to the negotiation list; see the AI Contract Clauses Guide.
This checklist does not replace your general supplier security due diligence. Run it alongside your standard questionnaire. See the Security Questionnaires Guide.

1. Use case, owner and risk tier

Is there a named business owner at [Organisation] who is accountable for the AI use, its outcomes and its retirement?
What decisions or actions will the AI output inform or take, and who is affected: staff, customers, applicants, the public?
Has the use case been assigned a [risk tier] under [Organisation]'s tiering method? See the AI Use Case Risk Tiering Guide.
Does the use case fall into a prohibited or high-risk category under the EU AI Act (Article 5, Annex I or Annex III), or into a high-impact category under any sector rule that applies to [Organisation]?
Has an AI impact assessment been started, with go/no-go decision points recorded? See the AI Impact Assessment Template.

2. Regulatory role and obligations

For EU purposes, is [Organisation] a deployer, or will it become a provider? Under EU AI Act Article 25(1), a deployer is treated as the provider of a high-risk AI system if it puts its own name or trademark on it, makes a substantial modification, or changes the intended purpose of a system so that it becomes high-risk.
If [Organisation] will fine-tune, rebrand or repurpose the product, has legal confirmed the resulting role and obligations before purchase?
For a high-risk system, can [Organisation] meet the Article 26 deployer duties with what the supplier provides: use in line with the instructions for use, assigned human oversight, relevant and representative input data where [Organisation] controls it, monitoring, informing the provider of serious incidents, and keeping logs under its control for at least six months?
Will workers or their representatives need to be informed before use at work (Article 26(7)), and are affected individuals to be told that a high-risk system assists decisions about them (Article 26(11))?
Has the timeline been checked? The AI Omnibus, published in the Official Journal on 24 July 2026, moved the Annex III high-risk obligations to 2 December 2027 and the Annex I obligations to 2 August 2028. Contracts signed now will run into those dates.
If the product is built on a general-purpose AI model, does the supplier have the information the model provider must give downstream providers under Article 53(1)(b)?
Does a data protection impact assessment apply (GDPR Article 35), and will the supplier's Article 13 instructions for use support it, as Article 26(9) intends?
For US federal buyers: has the solicitation been checked against OMB M-25-22 and, for large language models, M-26-04?

3. Data use, training and residency

Which data categories will be sent to the product: prompts, uploaded files, connected repositories, customer records, special category data, source code, credentials?
Does the supplier, or any model provider it uses, train or fine-tune models on [Organisation]'s inputs or outputs? Is the default opt-in or opt-out, and is the answer in the contract rather than only in a settings page or privacy policy? OWASP LLM03 lists unclear terms and privacy policies, and operators changing them to require an opt-out, as supply chain risks.
How long are prompts, outputs, embeddings, logs and abuse-monitoring copies retained, and can retention be reduced or set to zero for [data classification]?
Where is data processed and stored, including inference endpoints, and which sub-processors are involved? Is there a GDPR Article 28 processor agreement with prior authorisation or objection rights for new sub-processors?
Are retrieval indexes, vector stores and fine-tuned models built from [Organisation] data deleted or returned at exit, with written confirmation?
Who owns outputs, and does the supplier claim any licence to use them? See the AI and GDPR Guide.

4. Security, identity and access

Does the product support SSO with SAML or OpenID Connect, enforce [Organisation]'s MFA policy, and support SCIM for provisioning and deprovisioning?
How does the product authenticate to [Organisation] systems: OAuth grants, API keys, service accounts, personal access tokens? What scopes does it request, and can they be reduced to least privilege?
Can API keys be scoped, rotated and revoked by [Organisation], and are they stored in the supplier's secrets management rather than in configuration? See the Third-Party Access Guide.
If the product includes AI agents, does each agent act under its own identity with defined permissions, and can high-impact actions require human approval?
Has the supplier tested for prompt injection, sensitive information disclosure and excessive agency (OWASP Top 10 for LLM Applications 2025, LLM01, LLM02 and LLM06), and will it share a summary of results?
Which audit logs are available to [Organisation] (sign-ins, admin changes, prompts and responses, tool calls, data exports), in what format, and for how long? Can they be streamed to [SIEM]?
What independent assurance exists: ISO/IEC 27001 certificate scope, SOC 2 Type II report period, ISO/IEC 42001 certification, penetration test summary?
What is the supplier's breach notification commitment in hours, and does it cover incidents at its model providers and sub-processors?

5. Model, performance and change management

Which models and versions does the product use, who provides them, and is the model hosted by the supplier, a cloud provider or a third-party API?
What accuracy, robustness and cybersecurity levels does the supplier state, with which metrics? EU AI Act Article 13(3)(b) requires these in the instructions for use of high-risk systems.
Can [Organisation] test the product on its own representative data before selection? OMB M-25-22 asks agencies to test proposed solutions and allow periodic independent evaluation with agency data the vendor cannot access.
How will [Organisation] be told before a model is replaced, retrained or materially changed, and how much notice is given? Can [Organisation] pin a version, or roll back?
What usage limits, rate limits and cost controls exist, so that a runaway workload or stolen key cannot generate unbounded spend? See the LLMjacking breach for how stolen AI API keys are abused.

6. Transparency, documentation and supply chain

Will the supplier provide documentation of capabilities, limitations, intended and precluded uses, and known failure modes (a model card or equivalent)?
Can the supplier list its AI supply chain: foundation models, datasets, open-source libraries, plug-ins and hosting, ideally as an AI-BOM? See the AI Supply Chain and AI-BOM Guide.
How does the supplier vet and monitor the models and packages it depends on (MITRE ATLAS AML.T0010 AI Supply Chain Compromise, AML.T0020 Poison Training Data)?
What are the licence terms of the models and datasets involved, and do any restrict [Organisation]'s intended commercial use?
Does the supplier offer an IP or copyright indemnity for outputs, and what conditions and exclusions apply?

7. Human oversight, monitoring and incidents

Which oversight measures does the product provide: approval steps, confidence indicators, explanations, the ability to override or stop?
Who at [Organisation] will perform oversight, and do they have the competence, training and authority Article 26(2) expects?
Which metrics (sampled accuracy, override rate, complaints, incidents) will be monitored after go-live, and by whom?
Is there a channel for reporting harmful or incorrect outputs to the supplier, and a committed response time?
Is there a contingency plan if the AI system or its model provider fails? NIST AI RMF GOVERN 6.2 expects contingency processes for failures or incidents in high-risk third-party data or AI systems.

8. Commercial terms, exit and review

Do the contract terms cover the findings from sections 2 to 7: no training on [Organisation] data, change notification, audit and information rights, incident notification, liability caps and indemnities?
Can the supplier change its terms of service or acceptable use policy unilaterally, and does [Organisation] have the right to terminate if a change is material?
What happens at exit: export of data, prompts, configurations, fine-tuned models and logs in a usable format, deletion certificates, transition assistance? NIST CSF 2.0 GV.SC-10 calls for supply chain plans to cover activities after a relationship ends.
Has the risk of lock-in been assessed, including dependence on one model provider? UK government guidance warns against black-box algorithms and lock-in.
When is the next review: [renewal date], [annual review date], or earlier on a material model change?

Implementation guidance

Use the risk tier to decide how much of the checklist to apply and who must sign off. The tiers below are examples; align them to your tiering method.

Risk tier (example)	Typical use	Sections required	Sign-off
Low	Drafting aid with no confidential data, no system access	1, 3, 4	Business owner, [Security team]
Medium	Internal data, integrations via OAuth or API keys, staff-facing decisions	1 to 6, 8	Plus [Privacy officer], [Procurement]
High	Customer-facing, decisions about individuals, EU AI Act high-risk, agents with write access	All eight, plus testing on own data	Plus [AI governance body], [Legal]

Embed the checklist in intake. Add a mandatory "does this product or feature use AI?" question to purchase requests, software requests and renewals, so the checklist triggers without relying on requesters to know about it.
Prefer evidence over assertion. Ask for contract clause numbers, documentation and test results rather than yes/no answers. A settings-page statement about training use is weaker than a contractual commitment.
Test before you commit. For medium and high tiers, run a proof of concept on representative data in a sandbox with test credentials only, and check logs, access scopes and retention settings in practice.
Plan the rollback. Before go-live, document how to revoke OAuth grants and API keys, disable the feature for all users, and retrieve or delete data if the deployment is stopped.
Re-run on change. Re-run the relevant sections when the supplier announces a new model, new AI features, new sub-processors or changed terms.

Measure it: track the share of AI purchases that passed through the checklist, the number of AI features discovered outside procurement, open findings by age, the share of AI contracts with a no-training clause, and the share of AI suppliers reviewed in the last 12 months.

Checklist section	NIST CSF 2.0	NIST SP 800-53 Rev. 5	NIST AI RMF
1, 2 Use case and obligations	GV.SC-06	SA-4 Acquisition Process	GOVERN 6.1, MAP 4.1
3, 4 Data and security	GV.SC-05	SA-9 External System Services, SR-3	MAP 4.2
5, 6 Model and supply chain	GV.SC-07	SR-5, SR-6 Supplier Assessments and Reviews	MANAGE 3.1, MANAGE 3.2
7 Oversight and incidents	GV.SC-07	SR-8 Notification Agreements	GOVERN 6.2
8 Exit and review	GV.SC-10	SA-4	MANAGE 3.1

Common mistakes: checking only new purchases while existing suppliers switch on AI features in place; accepting a privacy policy as evidence of no training use; approving an integration without reviewing the OAuth scopes it requests (the Vercel Context.ai OAuth breach began with a third-party AI app's token); and never re-running the checklist after a model change.

For wider supplier risk, see the Third-Party AI Risk Guide. For choosing a platform to manage AI governance, see the AI Governance Platform Buyer's Guide.

Sources used in this checklist:

This template is provided for guidance and must be adapted to your organisation. It is not legal advice. Related NHI Mgmt Group resources: AI Governance Guide · Third-Party Risk Management Guide · EU AI Act Compliance Guide · AI Security Architecture Reference Guide

AI Procurement Checklist: Questions to Ask Before Buying AI