They often focus on model capability and ignore governance boundaries. The real issue is whether the vendor can prove what data is used, who can access it, how it is retained, and whether the organisation has a clear opt-in decision before the capability becomes active.
Why This Matters for Security Teams
AI in procurement is often assessed like a feature checklist, but security teams are buying a governance boundary that can expand, persist, and touch sensitive data. The main failure is assuming the vendor’s model capability matters more than the operational reality around access, retention, and operator control. That is the wrong risk lens. NIST’s NIST Cybersecurity Framework 2.0 is useful here because it forces buyers to treat third-party risk, data handling, and accountability as control questions, not marketing claims.
This is especially important when vendors promise copilots, assistants, or embedded agentic features that can read tickets, documents, chats, or code. If a product can ingest internal data, retain it for training, or expose it to subprocessors without a hard opt-in gate, procurement has already crossed into security design. NHIMG’s coverage of the DeepSeek breach shows why exposure is not theoretical: once secrets, chat histories, or backend credentials are introduced into an AI supply chain, the damage is difficult to reverse.
In practice, many security teams discover the boundary problem only after the contract is signed and the capability is already active.
How It Works in Practice
Practical AI procurement review starts with four questions: what data the system can access, who can see it, how long it is retained, and what happens when the buyer says no. Those questions matter more than whether the model scores well in a demo. Security teams should ask for data-flow diagrams, retention schedules, subprocessors, training-use terms, and a clear statement on whether customer content is excluded from model training by default. If the vendor cannot answer those points crisply, the procurement is incomplete.
For technical review, current guidance suggests separating model capability from deployment controls. A vendor can have a strong model and still be unsafe if access is broad or opaque. Buyers should map the AI service to existing governance controls in NIST Cybersecurity Framework 2.0, then verify whether the service supports least privilege, audit logging, customer-managed keys, tenant isolation, and configurable retention. If the product includes tools, connectors, or autonomous actions, the review should also cover approval workflows and revocation paths so that capability does not become silent authority.
- Require written confirmation of training, retention, and deletion terms.
- Ask whether prompts, outputs, and attachments are used for vendor improvement.
- Verify whether administrators can disable high-risk features by default.
- Test whether access can be scoped to a narrow business use case.
- Confirm whether logs are sufficient for incident response and audit.
NHIMG’s research on the LLMjacking: How Attackers Hijack AI Using Compromised NHIs research shows why this matters operationally: exposed credentials can be abused very quickly, so AI procurement must include secret-handling and identity controls, not just model assurances. These controls tend to break down when the AI service is embedded into business workflows without a documented owner because accountability becomes fragmented across procurement, IT, and the vendor.
Common Variations and Edge Cases
Tighter AI procurement review often increases sales friction and slows onboarding, so organisations have to balance speed against the cost of later exposure. That tradeoff is real, especially when business teams want rapid adoption and security is asked to approve “just a pilot.” Best practice is evolving, but there is no universal standard for this yet, which means the buyer must define its own threshold for acceptable data use, retention, and user scope.
Edge cases usually appear when AI is bundled into platforms the organisation already trusts. A productivity suite, CRM, or service desk may introduce AI as a default feature, and the risk is not the underlying model alone but the newly expanded processing boundary. In those cases, procurement should distinguish between optional features and mandatory processing, then require an explicit opt-in decision before any sensitive data is exposed. Vendor claims about “enterprise grade” controls are not enough unless they are testable and contractually enforceable.
Another common exception is regulated data. If the system touches customer records, health data, financial data, or source code, procurement should escalate to legal and privacy review even when the vendor says content is encrypted. Encryption does not answer who can access the plaintext, how long it lives, or whether prompts are reused elsewhere. NHIMG’s DeepSeek breach coverage is a reminder that AI procurement failures often become data-governance failures first and incident-response problems second.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0, NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.2 | AI procurement needs governance, ownership, and risk decisions before activation. |
| NIST CSF 2.0 | PR.DS | The question centers on data retention, use, and access boundaries. |
| NIST AI RMF | AI RMF addresses governance and risk management for AI adoption decisions. |
Define approval authority, data-use limits, and review cadence before enabling any AI capability.
