What do organisations get wrong about third-party AI risk reviews?

Why This Matters for Security Teams

Third-party AI reviews go wrong when they are treated like a procurement checkbox instead of a living risk decision. AI-enabled services can alter prompts, model routing, retention, embedded tools, and downstream data exposure after contract signature. That means the original approval can become stale before the first quarterly review. Guidance from the NIST AI Risk Management Framework and the OWASP Non-Human Identity Top 10 both point to runtime risk, not just initial assurance, because machine behaviour and machine credentials evolve differently from human users.

Security teams also underestimate how quickly a vendor dependency can become an identity problem. A service may be approved for low-risk content analysis, then later gain access to production tickets, internal knowledge bases, or agent-to-agent workflows that expand blast radius without a new review. In the field, this shows up as “approved” vendors quietly becoming privileged integrations.

NHIMG research on The 2024 ESG Report: Managing Non-Human Identities found that 72% of organisations have experienced or suspect a non-human identity breach, which underscores how often trust decisions are broader and weaker than teams assume. In practice, many security teams discover AI vendor drift only after a data-sharing or access change has already propagated through production.

How It Works in Practice

An effective third-party AI review starts by classifying what the service actually does, not what the sales material says it does. The review should separate three risk surfaces: data handling, decision influence, and dependency chains. Data handling covers whether prompts, files, logs, embeddings, or outputs are stored, reused, or sent to subprocessors. Decision influence asks whether the service recommends, filters, ranks, or automates outcomes that affect customers or employees. Dependency chains cover model providers, plugin ecosystems, API calls, and any autonomous agent or workflow the vendor has wired in.

Practitioners should make the approval conditional on measurable controls, then re-evaluate on change events. Useful triggers include new model versions, new subprocessors, expanded retention, new tool access, new regions, and any shift from assistive to autonomous behaviour. That aligns with the runtime posture promoted by the NIST Cyber AI Profile, which emphasizes continuous governance rather than static assurance.

• Require a current data-flow map that shows inputs, outputs, storage, and subprocessors.
• Ask whether the vendor can change models or tools without notice, and whether notice is contractually required.
• Review whether logs may contain secrets, regulated data, or prompt content.
• Validate whether the service can trigger actions through email, tickets, code, or API calls.
• Set a review cadence tied to material changes, not just annual procurement cycles.

NHIMG incident research such as the Klue OAuth Supply Chain Breach shows why dependency mapping matters: third-party integrations can create broad exposure even when the core service seems benign. These controls tend to break down in fast-moving SaaS environments where vendors can change subprocessors, integrations, or model behaviour without synchronised customer review.

Common Variations and Edge Cases

Tighter review requirements often increase procurement friction and slow product adoption, so organisations have to balance speed against confidence. That tradeoff becomes sharper when the service is low-risk in isolation but high-risk once connected to internal data, ticketing, or agent workflows.

There is no universal standard yet for how often AI vendor reviews should recur, but current guidance suggests using change-based reassessment rather than fixed annual attestation alone. For high-impact use cases, a vendor that can modify prompts, fine-tuning, or tool permissions without customer approval should be treated as a live risk rather than a signed-off exception. The NIST Cybersecurity Framework 2.0 is useful here because it frames third-party governance as an ongoing identification and monitoring problem, not a one-time acceptance decision.

Edge cases often include open-source wrappers, embedded copilots, and agent platforms that appear low risk because they are “just interface layers.” In reality, those layers can become the control point for prompts, credentials, or approvals. NHIMG coverage of the Mastra npm Supply Chain Attack shows how quickly AI-related dependencies can expand the attack surface once trust is extended beyond the original vendor. Best practice is evolving, but the safest posture is to review the vendor whenever the service’s behaviour, data paths, or connected identities change.

Related resources from NHI Mgmt Group

• What do organisations get wrong about AI-enabled third-party apps?
• What do insurers get wrong about third-party AI risk?
• What do organisations get wrong about third-party risk management?
• What do organisations get wrong about AI-driven cyber risk?