Most AI incidents are not model failures but governance failures: nobody knew the system existed, nobody owned its output, or nobody checked the vendor. This list ranks the ten governance and risk management failures that most often sit behind AI harm, from missing inventories to oversight that exists only on paper. It is for CISOs, risk and compliance leads and AI governance committees deciding where to put effort first. Technical AI threats are covered in the Top 10 AI Agent Security Risks for CISOs and the Top 10 Shadow AI Risks; this page covers the programme around them. For building the programme itself, see the AI Governance Guide.
The Top 10 at a glance
The table ranks ten AI governance failures, what each looks like in practice and the first control that fixes it. Each is explained below.
| # | Failure | What it looks like | First control |
|---|---|---|---|
| 1 | No AI register with owners | Nobody can list the AI systems in use, their data or their owner | AI register with a named accountable owner per system |
| 2 | Policy without a path to yes | A ban or a principles document, with no approved tools or intake route | Approved tools plus a lightweight intake process |
| 3 | Risk tiering done once | A low-risk use case gains tools, data or users and is never reassessed | Re-tiering triggers on change |
| 4 | Human oversight on paper | A named reviewer with no time, information or authority to intervene | Oversight designed into the workflow, tested |
| 5 | Nobody accountable for AI output | Customers act on wrong answers and the organisation disowns them | Business owner accountable for each AI-facing channel |
| 6 | Thin AI vendor due diligence | AI suppliers onboarded without security, data use or access review | AI-specific vendor questionnaire and contract clauses |
| 7 | AI identities and access ungoverned | Agents, API keys and OAuth grants live outside identity governance | NHI inventory, least privilege and expiry for AI identities |
| 8 | No AI incident route | AI harms are handled ad hoc, outside incident and reporting processes | AI categories in the incident process, with reporting triggers |
| 9 | Working to the wrong rules or dates | Plans built on superseded deadlines or the wrong role (provider vs deployer) | A maintained obligations register per jurisdiction |
| 10 | Approve and forget | No monitoring, re-evaluation or decommissioning after go-live | Post-deployment monitoring and a retirement step |
1. No AI register with owners
The failure: the organisation cannot say which AI systems it builds, buys or uses, what data they touch and who owns them, so every audit, incident and regulatory question starts with a hunt. Fix it: keep an AI register that includes embedded AI features in existing SaaS, not only new projects, with one accountable owner per entry. Use the AI Register and AI Inventory Template.
2. Policy without a path to yes
The failure: a blanket ban or a high-level principles document with no approved tools. Use moves to personal accounts and devices, where it is invisible. The Samsung ChatGPT source code leak is the familiar example of staff using a public tool for work. Fix it: pair the AI Acceptable Use Policy Template with approved enterprise tools and the AI Use Case Intake Form Template, so asking is faster than working around.
3. Risk tiering done once
The failure: a use case approved as low risk later gets tool access, customer data or a public audience, and keeps its original tier. Fix it: define triggers that force re-tiering: new data category, new tools or actions, new user population, model change, or a move from internal to customer-facing. See the AI Use Case Risk Tiering Guide.
4. Human oversight on paper
The failure: the impact assessment names a reviewer, but that person reviews hundreds of outputs a day, cannot see the inputs, or has no authority to stop the system. Fix it: design oversight into the workflow, give reviewers the context and a stop control, measure override rates, and test it. See the EU AI Act Article 14 Guide.
5. Nobody accountable for AI output
The failure: a customer-facing assistant gives a wrong answer and the organisation argues the AI is responsible, not it. In the Air Canada chatbot ruling, a tribunal treated the chatbot as part of the airline's website and held the airline liable for its misstatement. Fix it: make a business owner accountable for every AI-facing channel, ground answers in approved content, and route policy questions such as refunds and eligibility to authoritative sources.
6. Thin AI vendor due diligence
The failure: AI suppliers are onboarded on a demo, without review of their security, data retention, training use or how they access your data. In the McHire case, researchers reached applicant data through a forgotten vendor test administrator account with default credentials outside single sign-on. Fix it: use the AI Vendor Security Questionnaire Template and the AI Procurement Checklist, and require SSO, test account hygiene and breach notice in the contract.
7. AI identities and access ungoverned
The failure: agents, model API keys and OAuth grants to AI apps are created by project teams and never enter identity governance. Grants are rarely reviewed, so they stay valid long after anyone needs them, as the Vercel Context.ai OAuth breach showed. Fix it: register every AI identity with an owner, scope and expiry, and include them in access reviews. See the Identity Governance for AI Agents Guide.
8. No AI incident route
The failure: harmful output, data leakage through a model or a misbehaving agent is treated as a product bug, so it misses severity rating, evidence preservation and any reporting duty. Fix it: add AI categories to the incident process and map reporting triggers, such as serious incident reporting under the EU AI Act for providers of high-risk systems. See the AI Incident Reporting Guide.
9. Working to the wrong rules or dates
The failure: plans built on the original EU AI Act high-risk dates, which the Digital Omnibus on AI moved, or on the wrong role: most organisations are deployers for most systems and providers for a few. Fix it: keep an obligations register per jurisdiction and per role, owned by legal, and review it at each regulatory change. The AI Governance Guide carries the current EU timeline, and the EU AI Act AI Literacy Obligations (Article 4) Guide covers the Article 4 duty already in force.
10. Approve and forget
The failure: governance effort stops at approval. Model updates, drift, new prompts and new integrations go unreviewed, and retired systems keep their data and credentials. Fix it: set monitoring and re-evaluation intervals by risk tier, log model and prompt changes, and add a decommissioning step that deletes data, revokes credentials and closes the register entry. See the AI Model Risk Management Guide.
Mapping to frameworks
| Failure | NIST AI RMF function | EU AI Act reference |
|---|---|---|
| 1. No register | Govern, Map | Article 26 (deployer duties presuppose knowing what you deploy) |
| 2. Policy without a path | Govern | Article 4 (AI literacy) |
| 3. Tiering once | Map, Measure | Article 6 (classification) |
| 4. Oversight on paper | Manage | Articles 14 and 26 |
| 5. Output accountability | Govern | Article 50 (transparency) |
| 6. Vendor diligence | Govern, Map | Article 26 |
| 7. AI identities | Manage | Article 15 (robustness and cybersecurity, for high-risk systems) |
| 8. Incident route | Manage | Article 73 (serious incident reporting, providers) |
| 9. Wrong rules | Govern | Article 113 (application dates, as amended by Regulation (EU) 2026/1744) |
| 10. Approve and forget | Measure, Manage | Article 72 (post-market monitoring, providers) |
NIST released the AI RMF on 26 January 2023 for voluntary use, with the Generative AI Profile (NIST AI 600-1) on 26 July 2024; NIST notes the framework is being revised. See the NIST AI RMF Guide and the ISO/IEC 42001 Guide.
Where to start
- Build the register. List AI systems, embedded features and AI identities with owners (failures 1 and 7). Everything else depends on it.
- Open the front door. Publish approved tools and a short intake form so new use comes to you (failure 2).
- Set re-tiering and monitoring triggers. Tie them to the change process rather than calendar reviews alone (failures 3 and 10).
- Assign output accountability. Name a business owner for every customer-facing AI channel and test the oversight design (failures 4 and 5).
- Close the outside gaps. Add AI clauses to procurement, AI categories to incident response and an obligations register to legal (failures 6, 8 and 9).
Related NHI Mgmt Group resources: AI Governance Guide · AI Use Case Risk Tiering Guide · AI Register and AI Inventory Template · Identity Governance for AI Agents Guide