Subscribe to the Non-Human & AI Identity Journal

What signals show that an AI risk repository is not working?

The clearest signals are duplicated risk entries, missing owners, stale assessments, and no linkage to access controls or remediation. If teams cannot answer which AI systems touch sensitive data or which permissions they hold, the repository is not providing operational visibility.

Why This Matters for Security Teams

An AI risk repository is only useful if it supports decisions about governance, access, and remediation. When it devolves into a static register, it stops answering the questions that matter most: what AI systems exist, who owns them, what data they touch, and which controls are compensating for known risk. That gap creates blind spots for security, privacy, and model governance. The NIST AI Risk Management Framework is helpful here because it treats AI risk as a lifecycle discipline, not a filing exercise.

Practitioners often miss the early warning signs because the repository still looks complete on paper. Entries exist, fields are populated, and dashboards render, yet the content is not trusted or used in day-to-day operations. That usually means the repository has lost its connection to change management, access reviews, and incident response. If it does not reflect new models, new data sources, or new permissions quickly, it becomes a compliance artifact rather than a control surface. In practice, many security teams encounter the failure only after a model has already been deployed with unclear ownership and undocumented data access, rather than through intentional governance.

How It Works in Practice

A functioning AI risk repository should behave like an operational inventory, not a document archive. At minimum, each AI system entry should identify the business owner, technical owner, model type, deployment environment, data classification, third-party dependencies, review date, and linked controls. That makes it possible to trace risk from discovery through remediation and to tie the record to authoritative governance processes such as the NIST Cybersecurity Framework 2.0.

Common signs of a healthy repository include:

  • Risk entries are unique, current, and versioned alongside the AI system lifecycle.
  • Owners are assigned and accountable for acceptance, treatment, and review.
  • Material changes trigger reassessment, not just a manual annual update.
  • Controls link to concrete actions such as access restriction, logging, testing, and approval gates.
  • High-risk systems are visible to security, legal, privacy, and model governance stakeholders.

Operationally, the repository should also connect to evidence. For example, if a model can access sensitive data, the record should show whether that access is mediated, what review approved it, and whether the current permission set still matches the intended use. This is where NHI governance starts to matter: AI systems, service accounts, and tool-connected agents can accumulate permissions faster than teams update the register. Guidance is still evolving on how to represent agentic systems cleanly, but the principle remains the same: records must reflect actual execution authority, not assumed intent. These controls tend to break down in fast-moving MLOps environments because deployments, prompts, datasets, and service permissions change faster than review workflows.

Common Variations and Edge Cases

Tighter repository governance often increases administrative overhead, requiring organisations to balance faster AI delivery against stronger risk traceability. That tradeoff is real, especially where teams run many short-lived experiments or use external model APIs. Current guidance suggests that not every proof-of-concept needs the same depth of treatment as a production system, but there is no universal standard for that threshold yet.

The edge cases are usually where repositories fail most visibly. A central register may work for a small number of enterprise models, but it often breaks down when AI is embedded in SaaS products, spread across business units, or created by shadow IT. Another common failure mode is treating third-party model services as “vendor risk” only, when the real issue is local accountability for prompts, outputs, and data flows. The NIST Cyber AI Profile (IR 8596) is useful where AI is part of the security stack, because it helps frame cyber-related AI risk in operational terms rather than abstract policy language.

Where personal data, regulated decisions, or high-impact automation are involved, the repository should also be checked against management-system discipline such as ISO/IEC 42001:2023 AI Management System Standard. The main warning sign is simple: if stakeholders use the repository only during audits, it is not working as an operational control.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

NIST AI RMF, NIST CSF 2.0, NIST AI 600-1, NIST IR 8596 and ISO-IEC-42001 set the governance and control requirements practitioners need to meet.

Framework Control / Reference Relevance
NIST AI RMF AI risk repositories should support lifecycle governance, not static recordkeeping.
NIST CSF 2.0 GV.RM Risk management governance is the core test for whether the repository is operational.
NIST AI 600-1 Generative AI records need current context on use, controls, and exposure.
NIST IR 8596 Cyber AI profiles help assess whether AI used in security workflows is tracked correctly.
ISO-IEC-42001 AI management systems require controlled, auditable risk records and governance.

Define ownership, review cadence, and risk acceptance so the repository drives security decisions.