How should organisations govern AI systems when legacy risk frameworks are too slow?

Why This Matters for Security Teams

Legacy risk frameworks were built for periodic review, not for AI systems that change after deployment through new prompts, updated context sources, model refreshes, or shifts in user behaviour. That gap matters because AI risk is often operational rather than purely theoretical: unsafe outputs, data leakage, policy drift, and model misuse can appear between review cycles. Current guidance increasingly treats AI governance as a continuous control problem, not a one-time approval event. The NIST Cybersecurity Framework 2.0 is useful here because it reinforces ongoing governance, risk management, and outcome-based accountability rather than relying only on annual assessments.

Security teams also need to distinguish model risk from surrounding system risk. An approved model can still become unsafe if retrieval sources degrade, tool permissions expand, or downstream integrations change. The real failure mode is assuming the model is the only object under control, when the operating environment is what usually creates exposure. In practice, many security teams encounter AI governance gaps only after unsafe outputs, data leakage, or unauthorized tool use has already occurred, rather than through intentional lifecycle monitoring.

How It Works in Practice

Continuous AI governance works by attaching controls to the model lifecycle, not just the initial sign-off. That means defining who owns the model, what evidence is required, how exceptions are approved, and which events trigger re-review. A workable approach combines policy, telemetry, and change management so that new prompts, new data sources, model updates, and tool integrations all produce audit-ready signals. The goal is not to slow delivery, but to make changes visible before they become incidents.

Operationally, teams should treat AI systems like high-change services. Key control points include:

• pre-deployment review of training data provenance, evaluation results, and intended use
• runtime monitoring for harmful outputs, prompt injection attempts, and unusual tool calls
• access and privilege checks for agents, connectors, and service accounts
• logging that captures prompts, responses, retrieval sources, and policy decisions
• incident workflows that can pause, roll back, or constrain a model quickly

Framework mapping is strongest when governance is tied to existing control families. The NIST SP 800-53 Rev 5 Security and Privacy Controls helps translate AI oversight into access, audit, configuration, and incident response requirements. For AI-specific threat behaviour, organisations should also align monitoring to known attack patterns such as prompt injection, model evasion, and data extraction. Where agentic systems can call tools or access enterprise data, governance must include identity and permission boundaries, because the model is only as safe as the authority it is given. These controls tend to break down when AI tools are embedded in legacy business workflows with no clear asset owner because accountability and evidence collection become fragmented.

Common Variations and Edge Cases

Tighter governance often increases review overhead, requiring organisations to balance faster experimentation against stronger assurance. That tradeoff becomes sharper when AI is used in customer-facing, regulated, or high-impact decisions, where the tolerance for silent drift is much lower.

There is no universal standard for this yet, so best practice is evolving. Some organisations use tiered governance, where low-risk internal assistants get lightweight monitoring and higher-risk systems require formal approvals, red-team testing, and periodic recertification. Others add human-in-the-loop review only for specific outputs, such as financial decisions or sensitive content. The right model depends on how much autonomy the system has and whether its outputs can trigger real-world actions.

Edge cases also matter. A frozen model can still become risky if its retrieval corpus changes. A well-tested model can still fail if a downstream agent gains broader tool permissions. In hybrid environments, AI governance should include identity, secrets, and access reviews for connectors, service principals, and non-human identities that operate the system. For teams building on autonomous agents, governance should also track whether the agent can meaningfully change state, not just generate text. The important question is not whether the model was once approved, but whether its current operating conditions still match the approval assumptions.

Related resources from NHI Mgmt Group

• How should organisations govern AI systems that can make consequential decisions?
• How should organisations govern access to data used by AI systems?
• How should healthcare organisations govern AI when data comes from many systems?
• Why do chat-based AI systems create new identity risk for organisations?