Who should decide whether a high-risk AI model is allowed in enterprise use?

Why This Matters for Security Teams

Allowing a high-risk AI model into enterprise use is not just a technical deployment decision. It creates exposure across data handling, model behaviour, access control, legal accountability, and incident response. A governance group is needed because the approval question spans multiple risk domains at once, including privacy, security, compliance, and business continuity. The NIST Cybersecurity Framework 2.0 is useful here because it reinforces that governance, risk management, and control ownership must be explicit rather than implied.

Practitioners often make the mistake of treating model approval as a procurement sign-off or a one-time architecture review. That misses the operational reality that AI risk changes with prompts, retrieval sources, integrations, and user groups. A high-risk model can be acceptable in one workflow and unsafe in another if it has access to regulated data, internal secrets, or action-bearing tools. Security teams also need to distinguish between model capability and deployment context, because the same model may carry very different risk depending on whether it is used for summarisation, decision support, or autonomous action.

In practice, many security teams encounter model misuse only after data exposure or uncontrolled output has already occurred, rather than through intentional pre-approval.

How It Works in Practice

Effective approval starts with a formal intake that defines the model’s purpose, data inputs, outputs, and downstream actions. The governance group should include security, privacy, legal, risk, and the business owner, with clear authority to approve, reject, or conditionally approve use. Conditional approval is often the right outcome when controls can reduce exposure to an acceptable level, but current guidance suggests those conditions must be specific and testable.

At a minimum, the review should cover data classification, retention, logging, human oversight, and whether the model can reach tools, APIs, or production systems. If the model uses retrieval-augmented generation, the team should also review source trust, document provenance, and prompt injection exposure. This is where identity and privilege matter: if an AI agent can act through service accounts or inherited permissions, the approval must include those identities as part of the risk boundary, not as an afterthought.

• Define allowed data types and prohibited data types before rollout.
• Restrict model routing so sensitive prompts do not leave approved boundaries.
• Require runtime guardrails for outputs, tool use, and escalation paths.
• Log prompts, responses, policy decisions, and administrative changes for auditability.
• Test for prompt injection, data leakage, and unsafe action execution before production use.

Where the model is integrated into workflows with payment, HR, customer, or regulated data, approval should also map to the organisation’s incident response and control monitoring process. That is consistent with the broader control logic used in AI governance profiles such as the NIST Cybersecurity Framework 2.0, even though the framework is not AI-specific. These controls tend to break down when model access is embedded in shadow IT deployments because ownership, logging, and routing decisions are never centralised.

Common Variations and Edge Cases

Tighter approval often increases friction and slows experimentation, requiring organisations to balance speed against review depth. That tradeoff is real, especially for low-risk internal use cases where a full governance board may be too heavy. Best practice is evolving, but there is no universal standard for when a model can be approved by a product owner versus when it requires cross-functional sign-off.

Edge cases usually appear when the model is not autonomous on its own but becomes high risk once paired with retrieval, plugins, or agentic workflows. A model that only drafts text may be manageable under a lighter review, while the same model connected to ticketing, finance, or identity systems needs stricter controls. Another common exception is vendor-hosted AI: even if the enterprise does not train the model, it still owns the decision to expose internal data and users to it. For that reason, approval should cover both the model and the operating environment.

Where regulated or personal data is involved, the governance group should record the rationale for approval, the compensating controls, and the conditions for revocation. That record is essential because the most difficult failures are rarely model-only failures; they usually emerge when a model, a dataset, and a privileged workflow intersect without a clear owner.

Related resources from NHI Mgmt Group

• How should financial institutions govern explainable AI in high-risk use cases?
• How do IAM teams decide whether an AI use case needs new controls or better NHI hygiene?
• How do organisations decide whether an AI agent should be allowed to act autonomously?
• How should security teams govern AI use when the same model creates different risk in different contexts?