How should organisations govern AI systems that do not return the same result twice?

Governing AI When the Output Changes Each Time

Non-deterministic AI should be governed as a decision system with controlled variance, not as a tool that must repeat the same sentence on demand. The governance task is to define acceptable output ranges, route low-confidence or high-impact cases to review, and prevent the model from taking autonomous action when a variable answer could create legal, operational, safety, or customer harm.

The practical distinction is between variation that is acceptable because the task is probabilistic, and variation that is unacceptable because the consequence is material. That means organisations need policy thresholds, review criteria, and exception paths tied to business impact, not just model behaviour.

How to Set Acceptable Variance for Non-Deterministic AI

Governance starts by classifying use cases by impact. For low-risk drafting, summarisation, or triage, some output drift may be acceptable if the result stays within a defined policy envelope. For higher-impact use cases, the organisation should define what counts as equivalent, what counts as ambiguous, and what counts as a fail condition that blocks automation.

Those thresholds should be expressed in operational terms. For example, teams should specify whether the AI may choose among approved options, whether it must cite evidence, whether it can only recommend and not decide, and which fields or decisions require deterministic validation before use.

Where the AI is embedded in a workflow, the control point matters as much as the model itself. A variable recommendation may be tolerable if a human makes the final call, but the same variability is much more serious if the system is writing to a ticketing system, triggering a payment, or changing access without review.

Review, Exception Handling, and Safe Use in Production

Because non-deterministic output cannot be treated as a fixed rule, organisations need explicit review points for unclear, novel, or high-impact cases. The review path should identify who has authority to approve an exception, what evidence they need, and how the final decision is recorded for auditability and later challenge.

Exception handling is especially important when the model is confident but wrong in a way that is hard to spot. A good governance model treats “unusual but plausible” outputs as a separate category from “low confidence” outputs, since both can require escalation even when the text appears polished.

Operationally, the strongest control is to separate recommendation from execution. That allows teams to use NIST AI Risk Management Framework style governance around measured risk, while keeping irreversible actions behind approval gates and policy checks.

For organisations operating under formal AI governance programmes, ISO/IEC 42001:2023 AI Management System Standard is useful because it supports repeatable governance, accountability, and documented control ownership rather than ad hoc judgment. For higher-risk deployments, the EU AI Act regulatory framework is relevant where compliance obligations, oversight, and traceability become part of the deployment decision.

Risk and Threat Considerations

Non-deterministic behaviour creates governance risk when teams assume the model is stable enough to be trusted like business logic. The danger is not only inconsistent wording, but inconsistent recommendations, classifications, or actions that drift across time, prompts, or environments and produce uneven decisions at scale.

Failure mechanism: Ambiguous output is allowed to pass without defined thresholds, so the system makes different decisions for similar inputs, or an automated workflow executes a variable answer that should have been reviewed.

Impact: Organisations can create inconsistent customer outcomes, compliance defects, or unsafe actions, and they may struggle to prove why one result was accepted while another was escalated.

Practitioner Guidance

What to prioritise: Start by ranking AI use cases by consequence, not by model type. If a variable answer can change money, access, safety, or legal position, treat the workflow as controlled decision support and require human review or deterministic validation at the final step.

What to verify: Confirm that every high-impact use case has a written variance policy, a clear exception owner, and an observable stop condition. If teams cannot explain when the system must defer, the governance model is too vague to rely on.

Decision rule: If two different model outputs would still lead to the same approved business outcome, the variance may be acceptable. If different outputs could produce different actions, different customer treatment, or different compliance positions, tighten the control and require review.

Practitioner takeaway: Govern the business effect of the answer, not the illusion of a single correct output; non-deterministic AI is acceptable when the organisation can bound variance, explain exceptions, and block autonomous action where consequences matter.

Related resources from NHI Mgmt Group

• How should security teams govern API keys used for generative AI access?
• How should organisations govern AI systems that can make consequential decisions?
• How should organisations govern access to data used by AI systems?
• How should healthcare organisations govern AI when data comes from many systems?