AI governance is becoming a GRC problem, not just a rules problem

At a glance

What this is: This is a podcast-driven analysis of why AI governance is shifting from rules-first compliance to GRC judgment because non-deterministic systems do not behave predictably.

Why it matters: It matters because AI, IAM, and security teams now need governance models that can handle uncertain outcomes, human review, and red teaming instead of relying on static policy enforcement alone.

Context

AI governance becomes harder when the same input can produce different outputs, because static rules assume repeatable behaviour. That makes the problem less about writing more policy and more about governing uncertainty, especially where AI systems influence decisions, content, or access workflows.

For identity and security teams, the useful question is not whether rules exist, but whether the organisation can judge model behaviour safely when the system is non-deterministic. That shifts accountability, review, and escalation into the centre of the governance model.

Key questions

Q: How should organisations govern AI systems that do not return the same result twice?

A: Organisations should govern non-deterministic AI with outcome thresholds, human review points, and exception handling instead of assuming one fixed response. The key is to define what acceptable variance looks like, who reviews ambiguous outputs, and when the system is not allowed to act without oversight.

Q: Why does AI governance become a GRC issue instead of just a policy issue?

A: Because policies assume predictable behaviour, while non-deterministic AI can produce different outputs in the same operating context. GRC has to absorb that uncertainty through accountability, escalation, and evidence-based judgment, or the policy becomes a paper control that does not govern real decisions.

Q: What do teams get wrong about AI red teaming coverage?

A: Teams often overfocus on prompt-level failures and miss the wider attack surface. That leads to gaps in retrieval-layer risks, tool call interception, indirect prompt injection, and multi-agent trust boundaries. Another common mistake is treating behavioral testing like traditional penetration testing, where a single fix closes the issue. AI failures are probabilistic, so coverage must be continuous and updated as models, prompts, and corpora change.

Q: Who should own exceptions when AI output is ambiguous or inconsistent?

A: A clearly named business or governance owner should own exceptions, not an unnamed committee or the model team by default. The owner needs authority to approve, reject, and document edge cases, because ambiguous AI outcomes only become manageable when accountability is attached to the decision.

Technical breakdown

Why non-deterministic AI breaks rule-based governance

Deterministic systems behave the same way for the same input, which is why classic control design leans on policy, testing, and expected outcomes. Non-deterministic AI systems can vary their output even when the prompt or context looks unchanged, so the control objective changes from enforcing one fixed result to bounding acceptable behaviour. That creates a GRC problem because governance must now tolerate variance, document acceptable risk, and decide when human review is required. In practice, the organisation is no longer certifying a single action path. It is governing a range of possible outputs and the decision process around them.

Practical implication: Treat AI policy as a decision framework, not a guarantee of uniform outcomes.

Red teaming as a governance control, not just a security test

Red teaming is often treated as adversarial testing, but in AI governance it also becomes a way to expose where policy assumptions fail under realistic behaviour. A useful red team is not only looking for prompt injection or unsafe output. It is testing whether governance can detect drift, ambiguity, and unexpected model responses before the system is relied on operationally. That matters because non-deterministic systems make traditional checklist assurance less reliable. Human-led testing helps find edge cases, while automated checks provide scale. The control is the combination, not one method alone.

Practical implication: Use human and automated red teaming together to test governance assumptions before deployment.

The GRC gap between policy intent and operational judgment

The article's core point is that governance breaks when teams confuse written rules with usable decision support. In AI settings, policy intent may be clear while the operational question is messy: who reviews exceptions, who owns the escalation path, and who signs off when output quality varies by context. That turns AI oversight into a judgment discipline. The strongest programmes will define thresholds, escalation routes, and accountability for ambiguous outcomes rather than relying on long control documents that assume stable behaviour. This is where AI governance starts to resemble enterprise risk management more than compliance administration.

Practical implication: Define who makes exception decisions and how ambiguous AI outcomes are escalated.

NHI Mgmt Group analysis

AI governance debt is now a real programme risk: organisations can accumulate policies faster than they can operationalise judgement. When AI behaviour is non-deterministic, a control library alone does not tell teams when a result is acceptable, when it is not, or who should decide. The governance burden shifts from documenting rules to making repeatable decisions under uncertainty. Practitioners should treat this as a maturity gap, not a documentation gap.

Red teaming belongs inside governance, not outside it: adversarial testing is one of the few ways to see how AI behaves when rules are stressed, bypassed, or interpreted differently than expected. That makes it a governance input as much as a security activity. Where AI influences identity, access, or operational decisions, the test must include how exceptions are judged and recorded. The practical conclusion is that governance needs evidence from behaviour, not just policy text.

Human judgment is becoming the control surface: the article's D&D analogy is useful because it captures the shift from fixed outcomes to bounded discretion. In that environment, the organisation's real control is no longer the rulebook itself but the quality of review, escalation, and accountability around model output. This is especially relevant where AI touches human identity workflows or agentic decisions, because confidence in the system must be earned at runtime, not assumed in advance.

Non-determinism exposes accountability gaps: once outcomes vary, it becomes harder to hide behind process ownership alone. AI governance needs clear responsibility for exceptions, model changes, and high-risk decisions, or the programme will drift into ambiguous oversight. That is a GRC problem first and a technical problem second. Practitioners should map accountability to decision points, not just to policy authorship.

From our research library:

• A Harris Poll survey of more than 300 technology decision-makers found that 86% expect agentic AI to deliver positive ROI, yet fewer than half had AI governance policies in place.
• Read next: Agentic AI Identity Guide

What this signals

AI governance debt: when policy grows faster than operational judgment, teams end up with documentation that cannot explain what to do when model output varies. The practical fix is to move governance closer to runtime decisions, especially where AI affects access, approvals, or other identity-adjacent workflows.

The next phase of AI governance will be measured by exception handling, escalation quality, and evidence from testing rather than by policy count. That means GRC, security, and platform teams need a shared operating model for ambiguous outputs, not separate control narratives.

For practitioners

• Define decision thresholds for AI outputs Set explicit thresholds for when AI output can be accepted, when it needs review, and when it must be rejected. Make the thresholds outcome-based so teams can apply them consistently even when the model behaves differently across similar prompts.
• Build red teaming into governance cadence Schedule human and automated red team exercises as part of governance review, not only as pre-release testing. Use the findings to update escalation paths, exception handling, and accountability for ambiguous outputs.
• Assign clear ownership for AI exceptions Name the role that approves exceptions, records rationale, and signs off on high-risk AI decisions. Without that ownership, non-deterministic behaviour becomes an accountability gap rather than a managed risk.
• Review identity-adjacent AI workflows Prioritise AI workflows that influence access, approvals, or user-facing decisions, because non-deterministic output can create inconsistent governance outcomes. Document where human review is mandatory and where agentic behaviour is not acceptable.

Key takeaways

• Non-deterministic AI makes static rulebooks insufficient, because the same prompt or input can produce different and operationally meaningful outcomes.
• Red teaming is becoming part of governance design, not just a pre-launch security exercise, because it reveals where policy assumptions fail under real behaviour.
• The control point has moved toward judgment, escalation, and accountability, especially in workflows where AI output affects decisions that matter.

Key terms

• Non-deterministic AI: An AI system that can produce different outputs or action paths from similar inputs. For identity governance, this matters because access scope, review evidence, and control testing cannot assume a fixed execution pattern the way they often can with scripted automation.
• API Governance Debt: API governance debt is the accumulation of weak, inconsistent, or missing controls around how APIs are designed, approved, documented, secured, and retired. It creates hidden risk over time. Technically, it includes gaps in authentication, authorization, versioning, inventory, ownership, logging, and policy enforcement that make APIs harder to trust and govern.
• Red Teaming: Red teaming is structured adversarial testing used to find how an AI system fails under realistic misuse or attack conditions. In AI security, it is a discovery method, not a proof of safety, because probabilistic behaviour and changing models prevent any lasting guarantee.
• Exception handling: Exception handling is the process for resolving requests that do not fit standard automation paths. In support operations, exceptions often require human judgment, policy override, or manual approval. When AI is introduced, exception handling becomes a key boundary for what the system can safely automate and what it must defer.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners connect governance decisions to the identity controls their programmes depend on.