AI governance is becoming a GRC problem, not just a rules problem

At a glance

What this is: This is a podcast-based analysis of why AI governance is moving from fixed rules to judgment-led GRC in non-deterministic systems.

Why it matters: It matters because IAM and governance teams must decide how to control AI-assisted decisions, delegate authority safely, and account for new forms of AI-driven access and risk.

👉 Read ActiveFence's podcast discussion on AI governance, D&D, and GRC judgment

Context

AI governance is becoming harder to manage because the same input can now produce different outputs, which breaks assumptions built into deterministic control models. In practice, that means policy, review, and approval processes need to handle variance, not just rule compliance, and the issue reaches into identity governance whenever AI systems make or influence access decisions.

The article frames this as a GRC problem rather than a pure technical one. That is directionally correct, because the control gap is not just how to block bad outputs but how to set accountable boundaries for AI systems that can act unpredictably in operational workflows.

Key questions

Q: How should organisations govern AI systems that can make consequential decisions?

A: Organisations should govern consequential AI systems with the same discipline used for high-risk identities: defined ownership, least privilege, logging, approval boundaries, and human override. The critical requirement is to connect model behaviour to real access paths so legal review, security review, and audit evidence all describe the same system.

Q: Why does human-in-the-loop matter for identity and access management?

A: IAM decisions affect who can reach sensitive systems, so errors have direct security consequences. Human-in-the-loop keeps accountability attached to those decisions, making them easier to explain, audit, and correct. It is most valuable when the model lacks context or the decision could create privileged access.

Q: What do enterprises get wrong about AI red teaming maturity?

A: Many teams stop at attack simulation and assume the test itself is the control. In practice, maturity depends on whether findings feed monitoring, policy enforcement, and audit-ready reporting in production. A strong programme reduces risk after the test, not just during the assessment window.

Q: How do organisations know whether AI governance is actually working?

A: AI governance is working when teams can prove that data access, identity permissions, and runtime controls line up with policy in practice. A useful test is whether the organisation can answer who accessed what, through which identity, and whether any out-of-policy movement was blocked or detected in time.

Technical breakdown

Why non-deterministic AI breaks traditional governance models

Traditional governance assumes stable inputs, repeatable outputs, and predictable decision paths. Non-deterministic AI weakens that model because the same prompt, policy, or workflow input can produce different results across sessions. That matters in governance because exceptions, approvals, and controls become harder to audit when the decision path is variable. The risk is not only wrong answers but inconsistent authority, where the system appears to behave within policy while still producing materially different outcomes.

Practical implication: define where AI output may vary and require explicit approval or human review for decisions with security, privacy, or access impact.

Human judgment and AI red teaming as control layers

The article’s strongest technical point is that AI governance cannot rely on automation alone. Red teaming finds failure modes that policy text will miss, while human review catches novel or context-dependent behaviour that scanners cannot score reliably. In AI security, this is especially important where models influence workflows, recommendations, or delegated actions. The control question is less about perfect prevention and more about detecting how the system behaves when it is stressed, misled, or used outside its intended context.

Practical implication: combine automated testing with human-led adversarial review before allowing AI into workflows that can influence access or business decisions.

Why AI governance now overlaps with identity governance

When AI systems recommend, trigger, or automate actions in enterprise workflows, they start to behave like decision actors even when they are not fully autonomous agents. That creates an identity governance problem because the organisation must know what the system is allowed to do, on whose behalf, and under what authority. The boundary matters most when AI is connected to tools, data sources, or privileged workflows through protocols such as MCP, because the scope of authority can expand faster than governance processes update.

Practical implication: inventory AI-connected workflows and treat any AI that can influence access or actions as part of the identity and privilege governance model.

NHI Mgmt Group analysis

AI governance debt is now a board-level risk: organisations are still writing controls for deterministic systems while deploying AI that behaves probabilistically. That gap creates a governance debt problem because review processes, approvals, and audit evidence all assume repeatability. The result is a control environment that looks complete on paper but fails under real operational variance. Practitioners should treat this as a governance design issue, not a documentation issue.

The named concept here is the non-determinism gap: the distance between how governance expects systems to behave and how AI actually behaves at runtime. This gap widens when teams rely on policies alone instead of testing behaviour under stress, ambiguity, or adversarial input. The field needs to stop assuming that a written rule equals a reliable control. Practitioners should design for observed behaviour, not declared intent.

GRC teams need adversarial validation, not just policy review: the article correctly points toward red teaming because it exposes the places where AI behaves outside expected boundaries. That makes adversarial testing a governance function, not a specialist side activity. In identity-adjacent use cases, the same principle applies to any AI that can affect approvals, entitlements, or workflow decisions. Practitioners should build structured testing into governance cycles.

AI systems that influence decisions must be governed like privileged actors: once an AI model can shape access, prioritisation, or operational decisions, the question becomes who controls its authority and how that authority is bounded. That is where AI governance intersects with IAM and PAM. The issue is not whether the model is human-like, but whether it can exercise decision power without adequate oversight. Practitioners should map that authority before expanding deployment.

Judgment will not replace controls, but it will determine whether controls work: the article’s D&D analogy lands because AI governance increasingly depends on contextual decisions that pure rules cannot fully capture. That does not weaken governance, but it changes where value lives: in escalation paths, exception handling, and clear ownership. Practitioners should use judgment as a governed input, not an excuse for control drift.

What this signals

AI governance is converging with identity governance because any system that can trigger actions, approvals, or access decisions now needs explicit authority boundaries. That is where models stop being abstract tooling and become governed participants in enterprise workflows. The practical shift is to document what an AI system can do, who remains accountable, and how exceptions are handled when the output changes.

Non-deterministic control debt: teams are accumulating risk when they write policies for predictable systems but deploy AI that behaves probabilistically. The answer is not more policy text. It is better evidence, stronger escalation paths, and testing that reflects how the system actually behaves at runtime.

For practitioners

• Map AI decision boundaries Document which AI outputs are advisory, which are operational, and which can influence access, approvals, or customer-impacting decisions. Require explicit ownership for each boundary and define escalation paths when the model behaves outside expected patterns.
• Add adversarial testing to governance cycles Use red teaming to test prompt sensitivity, workflow abuse, and inconsistent outputs before AI is allowed into production decisions. Include both automated checks and human review so novel failure modes are not missed by static policy controls.
• Treat AI-connected workflows as identity surfaces Inventory where AI connects to tools, datasets, and enterprise systems, then verify what authority it inherits and whether that authority is excessive. Where the workflow can trigger access or action, bring it under identity and privilege review.
• Define exception handling for non-deterministic behaviour Create a process for cases where AI outputs differ across runs or conflict with policy intent. The process should specify when to block, when to route for human approval, and when to retrain, retriage, or retire the workflow.
• Use governance evidence that reflects runtime behaviour Shift from policy-only evidence to records showing how AI behaved under stress, what tests were run, and how exceptions were handled. That creates audit material that is closer to operational reality than a static control document.

Key takeaways

• AI governance is moving from deterministic rule enforcement to runtime judgment, which makes policy-only controls insufficient.
• The main risk is not just incorrect output, but inconsistent authority and weak auditability when AI decisions vary across runs.
• Practitioners should combine red teaming, human review, and identity-aware authority mapping before expanding AI into operational workflows.

Key terms

• Non-deterministic AI: An AI system that can produce different outputs or action paths from similar inputs. For identity governance, this matters because access scope, review evidence, and control testing cannot assume a fixed execution pattern the way they often can with scripted automation.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
• Adversarial Red-Teaming: Adversarial red-teaming is the practice of actively trying to make a security model fail before it reaches production. The test uses crafted inputs, boundary probes, and mutation strategies to reveal weaknesses that ordinary accuracy testing will not show.
• Identity-Aware Workflow: A workflow that ties service actions to a verified identity, an authorised entitlement, and an auditable decision trail. In identity governance, it matters because speed alone does not prove control. The workflow must preserve who requested access, who approved it, and when it was removed.

What's in the full article

ActiveFence's full podcast discussion covers the conversational detail this post intentionally leaves for the source:

• The D&D-to-governance analogy and the speaker’s reasoning behind it.
• The podcast discussion of why judgment matters when controls cannot fully predict AI output.
• The red-team perspective on balancing human and automated testing.
• The closing discussion on how AI governance shifts when rules behave more like guidance than hard constraints.

👉 The full ActiveFence conversation adds the speaker exchange, governance analogy, and AI red-teaming perspective.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, machine identity security, and secrets management. It helps security and identity practitioners build the control model needed for AI-connected workflows and privileged automation.