Explainable AI in identity governance: why 99% accuracy falls short

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Nexis's blog post: Explainable AI in IAM needs transparent governance workflows, not just high model accuracy.

Why it matters: IAM, NHI, and autonomous governance teams need recommendations they can verify, because access, recertification, and SoD decisions must remain defensible under audit and operational scrutiny.

👉 Read Nexis's analysis of explainable AI in identity governance

Context

Explainable AI in identity governance is about making AI recommendations traceable, so practitioners can see why a role, entitlement, or risk flag was suggested before they act on it. The core governance problem is not whether a model can predict well, but whether its output can be explained, challenged, and recorded inside an IAM process.

That matters because identity governance is a control plane for access, segregation of duties, recertification, and documentation. When the workflow must stand up to audit or operational review, black-box recommendations create a governance gap even if the underlying model is statistically strong.

The article is focused on IAM and governance workflows, with MCP and LLMs appearing as enabling technologies rather than the subject themselves. The practical question is how to preserve deterministic governance when AI is inserted into decisions that still need human accountability.

Key questions

Q: When does explainable AI become necessary in identity governance?

A: Explainable AI becomes necessary when the output affects access, recertification, segregation of duties, or governance documentation that must be audited later. If a reviewer cannot understand why the system recommended a role, flag, or approval, the result may be statistically useful but operationally weak. Governance teams need reasoning they can verify, not just a score they can trust.

Q: Why can 99% accuracy still fail IAM governance needs?

A: Because IAM decisions are judged by the exception, not the average. A model that is usually correct can still approve the wrong entitlement, miss a segregation-of-duties conflict, or misclassify a risky role in the one case that matters for audit or exposure. Identity governance needs deterministic reasoning and reproducible evidence, not just strong aggregate performance.

Q: What do security teams get wrong about access reviews?

A: Teams often treat access reviews as proof of control, when they are really only a point-in-time check. If reviewers cannot see current activity and business context, they may approve access that is technically valid but operationally obsolete. The better test is whether the governance model can explain why access still exists.

Q: How should organisations govern MCP servers in identity programmes?

A: Organisations should inventory MCP servers like any other machine-facing access layer, then recertify exposed capabilities, remove unused tools, and keep state-changing actions tightly scoped. The goal is to reduce what an agent can even see as callable. That is a governance control, not just a software design preference.

Technical breakdown

Why 99% accuracy is not enough for identity decisions

Identity governance decisions are not judged by average performance alone. A recommendation engine can be 99% accurate and still fail at the exact moment an unusual entitlement, conflicting role, or compliance exception matters. In IAM, a single wrong approval can create audit exposure, segregation-of-duties conflict, or excessive privilege. Explainable AI addresses that by exposing the reasoning behind the recommendation, not just the output. That makes the model usable inside governance workflows where practitioners need repeatability, reviewability, and a defensible trail.

Practical implication: evaluate AI-assisted IAM features on explainability and auditability, not just accuracy metrics.

How explainable AI supports access reviews and role mining

In access reviews, explainable AI can surface outliers, changed users, and conflicting entitlements so reviewers spend time where risk is highest. In role mining and role management, it can show why a suggested role structure does or does not match the observed permissions pattern. The technical value is not autonomy, but triage and context. Deterministic governance still controls the final decision, while the AI reduces noise and highlights anomalies. That is a better fit for identity workflows than a system that simply recommends approvals without evidence.

Practical implication: use AI to prioritise review effort and explain anomalies, but keep approval authority in the governance workflow.

Why MCP and LLMs change the governance boundary

The article also points to MCP and BYO LLM as architectural shifts. MCP makes identity data queryable by other tools and agents without custom integrations, while BYO LLM keeps the model within enterprise control. That expands access to identity context, but it also widens the governance boundary because more systems can ask questions of authoritative IAM data. If those queries are not traceable and policy-bound, the organisation can multiply decision surfaces faster than it improves control.

Practical implication: govern identity data exposure to tools and agents as part of the IAM control design, not as a separate integration task.

NHI Mgmt Group analysis

Accuracy is not the same as governance fitness: Identity decisions fail when the 1% exception matters, not when the average model score looks weak. IAM programs need outputs that can be inspected, challenged, and reproduced because access reviews and role governance are accountability exercises, not prediction contests. The implication is that model quality alone cannot justify use in approval workflows.

Explainability is the control, not a feature request: In identity governance, the reasoning behind a recommendation is part of the control evidence. If a reviewer cannot see why an entitlement was flagged or a role was suggested, the workflow becomes harder to defend under audit and easier to rubber-stamp in practice. Practitioners should treat explanation quality as a requirement for operational acceptance.

Machine learning helps with triage, but deterministic governance still owns the decision: The strongest use case in IAM is surfacing outliers, inconsistent attributes, and review candidates so humans can focus attention. That preserves the distinction between assistance and authority, which is the right line for access governance, recertification, and segregation of duties.

Identity governance data is becoming an enterprise query layer: MCP and LLM connectivity turn IAM data into something other systems can interrogate directly. That raises the value of the governance record, but it also means the organisation must decide which consumers may ask what, and under what policy conditions. Practitioners should now manage identity data as a controlled enterprise intelligence asset, not a closed admin console.

Transparent AI exposes a named concept that matters here: auditable recommendation logic: This is the practical standard that separates explainable governance AI from opaque automation. A recommendation that cannot show the pattern, rule, or attribute set behind it is not ready for regulated identity decisions. Teams should test for auditable recommendation logic before they let AI touch recertification or SoD workflows.

From our research library:

• Nearly 60% of IT leaders cite restrictive cost and complexity as a weakness of legacy identity governance, according to the 2025 State of Identity Governance Report.
• Read next: IGA Buyer's Guide

What this signals

Auditable recommendation logic: Identity governance teams should now judge AI features by whether they can explain a recommendation in a way that survives audit, not by whether the model is impressive in demos. Once AI touches access reviews or role design, the explanation trail becomes part of the control itself.

That shifts procurement and architecture conversations away from generic AI capability and toward evidence quality, reviewer workflow design, and policy boundaries around identity data. Teams that cannot inspect the why behind the recommendation will struggle to defend the what later.

For practitioners

• Define an explainability threshold for IAM use cases Require every AI-assisted access review, role suggestion, or risk flag to show the rule, pattern, or attribute set behind the recommendation before it can influence a decision.
• Separate triage support from decision authority Use AI to surface outliers and prioritise reviewer attention, but keep approval, rejection, and remediation decisions inside the governed workflow.
• Treat MCP exposure as a governance boundary Inventory which systems and agents can query identity data through MCP, then constrain those queries to approved identity governance use cases and logged access paths.
• Test governance documentation for AI-assisted drift Check whether AI-generated role descriptions, access justifications, and governance documents remain accurate, sourceable, and reviewable after human edits.

Key takeaways

• Explainable AI matters in IAM because access and recertification decisions need to be defensible, not merely statistically likely to be correct.
• The article argues that AI should reduce review noise and expose patterns, while humans keep the final decision and accountability.
• For practitioners, the key test is whether AI outputs can be explained, verified, and recorded inside the governance workflow.

Key terms

• Explainable AI: Explainable AI is the practice of making an AI system’s decisions understandable to the people who have to review, validate, or rely on them. In financial services, that means producing explanations that can support compliance, model validation, customer communications, and audit, not just technical curiosity.
• Access Recertification: Access recertification is the periodic review of user or account permissions to confirm that access is still justified. It is useful, but it is not enough on its own because it reacts after entitlements already exist, which is why lifecycle governance must reduce the volume of exceptions before review time.
• Segregation of Duties: Segregation of Duties is a control principle that prevents one person or role from combining incompatible permissions that could create fraud, error, or undetected change. In ERP environments, it must account for roles, transactions, approvals, and compensating controls across business processes.
• Model Context Protocol: Model Context Protocol is an open protocol that lets AI agents connect to tools and data sources. It expands what an agent can reach, so governance has to cover not only the model and its prompts, but also every system that can receive or return agent-driven data.

What's in the full article

Nexis's full article covers the operational detail this post intentionally leaves for the source:

• Examples of how NICO explains flagged outliers during access reviews
• Workflow details for AI-assisted role management and governance documentation
• The BYO LLM architecture discussion and how it keeps model choice inside enterprise boundaries
• How MCP exposure is used to query identity data across enterprise tools

👉 The full Nexis article covers the access review workflow, role-management examples, and MCP architecture detail.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.