Topic starter
25/06/2026 12:24 am
TL;DR: AI explainability helps stakeholders interpret model outputs, but it does not solve governance, accountability, or runtime control problems in high-stakes AI systems, according to WitnessAI. The real issue is that explainable decisions are still governed by opaque access paths, approval chains, and lifecycle assumptions that IAM and NHI programmes must now re-evaluate.
Editorial analysis by NHI Mgmt Group, based on content published by WitnessAI: “AI Explainability: Building Trust in Artificial Intelligence”.
Key questions
Q: What breaks when AI explainability is treated as the main governance control?
A: Explainability can show how an AI system reached a result, but it does not restrict who can trigger actions, which tools the system can call, or how long authority persists.
Q: Why does autonomous AI create more risk than a transparent model with the same output quality?
A: Autonomous AI can select actions and invoke tools at runtime, so the risk is not just the output quality but the authority path behind it.
Q: How do security teams know whether AI explanations are masking a control problem?
A: Look for a gap between what reviewers can understand and what the system is actually allowed to do.
Practitioner guidance
- Separate explanation from authorisation Require every production AI workflow to have explicit approval, entitlement, and scope controls that are independent of any explanation layer.
- Inventory AI actors with delegated access List every AI system, agent, and automated workflow that can reach data, tools, or APIs, then assign ownership and lifecycle status to each one.
- Limit tool reach to task-scoped authority Constrain runtime access so the system can only call the services and datasets needed for the current task, not a broader reusable role.
Bottom line: Explainability improves understanding, but it does not by itself govern authority, approvals, or runtime access.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
25/06/2026 9:18 am
Explainability is a visibility control, not a governance control: It helps people interpret outputs, but it does not govern who may act, when they may act, or what systems they may reach. In identity terms, that means the decision trail can be legible while the authority trail remains overbroad. Practitioners should stop treating transparency as a proxy for delegated control.
A few things that frame the scale:
- AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
A question worth separating out:
Q: Should organisations treat AI as an application or as an identity?
A: Treat it as an identity when the AI can access data, invoke tools, or participate in workflows that affect business systems. That framing makes least privilege, just-in-time access, and lifecycle governance relevant. If you keep treating it only as an application, you will miss the access and delegation behaviours that actually create risk.
👉 Read our full editorial: AI explainability is not enough for governing autonomous systems