TL;DR: AI governance often relies on attestations, questionnaires, and framework mappings that say what should be true, but not whether AI systems are actually behaving that way, according to Cranium. The real control gap is the absence of runtime evidence such as model verification, event logs, and telemetry that turns policy into provable reality.
At a glance
What this is: This is a critique of checkbox-driven AI governance that says green checkmarks are not proof, because runtime evidence is missing.
Why it matters: It matters to IAM, IGA, and security teams because AI governance only becomes credible when policy, enforcement, and evidence are linked at runtime.
Context
AI governance is the set of controls that says what an organisation expects its AI systems to do, but it fails when those controls stop at paperwork. In this article, the primary problem is not policy design but proof: without runtime verification, governance cannot tell whether a model, use case, or AI workflow is behaving as approved.
For identity and access teams, that proof gap matters because AI systems now sit inside the same governance fabric as vendors, workloads, and delegated automation. If the estate can change beneath an attestation, then compliance artefacts become historical records rather than active controls.
Key questions
Q: What breaks when AI governance relies on attestations instead of runtime evidence?
A: Attestations become stale as soon as the system changes. Policies may still look approved while the live model, data path, or enforcement behaviour has drifted, so the organisation loses the ability to prove that the control still exists. Runtime evidence is what converts governance from a paper claim into a testable operating control.
Q: Why do AI model changes create governance risk even when approvals are still current?
A: Because approval often attaches to a declared model, not to the live service that users actually hit. If the provider swaps the model, hosting location, or upstream dependency, the organisation may inherit a new risk profile without a new review. That is why governance has to track operational identity, not just procurement records.
Q: How do security teams know if AI governance is working?
A: Look for evidence that access decisions are reviewable, permissions are revocable, and exceptions are not becoming permanent. If the team cannot explain who owns an AI workflow, what it can reach, and when its access was last reviewed, governance is incomplete. Control maturity shows up in traceability, not adoption volume.
Q: How should organisations govern sanctioned and unsanctioned AI use together?
A: They should use one enforcement and monitoring layer that covers approved models, shadow AI connections, prompt handling, and data movement. Splitting governance from detection leaves gaps where unsanctioned systems can operate outside policy while still touching production data. The practical aim is a single evidence trail across the whole AI estate.
Technical breakdown
Why AI governance without runtime evidence fails
Governance artefacts such as policies, questionnaires, and framework mappings describe intended state, not observed state. In AI environments, that distinction matters because models, prompts, tool use, and data paths can change after approval. Runtime evidence closes the gap by showing what version was active, what data was touched, and what action the system actually took. Without that evidence, a control can be complete on paper while the operating system underneath has drifted. The result is not simply weak compliance. It is a governance model that cannot falsify its own assumptions.
Practical implication: treat runtime evidence as a required control input, not a post-incident audit artefact.
Model identity, version drift, and the vendor card problem
The article’s vendor card example shows a common failure mode: an organisation approves one declared model and later consumes another without any governance signal. That is an identity problem for AI systems, because the governed subject is no longer the same subject that was approved. Version drift changes jurisdiction, data handling, and threat profile even when the contract, questionnaire, and risk register still look current. For practitioners, the core issue is that declared identity and operational identity have diverged, and policy alone cannot detect it.
Practical implication: bind approvals to observed model identity and version, not to self-reported vendor declarations.
Shadow AI and inline enforcement need the same control plane
The article links unenforceable shadow AI rules and ineffective PII redaction to the same architectural weakness: no sensor, no proof, no control. If an AI system can be connected, queried, or used without detection, governance becomes advisory. If redaction happens without a verifiable log, compliance cannot demonstrate that it occurred. This is why AI governance and AI security cannot be separated into different programmes. The controls that verify behaviour are the controls that make governance real.
Practical implication: design one control plane for detection, enforcement, and evidence across sanctioned and unsanctioned AI use.
Breaches seen in the wild
- 12,000 secrets in LLM training data: Truffle Security found 11,908 live API keys and passwords hard-coded in web pages captured by Common Crawl, a dataset used to train LLMs.
- DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
AI governance without runtime proof is a credibility failure, not a documentation gap. Attestations, questionnaires, and framework mappings can show intent, but they cannot show whether the AI estate actually behaved as approved. Once policy is detached from observed behaviour, the governance function becomes archival rather than control-bearing. The practitioner conclusion is simple: no evidence means no credible control.
The declared model is not the governed model when version drift is invisible. The article’s vendor card example exposes a broken assumption that the approved AI service remains the same service over time. That assumption fails when a provider changes models, hosting geography, or data path without the governance layer noticing. The implication is that AI governance must track operational identity, not just contractual identity.
Runtime telemetry is the named concept that separates AI trust theatre from AI trust evidence. Model verification, event logs, and prompt or agentic telemetry are what turn claims into facts in an AI programme. Without them, a policy is only a promise and a green checkmark is only decoration. The practitioner conclusion is that trust records must be continuously reconciled with observed system behaviour.
Shadow AI and redaction failures show that governance and security are the same enforcement problem. If an unsanctioned model can be attached to a production workflow without detection, governance has no operational boundary. If PII redaction cannot produce evidence of enforcement, compliance cannot defend the control. The implication is that AI security infrastructure is not adjacent to governance; it is the mechanism that makes governance verifiable.
Framework alignment matters less than evidentiary alignment when the system can change after approval. EU AI Act, NIST AI RMF, and ISO 42001 can define what should exist, but they do not by themselves prove that the live system still matches the paper record. The practitioner conclusion is to judge AI governance by whether controls can observe and prove behaviour, not by how complete the policy binder looks.
From our research library:
- 52% of respondents see AI security decision-making power shifting toward platform and infrastructure teams rather than the executive suite, according to the 2026 Infrastructure Identity Survey.
- Only 44% of organisations have implemented any policies to manage their AI agents, despite 92% agreeing that governing AI agents is critical to enterprise security, according to the 2026 Infrastructure Identity Survey.
- Read next: Agentic AI Security Guide
What this signals
Runtime trust record: AI governance now needs a living evidence layer that reconciles declared posture with observed behaviour. That changes programme design because approvals, monitoring, and audit artefacts must point to the same live system, not different snapshots.
When policy can change faster than verification, the control objective shifts from periodic review to continuous proof. Security teams should expect AI governance to converge with enforcement telemetry, because without a verifiable runtime record, the governance layer cannot defend its own decisions.
For practitioners
- Instrument runtime verification Require model probing, version checks, and event logging so the approved AI posture is continuously compared with live behaviour.
- Tie governance to observed identity Make vendor declarations provisional until the deployed model, jurisdiction, and data path are verified in production.
- Enforce inline policy controls Use policy enforcement points for prompts, data access, and agent actions so violations are blocked rather than merely recorded.
- Detect shadow AI connections Monitor for unsanctioned model connections to production systems and require an evidence trail for every new attachment.
Key takeaways
- AI governance breaks down when it stops at attestations and mappings that cannot prove what the live system is doing.
- The article’s central evidence problem is version drift and unenforced change, not lack of policy language.
- Security teams need runtime telemetry, inline enforcement, and verifiable logs if they want AI governance to be defensible.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 addresses the attack surface, NIST AI RMF and NIST CSF 2.0 set the technical controls, and ISO/IEC 42001:2023 and EU AI Act define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GOVERN — AI Governance and Accountability | The article is fundamentally about AI governance that cannot prove live behaviour. |
| Recommendation — Establish governance processes that require runtime evidence before declaring an AI control effective. | ||
| ISO/IEC 42001:2023 | AI Management System | The post critiques AI management systems that rely on documentation without operational proof. |
| Recommendation — Build AI management reviews around observed behaviour, not static attestations. | ||
| EU AI Act | Art. 9 — Risk Management System | The article highlights the need for continuous, evidence-based risk management in AI systems. |
| Recommendation — Use continuous monitoring to keep AI risk controls aligned with the deployed system. | ||
| NIST CSF 2.0 | PR.AA-05 — Access Permissions, Entitlements and Authorizations | AI governance here depends on proving that access and use remain authorised at runtime. |
| Recommendation — Verify AI entitlements continuously instead of trusting once-only approval records. | ||
| OWASP Agentic AI Top 10 | ASI03 — Identity & Privilege Abuse | The article discusses AI systems whose declared and actual operating state can diverge. |
| Recommendation — Monitor agent identity and privilege changes so runtime behaviour stays inside approved scope. | ||
Key terms
- Runtime proof: Evidence drawn from current system behaviour rather than static documentation. For identity and access management, runtime proof includes logs, entitlement state, and control signals that can be independently checked during an assessment or investigation without manual reconstruction.
- Living Trust Record: A living trust record is a continuously updated identity and governance record that reconciles declared posture with observed behaviour. It turns static documentation into an operational control object, allowing teams to verify, review, and revoke based on what the system is actually doing.
- Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
- Operational Identity Contact: An operational identity contact is the person or team responsible for receiving and acting on identity administration signals such as SSO setup, directory sync, or certificate renewal. It is not just metadata. It is part of the control plane because it determines who can respond when identity operations need intervention.
Deepen your knowledge
NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.
Published by the NHIMG editorial team on June 20, 2026.
Updated on October 10, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org