AI governance now hinges on continuous evidence, not policy

At a glance

What this is: This is an opinion and framework analysis arguing that AI governance has shifted from policy-led compliance to continuous evidence of runtime behaviour.

Why it matters: It matters because IAM, IGA, PAM, and AI governance teams now need auditable proof of what AI systems actually did, not just what approvals said they could do.

Context

AI governance is becoming an evidence problem because agentic systems do not stay still long enough for annual review cycles to capture what actually happened. The article frames the gap as a mismatch between traditional control models and runtime behaviour, where permissions, tools, and prompts can change between audits.

The governance question is no longer whether a policy exists, but whether the organisation can prove that the system stayed inside approved boundaries while it ran. That shifts AI oversight from periodic documentation to continuous assurance, which is the practical problem space behind NIST AI RMF, ISO/IEC 42001, and the EU AI Act.

Key questions

Q: What breaks when AI governance relies on attestations instead of runtime evidence?

A: Attestations become stale as soon as the system changes. Policies may still look approved while the live model, data path, or enforcement behaviour has drifted, so the organisation loses the ability to prove that the control still exists. Runtime evidence is what converts governance from a paper claim into a testable operating control.

Q: Why do NIST AI RMF, ISO 42001, and the EU AI Act all push continuous evidence?

A: They differ in form, but they converge on the same practitioner need: prove what the AI did, who was accountable, and whether it stayed within approved boundaries. Continuous evidence matters because model approval alone does not show runtime behaviour, and regulators, auditors, and boards increasingly care about the latter.

Q: How do teams know whether AI governance is actually working?

A: Look for evidence that every AI interaction can be traced end to end, from identity and intent to output and enforcement. If auditors can ask for a transaction and receive a complete record in hours, not weeks, the programme is producing usable control evidence rather than just documentation.

Q: Who is accountable when an AI agent acts outside its intended scope?

A: The organisation is accountable, but operational responsibility should sit with a named owner and a governance process that can explain the agent’s purpose, access, and recorded actions. Without that, autonomous behaviour becomes unassignable risk rather than managed automation.

Technical breakdown

Why annual attestations fail for runtime AI systems

Annual or point-in-time attestations assume the controlled system is stable long enough for evidence collected later to represent what happened earlier. Agentic AI breaks that assumption because prompts, tools, connectors, and permissions can change without a code deploy, and the meaningful event is often the action taken in-session rather than the model version on paper. In governance terms, the control boundary moves from static approval to live execution. That makes evidence freshness, log integrity, and action attribution part of the control itself, not just the audit trail after the fact. See the Audit Evidence and Continuous Controls Monitoring Guide.

Practical implication: Treat runtime telemetry and immutable logs as the governance artefact, not the annual review packet.

What continuous assurance means across NIST AI RMF, ISO 42001, and the EU AI Act

These three frameworks differ in force and form, but they converge on the same operational requirement: organisations must be able to show what the AI did, why it was allowed to do it, and who is accountable for the outcome. NIST AI RMF supplies the governance language, ISO/IEC 42001 defines a certifiable management system, and the EU AI Act imposes legal obligations for logging, oversight, and post-market monitoring. The technical common denominator is continuous, attributable evidence tied to a specific system boundary and owner. Read the Ultimate Guide to NHIs: Regulatory and Audit Perspectives.

Practical implication: Build one evidence model that can satisfy governance, certification, and legal review without re-creating records for each framework.

Why governance has to follow decisions, not models

A model is only one component in an AI system, and it is often not the component that creates risk. The article’s core point is that consequences arise from decisions and actions: what data was accessed, what tool was used, what changed, and what identity performed the action. That is why model-centric review misses the real control surface for agentic systems. The governance object becomes the decision event itself, which is closer to identity security, access control, and evidentiary logging than to classic model approval workflows. See the Agentic AI Security Policy Template.

Practical implication: Track each AI action back to an accountable identity and the exact permissions used at the time of execution.

Threat narrative

Attacker objective: The objective is to exploit runtime drift so that privileged AI actions occur outside the boundaries the organisation believes it approved.

1. Entry occurs when an agent inherits a connector or credential already carrying access into a downstream system, even if the change was made through configuration rather than code.
2. Escalation happens when the agent uses a newly added tool or expanded permission set to take actions that were never separately re-approved.
3. Impact follows when the organisation cannot reconstruct or prove the behaviour after the fact, leaving audit, legal, and operational teams unable to verify boundary compliance.

Breaches seen in the wild

• iOS apps leaking hard-coded secrets: Cybernews found 71% of 156,080 iOS apps leak hard-coded secrets, with open cloud storage and Firebase databases exposing user data.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Evidence is now the control surface: AI governance fails when organisations treat policy as proof. Policy states intent, but the frameworks cited in the article judge whether the system actually behaved within approved limits. That means evidence freshness, attribution, and runtime traceability are now core governance requirements, not audit decorations. The practical conclusion is that programmes built for static systems must be redesigned around live behaviour, not annual submission packets.

Runtime drift invalidates snapshot governance: The annual audit model was built for systems that hold still long enough to be reviewed later. Agentic AI changes tools, permissions, and behaviour midstream, so the approval state captured at design time can be obsolete by the time the action occurs. The implication is not simply more monitoring, but a different governance assumption: the governed object is a moving execution path, not a fixed model artefact.

Decision-level accountability is the missing identity layer: The article is right to shift attention from model risk to the decisions the system makes. Once an AI action is taken, it is already authorised in effect, which means identity, permission, and evidence have to be bound to the action itself. This is where AI governance starts to overlap with agentic AI security policy thinking, because accountability has to survive tool use, delegation, and runtime change.

The evidence gap is a governance gap, not a tooling gap: Organisations are not failing because they lack policies or committees. They are failing because their control model still assumes that approval and execution are separated by time, when agentic systems collapse that gap. The result is a programme that can describe intent but cannot prove state, and that is no longer sufficient for auditors, boards, or regulators.

What this signals

Continuous evidence will become the operational language of AI governance: programmes that cannot produce runtime proof will struggle to satisfy audit, certification, and legal scrutiny at the same time. The strategic shift is from periodic sign-off to always-on traceability, which moves AI governance closer to identity and access discipline than to classic model review.

The next governance boundary is not the model itself but the event trail around it. As runtime actions become the unit of control, identity owners will need to treat logs, ownership, and change records as one system of record rather than separate compliance artefacts.

For practitioners

• Define the evidence object before the control object Specify what proof must exist for each AI system, including action logs, tool use, boundary checks, and accountable ownership, before deciding which platform stores it.
• Bind every AI system to a named owner Require a responsible identity for each production AI workflow so that every action can be traced to an accountable decision owner and not just a vendor or model name.
• Treat runtime changes as governance events Require review and evidence capture when prompts, connectors, permissions, or downstream tools change, because those are the moments that alter the approved operating boundary.
• Align audit evidence to the three framework lenses Map the same evidence set to NIST AI RMF governance, ISO/IEC 42001 certification needs, and EU AI Act logging and oversight obligations so teams do not rebuild records three times.

Key takeaways

• AI governance is moving from policy compliance to proof of runtime behaviour, because agentic systems can change materially between audits.
• The article’s central evidence problem is that frameworks now expect continuous, attributable records of what the system did, not just what it was approved to do.
• Practitioners should align ownership, logging, and change control so the same evidence set can support operational review, certification, and legal scrutiny.

Related NHI Mgmt Group resources: Audit Evidence and Continuous Controls Monitoring Guide · Ultimate Guide to NHIs: Regulatory and Audit Perspectives · Agentic AI Security Policy Template

Key terms

• Continuous Evidence: Continuous evidence is operational proof that identity controls are working now, not just at the time of an audit. It replaces one-off snapshots with current signals about ownership, rotation, access, and usage, which is essential when identities change too quickly for periodic review alone.
• Runtime Governance: Runtime governance is the set of controls that verify what a system or agent is actually doing after deployment. It combines monitoring, authorization checks, and access validation so teams can detect drift, misuse, or excessive privilege in motion rather than assuming build-time policy still holds.
• Decision accountability: The ability to show who owned a decision, what information they used, and why the outcome was accepted or rejected. In AI governance, accountability comes from clear authority and recorded review, not from the mere presence of automation or a human approval step.
• Evidence Gap: The difference between having a control in policy and being able to prove it was applied in practice. In identity programmes, evidence gaps appear when access changes, reviews, and revocations must be reconstructed from emails, screenshots, or spreadsheets rather than generated continuously.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.