AI governance still fails without continuous verification

At a glance

What this is: This is Openlayer’s argument that AI reliability requires continuous governance, not static policy documents, because model behaviour changes faster than one-time controls can capture.

Why it matters: It matters to IAM, NHI and security teams because the same governance gap appears whenever automated systems make runtime decisions, consume sensitive data, or act with delegated access.

By the numbers:

• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read Openlayer's analysis of Constitutional AI and continuous AI governance

Context

AI governance fails when teams treat model behaviour as something they can define once and then leave alone. The operational problem is reliability across changing prompts, data, model versions and usage contexts, which is why static policy is not enough. In identity terms, the same issue appears when machine actors or AI agents operate with delegated access that outlives the assumptions behind it.

Openlayer frames this as a governance problem rather than a model-building problem. That distinction matters to IAM and NHI programmes because control quality now depends on continuous validation of what an AI system is allowed to do, what it actually does, and who is accountable when those two diverge.

Key questions

Q: How should security teams govern AI agents that can change behaviour at runtime?

A: Security teams should govern AI agents with runtime monitoring, behavioural baselines, and identity-triggered response, not just static approval workflows. The goal is to detect when an agent drifts from expected purpose, reduce privileges immediately, and keep accountability tied to a named owner. In agentic environments, governance must work at machine speed rather than review cadence speed.

Q: Why do AI tools create new identity governance risks for IAM teams?

A: AI tools create new identity governance risks because they combine fast adoption with broad access paths and subordinate permission objects. A user may look clean in the directory while the platform still holds project roles, service accounts, or keys that can act independently. That makes governance a control-plane problem, not a simple login problem.

Q: What do organisations get wrong about AI policy enforcement?

A: The common mistake is assuming that a written policy meaningfully constrains AI behaviour without technical enforcement. Policies do not stop data leakage, prompt injection, or agent tool misuse unless they are applied at runtime. Effective governance requires controls that are measurable, repeatable, and auditable in production.

Q: How do teams know whether AI governance is actually working?

A: Look for evidence that every AI interaction can be traced end to end, from identity and intent to output and enforcement. If auditors can ask for a transaction and receive a complete record in hours, not weeks, the programme is producing usable control evidence rather than just documentation.

Technical breakdown

Why AI reliability becomes a governance problem

AI reliability is the gap between intended behaviour and observed behaviour under real-world conditions. Foundation models are probabilistic systems, so they can change output quality across prompts, data distributions and release cycles even when the underlying application code is unchanged. That means policy written at design time quickly becomes stale unless it is tested against live behaviour. For security teams, this is the same class of problem seen in delegated access: the control assumption is valid only while the environment stays still, which it never does.

Practical implication: treat AI behaviour as a continuously monitored control surface, not a one-time approval state.

How constitutional AI maps to operational controls

Constitutional AI uses a declared set of rules or principles to shape model behaviour, but the useful insight is governance, not the training technique itself. The article’s three-branch structure translates well to security operations: define measurable rules, enforce them during use, and review exceptions when behaviour deviates. In practice, this resembles policy, enforcement and adjudication layers. For identity teams, the analogous lesson is that access policy only works when enforcement and exception handling are equally explicit.

Practical implication: define measurable behaviour rules before deployment and assign a clear review path for violations.

Why AI lifecycle monitoring matters more than pre-release testing

Pre-release evaluation catches only the behaviour visible in a controlled test environment. Real risk emerges after deployment, when prompts, integrations, user actions and upstream data all change the system’s operating conditions. That is why continuous monitoring is a core governance requirement for AI systems that touch sensitive data, automate decisions, or interact with other systems through APIs. Where those APIs are tied to secrets or machine identities, the AI control problem becomes an identity control problem as well.

Practical implication: extend monitoring beyond model QA to runtime behaviour, data access and delegated system actions.

NHI Mgmt Group analysis

Continuous AI governance is now an operational control, not a policy exercise. The article is strongest when it rejects the idea that model safety can be solved with a single set of rules written in advance. That position aligns with how security teams already manage identity, privilege and lifecycle risk: controls must adapt as systems change. For AI programmes, the practical conclusion is that governance has to be measurable, enforced and revisited continuously.

AI governance debt is the right concept for what most programmes are accumulating. Teams often define principles, then underinvest in the enforcement, exception handling and monitoring needed to make those principles real. That creates a growing gap between documented intent and actual behaviour. In a broader security sense, the same pattern appears in secrets management and access governance when processes exist on paper but drift in practice. The practitioner implication is to treat untested AI policy as technical debt.

Identity becomes part of AI governance the moment systems act through APIs and delegated credentials. Once an AI application can call tools, retrieve data or trigger workflows, its behaviour is inseparable from the identities and secrets that authorize those actions. That makes NHI governance relevant even in an article about AI reliability. The field should stop treating AI safety and identity governance as separate programmes, because runtime access is where model behaviour becomes operational risk.

The legislature, executive and judiciary model is useful only if accountability is explicit. A governance architecture is not complete because its parts are named. It only works when each layer has measurable duties, escalation criteria and ownership for exceptions. That is a familiar lesson from IAM and PAM, where policy, enforcement and review fail when they are not operationally separated. Practitioners should use this as a design pattern for AI controls, not a metaphor.

Openlayer’s framing points to a wider market shift toward control observability. Security teams will increasingly need evidence that an AI system stayed within defined boundaries, not just that it passed a launch checklist. That strengthens the case for control telemetry, behavioural validation and post-deployment review across AI and identity programmes. The industry is moving toward governance that can be audited as it runs, which is what practitioners should prepare for.

What this signals

AI governance debt: when organisations define AI policy but fail to instrument enforcement, the gap shows up later as drift, exception sprawl and weak accountability. That makes AI control effectiveness a lifecycle issue, not a launch issue, and it puts pressure on governance teams to adopt runtime evidence rather than narrative assurance.

If an AI system can reach business data through NIST SP 800-63 Digital Identity Guidelines is less relevant than whether the machine path behind it is observable, bounded and revocable. The practical signal for IAM and NHI teams is simple: any system that can call tools or read data through delegated credentials should be tracked as an identity with its own review and offboarding requirements.

The security programme implication is that AI assurance will increasingly depend on control telemetry across identities, data access and model behaviour. Teams that already manage secrets, service accounts and privileged workflows have the right operating model to extend into AI governance, but they need to treat model actions as auditable events rather than opaque outputs.

For practitioners

• Define measurable AI behaviour rules Translate policy statements into testable criteria for hallucination tolerance, data leakage, refusal behaviour and task boundaries. Put those criteria under version control so changes are reviewable across model releases and prompt updates.
• Separate enforcement from review Assign one team to enforce runtime controls and another to adjudicate exceptions, failed tests and out-of-policy behaviour. This avoids the common failure where the same group writes policy, checks compliance and approves exceptions.
• Monitor runtime actions, not just model output Track tool calls, data retrieval, secret use and downstream side effects for any AI system that can interact with business services. If the system uses delegated credentials, fold those events into identity monitoring and access review.
• Inventory AI systems with delegated access Map every model, agent and workflow that can reach data through API keys, service accounts or tokens. Classify them as machine identities so secret rotation, privilege scope and offboarding are managed as identity controls.

Key takeaways

• AI reliability is a governance problem because model behaviour changes faster than static policy can keep up.
• Identity and secrets become part of AI safety the moment a system can act through APIs, tokens or service accounts.
• Practical AI governance depends on measurable rules, runtime enforcement and exception review across the full lifecycle.

Key terms

• Constitutional AI: A governance approach that encodes behavioural rules or principles for an AI system and evaluates outputs against them. In practice, it shifts the problem from ad hoc moderation to repeatable rule application, although the rules still need continuous updating and enforcement to stay relevant.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
• Runtime validation: A control practice that tests how an AI system behaves while it is connected to real tools and data, rather than only reviewing configuration or design documents. It matters because agentic systems can appear safe on paper and still fail when prompted, chained, or given access to connected services.
• Delegated Access: Delegated access is permission granted to one identity to act on behalf of another user, service, or system. In NHI environments, this usually appears in OAuth-connected apps and automation tooling. It is powerful, but it must be tightly scoped and reviewed because it can persist long after the original business need ends.

What's in the full article

Openlayer's full article covers the conceptual and operational detail this post intentionally leaves at the governance layer:

• The article's explanation of Constitutional AI as a governance model for setting behavioural rules in AI systems
• The three-branch legislature, executive and judiciary structure used to describe AI governance functions
• The argument for continuous validation across the AI lifecycle rather than one-time pre-deployment testing
• The article's own framing of how teams can convert abstract principles into enforceable AI operating rules

👉 Openlayer's full post expands on the reliability problem, governance model and implementation framing in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance and secrets management for practitioners who need to connect identity controls to runtime access risk. It helps security teams build the governance foundations that modern AI and machine identity programmes depend on.