Who is accountable when an approved AI system drifts from its declared posture?

Why This Matters for Security Teams

When an approved AI system drifts from its declared posture, the immediate risk is not just model quality, but broken trust in the approval boundary itself. A system that was reviewed under one set of capabilities may later gain new tools, new data access, or new runtime behaviour without any fresh decision. That creates an accountability gap between the team that approved the original state and the team that should have detected the change. NIST’s NIST Cybersecurity Framework 2.0 treats governance, monitoring, and response as linked responsibilities, not isolated functions. For NHI Management Group, the key issue is observability at the control boundary. If posture drift is invisible, then even a correct approval process becomes unreliable after deployment. This is especially true where AI systems act through secrets, tokens, or delegated permissions that can be reused after a change in model behaviour. NHIMG research on the State of Secrets in AppSec shows how fragmented secret management already weakens centralised control, which is exactly the kind of weakness that makes drift hard to detect and harder to assign. In practice, many security teams discover posture drift only after the system has already used access in ways no reviewer explicitly approved, rather than through intentional lifecycle governance.

How It Works in Practice

Accountability for drift should follow the control chain, not just the original sign-off. In practice, that means three groups share responsibility: the team that approved the system, the team that operates monitoring and detection, and the team that can make or block changes. If any one of those groups is missing, drift becomes a blind spot instead of an auditable event. A workable operating model usually includes:

• declared posture baselines for model behaviour, tool access, data scope, and secret usage
• runtime telemetry that records when the system’s permissions, prompts, tools, or outputs change materially
• change control that requires re-approval when the declared posture no longer matches the deployed one
• identity-bound logging so actions can be tied to the AI system, the operator, and the approval record

This is where AI governance overlaps with NHI governance. If an AI system uses tokens, API keys, or service credentials, then drift can turn into privilege expansion even when the model itself has not been retrained. NHIMG’s LLMjacking coverage shows how compromised NHIs can be used to hijack AI access paths, which makes runtime identity controls part of the accountability model, not an implementation detail. Current guidance suggests treating drift detection as a policy enforcement problem, not a post-incident review task. That aligns with the NIST Cybersecurity Framework 2.0 and its emphasis on ongoing monitoring and response. Where organisations mature faster, they also tie approval records to immutable configuration and event logs so that posture changes can be proven, not inferred. These controls tend to break down when AI systems are updated through shadow deployments, vendor-managed backends, or rapid prompt-and-tool changes because the approved state and the live state diverge without a formal change event.

Common Variations and Edge Cases

Tighter drift control often increases operational overhead, requiring organisations to balance faster AI iteration against stronger re-approval and logging requirements. That tradeoff becomes sharper in environments with delegated admin rights, external model providers, or agentic workflows that can chain tools at runtime. There is no universal standard for this yet, but best practice is evolving toward posture drift thresholds that trigger action without waiting for a full incident. For example, a minor prompt update may only require review, while a new tool connector, a broader data domain, or a changed secret scope should force renewed approval. The important distinction is not whether the system is “the same model,” but whether the live control boundary still matches the declared one. This is also where accountability can become blurred. Compliance may define the policy, but it does not own runtime change detection. Platform teams may operate the system, but they should not be the only line of defence. The accountable answer is shared ownership with explicit decision rights, audit trails, and escalation paths. NHIMG’s Salesloft OAuth token breach illustrates the danger of trust assumptions that outlive the approved posture. In edge cases, especially managed AI services with limited telemetry, organisations may need contractual logging guarantees or compensating controls because drift cannot be proven or disproven locally.

Related resources from NHI Mgmt Group

• Who is accountable for deception controls in AI and NHI programmes?
• What makes agentic AI an NHI governance issue?
• Why is NHI governance critical in the age of AI attacks?
• Why is single-provider AI agent governance not enough for enterprise security?