AI risk management needs runtime controls, not audit-only oversight

At a glance

What this is: This guide argues that AI risk management must move from documentation to operational control across discovery, assessment, execution, monitoring, and response.

Why it matters: It matters to IAM practitioners because AI systems increasingly behave like governed identities with access, permissions, and audit obligations that must be controlled in runtime, not just reviewed after the fact.

By the numbers:

• IBM reported that 97% of organizations reporting AI-related security breaches lacked proper AI access controls.
• IBM reported that 63% of breached organizations lacked an established AI governance policy or were still developing one.

👉 Read TruFoundry's practical guide to AI risk management for enterprise teams

Context

AI risk management fails when organisations treat it as a policy exercise rather than a control problem. The first question is not whether a model is accurate in testing, but where it runs, what data it can reach, and which runtime controls can stop unsafe actions before they leave the request path. In practice, AI systems now need the same kind of governed access thinking that IAM brings to human and non-human identity programmes.

The article is strongest when it frames AI systems as operational actors that can call tools, move data, and create downstream impact without generating the obvious alerts security teams are used to. That is why the identity intersection is real here: once agents and model-backed workflows can access systems, identity, logging, approval boundaries, and revocation become part of AI risk management, not separate concerns.

Key questions

Q: How should organisations govern AI systems that can make consequential decisions?

A: Organisations should govern consequential AI systems with the same discipline used for high-risk identities: defined ownership, least privilege, logging, approval boundaries, and human override. The critical requirement is to connect model behaviour to real access paths so legal review, security review, and audit evidence all describe the same system.

Q: Why do conversational AI systems create new identity and access risks?

A: Because they can combine data retrieval, decision-making, and execution in a single interaction. That collapses the gap between information access and business action, which traditional IAM and security tools were not built to manage. The result is higher exposure when the system can modify records or disclose sensitive guest data.

Q: What do organisations get wrong about AI-driven cyber risk?

A: They often assume the main change is autonomous attackers, when the immediate change is faster and more variable abuse of existing identity pathways. That mistake pushes attention toward speculative defenses instead of scoped access, strong telemetry, and response readiness. The operational risk is already here, even if full autonomy is not.

Q: Who is accountable when an AI agent causes a security incident?

A: Accountability should sit with the business owner, the system owner, and the security function together, because agent behaviour crosses operational boundaries. Organisations need a defined owner for approval, monitoring, and retirement, plus audit evidence that shows what the agent accessed and why.

Technical breakdown

Why AI risk management starts with production visibility

AI risk is different because a system can appear healthy while still producing harmful, biased, or unsafe output. Traditional monitoring looks for crashes, exceptions, or outages. AI needs observability across prompts, responses, tool calls, and access patterns because the failure may be semantic rather than technical. That means teams must inventory sanctioned and shadow AI, map where models and agents run, and understand which data and actions they can reach. Without that baseline, every later control is partial because no one knows the true scope of exposure.

Practical implication: build an inventory of models, agents, tools, and embedded AI features before you try to govern their behaviour.

How identity and tool permissions shape AI agent risk

When an AI agent can call APIs, update records, or trigger transactions, it behaves like a non-human identity with delegated authority. The core risk is not only model quality but permission scope, token lifetime, and what the agent can do before a human notices. This is where IAM and PAM thinking becomes essential. Identity binding, least privilege, approval checkpoints, and revocation matter because the agent’s access can be wider than its task. If permissions are persistent or poorly scoped, agent actions become difficult to contain once they are in motion.

Practical implication: enforce task-scoped permissions and approval boundaries for every agent that can reach production systems.

Why monitoring and response matter more than audit after the fact

AI governance breaks down when organisations rely on periodic review instead of continuous monitoring. Models drift, prompts change, providers update behaviour, and agent loops can amplify cost or risk long after launch. The control point is not just the model itself but the production path around it, including logs, budgets, thresholds, and response playbooks. A mature program treats incidents, policy failures, and access anomalies as operational events that must be detected and contained in real time, then fed back into policy and design.

Practical implication: monitor AI requests continuously and predefine containment actions for misuse, drift, and access anomalies.

Threat narrative

Attacker objective: The objective is to exploit ungoverned AI access paths to drive data exposure, unsafe actions, or operational disruption at scale.

1. Entry occurs when sanctioned or shadow AI systems are introduced without complete inventory, leaving exposed models, tools, and workflows outside governance.
2. Escalation happens when agents or model-backed workflows inherit broad permissions, allowing tool use, data access, or transaction execution beyond the intended task.
3. Impact follows when unsafe output, unauthorised action, cost blowouts, or compliance exposure occur without timely detection or containment.

NHI Mgmt Group analysis

AI risk management is becoming an identity governance problem as much as a model governance problem. Once agents, gateways, and tool-using workflows can act in production, the programme has to know who or what is authorised to do what, for how long, and under which conditions. That shifts the centre of gravity from static review to lifecycle control, especially for permissions, logging, and revocation. Practitioners should treat AI systems as governed actors, not just software features.

Silent failure is the defining AI risk pattern, and that changes how security leaders measure control effectiveness. Traditional security programs often rely on alerts, exceptions, or service disruption as proof that something went wrong. AI can appear to succeed while still producing harmful outcomes, so control testing has to examine output quality, access scope, and downstream action, not just uptime. The practical conclusion is that evidence of safe operation must be designed into the workflow, not inferred later.

Shadow AI creates governance debt that compounds exactly where identity boundaries are weakest. Unapproved models, embedded copilots, and unmanaged agents often appear before ownership, access boundaries, and audit requirements are defined. That is why the first control question is discovery, followed immediately by entitlement scope and evidence retention. If the organisation cannot see the AI estate, it cannot govern the identities operating inside it.

Runtime enforcement is the named concept that separates AI governance from documentation theatre. AI risk programs fail when policy exists only in review documents and not at the point where prompts, tools, budgets, and approvals are executed. The control gap is not a missing committee, but the absence of enforceable boundaries on the request path. Practitioners should build systems where risky actions are blocked before they happen, not explained after they occur.

NIST AI RMF is relevant here, but it is not sufficient on its own for production control. The framework gives a useful structure for govern, map, measure, and manage, yet enterprise teams still need identity enforcement, logging, and task-scoped access to make those functions real. That is where AI governance intersects with IAM, PAM, and non-human identity controls. The discipline is to pair risk frameworks with runtime controls that actually constrain action.

What this signals

Runtime enforcement will become the differentiator between mature AI governance and policy theatre. As organisations move from experimentation to production, the control plane has to constrain prompts, tools, budgets, and access at the moment of execution. That makes AI governance inseparable from IAM and PAM discipline, especially where agents and MCP-connected tools can act on real systems.

The immediate programme signal is to align your AI control model with the NIST AI Risk Management Framework and the OWASP Agentic AI Top 10, then map those principles to runtime enforcement rather than policy statements. If an AI use case cannot be inventoried, bounded, logged, and revoked, it is not ready for broad production use.

Shadow AI will keep expanding until discovery becomes continuous rather than periodic. Organisations that only review AI assets during procurement or annual risk assessments will miss the systems that create the most exposure. The practical response is to treat AI discovery, entitlement scope, and evidence retention as a standing control cycle, not a project milestone.

For practitioners

• Inventory all AI systems and shadow AI use Create a live register of models, agents, embedded AI features, external APIs, and unsanctioned tools. Tie each entry to an owner, business purpose, data access scope, and production environment so the organisation can see what is actually in use.
• Bind every agent to task-scoped identity controls Require least-privilege permissions, short-lived credentials, and explicit approval boundaries for agents that can call tools or change records. Treat agent access like any other high-risk non-human identity and revoke it when the task ends.
• Enforce request-path guardrails before execution Place logging, budgets, content filters, and action policies in the path of every production AI request. Block unsafe tool calls and anomalous behaviour before they reach downstream systems, rather than relying on post-event review.
• Operationalise monitoring for drift and misuse Track output quality, access anomalies, repeated tool usage, and cost spikes continuously. Feed the resulting signals into response playbooks that define containment, investigation, and rollback steps for AI incidents.
• Preserve audit evidence for regulatory review Maintain ownership records, structured logs, risk assessments, and incident notes for each material AI use case. This creates evidence for compliance, investigations, and control validation when regulators or internal assurance teams ask for proof.

Key takeaways

• AI risk management fails when organisations rely on reports and committees instead of controls that operate inside the production request path.
• The evidence in this article shows that shadow AI and weak access controls are already producing measurable breach cost and governance exposure.
• Practitioners should prioritise discovery, task-scoped permissions, continuous monitoring, and revocation so AI behaviour stays inside approved boundaries.

Key terms

• Runtime AI Risk Management: The ongoing operational discipline of identifying, controlling, and evidencing AI risk while the system is in production. It focuses on live prompts, responses, policy outcomes, and audit trails rather than static policy documents or one-time approval decisions.
• Shadow AI: AI agents, copilots, or connected tools operating without full visibility or governance from security teams. Shadow AI becomes an identity problem when those systems authenticate with unmanaged tokens, service accounts, or OAuth apps that can reach production resources.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.

What's in the full article

TruFoundry's full article covers the operational detail this post intentionally leaves for the source:

• The article expands the step-by-step AI risk management lifecycle from identify through respond for enterprise teams that need a working process.
• It breaks down the four AI risk categories with examples of how technical, data, operational, and governance exposure show up in production.
• It outlines how regulation, including the EU AI Act and privacy obligations, changes evidence expectations for teams operating AI systems.
• It maps practical controls such as access restrictions, budgets, guardrails, logging, and human review to the AI request path.

👉 TruFoundry's full article covers the lifecycle, control categories, and regulatory detail behind the framework.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and identity lifecycle control. It gives security and identity practitioners a practical way to connect runtime access decisions to broader governance programmes.