AI security and governance now hinge on runtime control

At a glance

What this is: This explainer argues that AI security and governance now depend on runtime control because most AI-specific failures occur through language manipulation, data leakage, and policy bypass rather than classic malware.

Why it matters: For IAM, NHI, and AI governance teams, the key implication is that visibility and policy alone are insufficient unless they are translated into enforceable controls at the moment an AI system receives inputs, calls tools, or exposes data.

By the numbers:

• AI-related incidents reached 233 in 2024, a 56.4% jump from the previous year.
• 13% of organisations experienced breaches of AI models or applications, according to IBM’s 2025 Cost of a Data Breach Report cited by AccuKnox.
• Of those compromised, 97% reported lacking proper AI access controls.
• Another 8% did not know whether their AI systems had been compromised.

Context

AI security governance is the discipline of deciding what AI systems may do, what data they may touch, and how their behaviour is monitored after deployment. The article’s central claim is that traditional security models fail because AI risk often emerges at runtime, through prompts, tool use, inference paths, and data handling rather than through conventional exploit chains.

That matters because AI is now embedded in production workflows, but many organisations still treat it as a software feature instead of a governed system. Where AI systems interact with personal data, customer records, code, or privileged APIs, the governance problem becomes an identity and access problem as much as a model-risk problem.

Key questions

Q: Why do AI governance policies fail when they are written without usage data and enforcement?

A: They fail because they assume people will read, interpret, and comply with policy text on their own. In practice, employees keep using unsanctioned tools and sensitive data keeps flowing unless controls block or warn in real time. A policy that is not grounded in discovery data also misses the actual risk surface, which leaves gaps auditors and attackers can exploit.

Q: Why do AI systems increase security risk across modern infrastructure?

A: AI systems increase risk because they expand the number of assets, data paths, and operational decisions that must be secured at once. When training data, workloads, and supporting services span multiple clouds and applications, attackers gain more opportunities to poison data, steal models, or abuse exposed systems. The result is greater complexity, slower defense updates, and more ways to compromise confidentiality, integrity, and availability.

Q: What are the signs that AI security controls are failing in production?

A: Common warning signs include unapproved model behavior, unexpected data access, prompt leakage, suspicious outbound calls, and runtime actions that do not match the workload’s intended function. If teams also see weak visibility into pipelines, inconsistent policy enforcement, or repeated attempts to access sensitive datasets, the control layer is not keeping pace with the AI threat surface.

Q: How should organisations govern external tools used by AI agents?

A: Organisations should review external tools as security inputs, not convenience features. Each tool needs ownership, approval, metadata inspection, and ongoing monitoring for hidden instructions or unexpected behaviour. If an AI agent can act on a tool, then the tool’s provenance and control status should be governed like any other sensitive integration.

Technical breakdown

Why runtime controls matter more than perimeter defences for AI

AI systems do not fail only when an attacker breaks infrastructure. They also fail when prompts, context, or tool calls induce the model to produce unsafe actions, reveal data, or bypass intended policy. Traditional scanners and network filters are weak here because the attack surface is semantic, not just technical. Runtime controls sit in the execution path, so they can inspect inputs, restrict actions, and stop unsafe output before it is acted on. This is especially important when AI systems can call APIs, access datasets, or trigger downstream workflows.

Practical implication: Move enforcement to the moment of inference and tool use, not just to model approval or network perimeter checks.

How data leakage and model misuse arise across the AI lifecycle

AI systems handle data at multiple stages: training, fine-tuning, inference, logging, and post-deployment analytics. Leakage can occur when sensitive prompts are stored, when embeddings preserve recoverable context, or when models are queried until they reveal information they were not meant to expose. Model misuse also includes behavioural abuse, where the model remains technically available but is coerced into unsafe disclosure or decisioning. Lifecycle control means the data policy must follow the model into every environment where it runs or learns.

Practical implication: Treat datasets, prompts, logs, and embeddings as governed assets with explicit access and retention boundaries.

Why AI governance now needs auditable policy enforcement

Governance is not the same as policy documentation. Frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 expect organisations to define accountability, assess risk, and prove that controls are operating. That requires evidence trails, change visibility, and continuous reassessment as models, prompts, dependencies, and risk profiles change. Without auditability, AI governance becomes a paper exercise that cannot support compliance, incident response, or executive oversight.

Practical implication: Link policy, risk assessment, and runtime evidence so governance decisions can be verified after deployment.

Threat narrative

Attacker objective: The attacker’s objective is to manipulate the AI system into disclosing information, misbehaving, or executing unsafe actions without needing traditional malware.

1. Entry occurs when an attacker supplies crafted prompts or malicious inputs that exploit the model’s interpretation boundaries rather than its code.
2. Credential or data access follows when the model is induced to reveal sensitive context, accept unsafe instructions, or trigger downstream tool calls.
3. Impact is achieved when the model’s output or actions are used to leak data, abuse APIs, or violate policy in production.

Breaches seen in the wild

• Langflow Flodrix botnet 2025: Attackers used Langflow's unauthenticated code endpoint to dump AI servers' environment variables and enlist them in the Flodrix botnet.
• DeepSeek database exposure 2025: An unauthenticated DeepSeek ClickHouse database exposed over a million log lines with plaintext chat history and API keys in 2025.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime control is the new control plane for AI governance: policy documents and model inventories do not stop a prompt from changing behaviour in production. The article’s core evidence is that many failures occur at the point of interaction, where language, context, and tool use matter more than classic exploit mechanics. For practitioners, the decisive question is whether the runtime layer can enforce policy when the model is already live.

AI governance debt is accumulating faster than organisations are closing it: the gap is not just missing documentation, but missing operational controls that translate intent into enforcement. That gap becomes more serious when AI systems touch data, APIs, or user decisions, because the same control failure can create security, compliance, and trust issues at once. Practitioners should treat governance maturity as an execution problem, not a policy-writing problem.

AI systems introduce an identity and delegation problem, not only a model-risk problem: when an AI system can call tools, query data, or act on behalf of users, it starts behaving like a governed software identity. That makes access scope, authorization boundaries, and delegated action trails central to security. Practitioners should align AI governance with identity and privilege controls, especially where agents or copilots can influence production systems.

Continuous testing is now part of operational assurance, not optional red-teaming: static validation cannot keep pace with shifting prompts, dependencies, and adversarial techniques. The article points to the need for continuous adversarial testing and runtime monitoring because exposure changes as the system changes. For practitioners, the lesson is to measure AI controls as living controls, not one-time certifications.

AI security must be built as a lifecycle discipline, not a point solution: discovery, deployment, data protection, runtime enforcement, and compliance evidence are interdependent. If any one layer is missing, the programme inherits blind spots that attackers and governance failures can both exploit. Practitioners should assess AI risk as a connected control stack rather than as isolated tools.

From our research library:

• 13% of organisations suffered breaches of their AI models, according to IBM's Cost of a Data Breach 2025 report.
• Read next: Agentic AI Identity Risk Board Briefing

What this signals

AI governance is becoming a runtime control problem: as more systems embed models into live workflows, the controls that matter are the ones that can intervene during inference, not after a quarterly review. Organisations that only inventory models without enforcing their behaviour will keep discovering risk too late.

AI governance debt will surface as access debt: when a model can query data or trigger tools, every weak boundary becomes an opportunity for misuse. The practical response is to govern AI inputs, outputs, and delegated actions with the same discipline applied to privileged identities and sensitive APIs.

52% of respondents see AI security decision-making power shifting toward platform and infrastructure teams rather than the executive suite, according to the 2026 Infrastructure Identity Survey. That shift signals that AI security is being operationalised where runtime controls, deployment paths, and access boundaries are actually administered.

For practitioners

• Map every AI workload and dependency Build an inventory of models, prompts, datasets, embeddings, APIs, and runtime locations so you know where AI is actually being used and what it can reach.
• Enforce runtime policy at inference time Block unsafe prompts, restrict tool use, and terminate policy-violating sessions in the execution path rather than relying on pre-deployment approval alone.
• Separate sensitive data from model context Apply access boundaries to training data, inference inputs, and logs so regulated or confidential information is not exposed through embeddings or prompt history.
• Run continuous adversarial testing Use red-team scenarios for prompt injection, jailbreaks, and data leakage so model risk scores reflect current behaviour instead of stale assessments.
• Tie governance evidence to audit trails Record policy decisions, runtime violations, and control outcomes so compliance teams can demonstrate that AI governance is operating, not just documented.

Key takeaways

• AI security failures are increasingly driven by prompt manipulation, data leakage, and tool misuse, which means the control point has shifted from static review to runtime enforcement.
• The article links governance gaps to operational blind spots, especially where organisations cannot show evidence that models, prompts, and delegated actions are being monitored.
• For practitioners, the immediate priority is to inventory AI workloads, constrain what they can access, and prove that policy is enforced where the model runs.

Key terms

• Runtime Enforcement: Runtime enforcement is the practice of blocking malicious behaviour while software is running, rather than only detecting it after the fact. It monitors process activity, network actions, and privilege changes so a live attack can be interrupted at the point of execution.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• AI Governance: AI governance is the set of controls used to discover, classify, approve, restrict, monitor, and revoke AI-enabled access. It connects identity, data, and policy so organisations can manage what AI can reach, what it can share, and when it should be stopped.
• Delegated Tool Selection: The practice of allowing an agent to choose software, skills, or actions on behalf of a user. This shifts security responsibility from the end user’s judgment to the governance of selection inputs, install-time checks, and permission boundaries.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, helps practitioners understand NHI governance, workload identity, and secrets management in the systems that increasingly support AI security and access control. It gives security and identity teams a shared language for governing privileged automation and machine access.