AI runtime security shifts into governance for enterprise AI workloads

At a glance

What this is: This is a governance-focused announcement about HiddenLayer joining the Databricks Unity AI Gateway ecosystem to extend AI security into runtime workflows across models, agents, tools, and MCP integrations.

Why it matters: It matters because IAM, security, and platform teams now have to govern AI behaviour at execution time, not just approve models before deployment, especially where AI can call tools and interact with business systems.

Context

Enterprise AI security changes materially when systems move from isolated model calls to runtime workflows that retrieve data, invoke tools, and execute actions. In that state, access control alone is not enough because the security problem becomes one of governing behaviour across models, agents, prompts, responses, tool calls, and MCP-enabled interactions.

HiddenLayer’s announcement is best read as a statement about where enterprise AI governance is headed: closer to runtime, closer to operational controls, and closer to detection-and-response workflows. The article frames this as a distinct security surface, separate from traditional application governance and separate from pre-deployment model review.

For IAM and platform teams, the practical question is no longer whether AI is approved for use. It is whether the organisation can observe, constrain, and investigate what AI systems do after they are connected to real data and tools.

Key questions

Q: How should teams govern AI agents that use MCP?

A: Treat each connected agent as a non-human identity with an owner, a scope, and a review cycle. The practical control set is familiar: least privilege, secret rotation, access expiration, and auditability across the systems the agent can reach.

Q: Why do governance checklists fall short for production AI workloads?

A: Governance checklists confirm that a system was approved, but they do not control what happens when the AI is manipulating prompts, invoking tools, or moving through connected workflows. The risk appears at runtime, so security has to observe and constrain live behaviour, not just validate design-time intent.

Q: What are the warning signs that an AI runtime security programme is failing?

A: Look for broad tool access, missing ownership for agents and connectors, weak audit trails, and AI actions that cannot be tied back to a clear task or policy decision. If you cannot answer what the agent accessed, why it accessed it, and who can revoke it, the programme is not controlling runtime risk.

Q: What should organisations watch when AI agents are connected to business systems?

A: Look for tool sprawl, broad delegated permissions, and weak audit trails around agent actions. The key issue is not that the agent uses tools, but whether it can choose actions at runtime beyond a fixed workflow. If it can, then governance must cover scoping, logging, and revocation at the level of action, not just account creation.

Technical breakdown

Runtime governance for AI workloads

Runtime governance means controlling what an AI system can do while it is actually operating, not only what it is allowed to do on paper. In enterprise settings this spans prompts, responses, model behavior, tool usage, agent actions, and MCP workflows. That matters because AI systems increasingly sit inside business processes rather than beside them, so harmful actions can occur after a model is already trusted and deployed. Governance at this layer is about context, enforcement, and detection across the execution path, not just policy approval before release.

Practical implication: teams need control points that observe and constrain AI activity at execution time, not only at model onboarding.

Why MCP and tool use change the security boundary

Model Context Protocol integrations and tool-using agents expand the AI boundary from text generation into action execution. Once an AI system can query data sources, call APIs, or trigger workflows, it begins to exercise delegated authority through runtime decisions. That creates a different problem from securing a static model, because the risk is no longer limited to output quality. The risk becomes whether the system can be induced to take unsafe actions, misuse tools, or move sensitive information through legitimate integrations.

Practical implication: security review must cover the tools and data paths the AI can reach, not just the model endpoint.

AI-specific detection and response

AI detection and response focuses on the signals that reveal compromised, manipulated, or unsafe AI behavior in production. The article points to prompt injection, data leakage, model manipulation, unsafe tool use, model theft, and adversarial ML techniques as examples of that threat surface. Traditional monitoring may show that an API was called, but it will not necessarily explain whether the AI’s decision path was coerced or whether an apparently normal tool call represented misuse. AI telemetry therefore has to be interpreted in behavioural context.

Practical implication: security operations should treat AI interactions as an investigation stream with its own indicators, triage logic, and response paths.

Threat narrative

Attacker objective: The attacker aims to bend a trusted AI runtime into leaking data, misusing tools, or performing business actions that expand the effective blast radius of the system.

1. Entry occurs when an attacker reaches an AI runtime through prompts, tool integrations, or MCP-connected workflows that look legitimate to the platform.
2. Credential or trust abuse follows when the AI system uses its delegated access to retrieve data, call APIs, or invoke tools outside the intended security intent.
3. Escalation happens when unsafe tool use, model manipulation, or prompt injection steers the system into broader business actions or data exposure.
4. Impact is realised through data leakage, model theft, or compromised enterprise workflows that the organisation assumed were governed by pre-deployment controls.

Breaches seen in the wild

• LiteLLM MCP auth bypass 2026: An exploited LiteLLM MCP auth bypass and default sk-1234 master keys let attackers steal AI gateway master and provider API keys.
• ShadowRay 2024: Attackers took over internet-exposed Ray AI clusters via the disputed CVE-2023-48022, exposing cloud keys, AI API tokens and GPU compute.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime AI governance is becoming the control plane for enterprise AI security: once models are connected to tools, APIs, and business workflows, pre-deployment review is no longer sufficient. The central problem is not whether the model is approved, but whether its live behaviour can be observed and constrained. That shift makes runtime telemetry, policy enforcement, and response workflows core governance functions, not optional extras. Practitioners should treat runtime governance as the point where AI becomes operationally real.

AI runtime security is a distinct discipline, not a rebranding of application security: enterprise AI systems combine model behaviour, delegated tool use, and dynamic interaction paths that do not map cleanly to traditional web or infrastructure controls. Governance checklists can confirm that a system was approved, but they cannot by themselves detect unsafe tool use, prompt injection, or model manipulation in production. The implication is that AI security programmes need their own operating assumptions and response models.

Visible AI behaviour matters because invisible AI authority is ungovernable: if teams cannot see prompts, responses, tool calls, and MCP-enabled workflows, they cannot establish accountability for the actions the system takes. This is especially important where AI systems are embedded in business-critical environments and inherit real access paths. Practitioners should regard observability as an identity and access requirement for AI workloads, not only a monitoring preference.

AI runtime policy must align with the blast radius of delegated access: once an AI agent can invoke tools or reach data sources, the meaningful question becomes how far that authority can move before containment triggers. The article points to the need for policy controls, detection, and response that operate across models, agents, and tools. That is the right boundary for governance because the security outcome is shaped by the AI’s reachable action space, not by the model alone.

Model security before deployment and runtime protection are now sequential, not competing, controls: scan and assess models before release, then monitor how those models behave after they are connected to production workflows. HiddenLayer’s framing reinforces a broader market direction in which AI governance is maturing into lifecycle control across build, deploy, and operate stages. Practitioners should stop treating these as separate programmes and instead align them as one continuous control chain.

From our research library:

• Enterprise AI use rose from 55% of organisations in 2023 to 88% in 2025, according to McKinsey’s Global Surveys on the State of AI.
• Read next: Agentic AI Security Guide

What this signals

Runtime AI governance is now the practical boundary between experimentation and production. Once AI systems can call tools, retrieve data, and trigger workflows, the organisation needs control over live behaviour, not just pre-launch approval. That makes runtime telemetry and policy enforcement part of the IAM and security operating model for AI workloads.

AI-native security programmes should be built around observable action paths. Security teams need to know which prompts, responses, API calls, and tool invocations are attributable to each AI workflow so that misuse can be investigated without guesswork. The more business-critical the workflow, the more important that evidence becomes for audit and containment.

For practitioners

• Map AI runtime trust boundaries Inventory which models, agents, tools, and MCP-connected workflows can reach business data or execute actions, then define where delegated authority begins and ends.
• Instrument runtime telemetry for AI interactions Capture prompts, responses, tool calls, model decisions, and workflow context so security teams can investigate AI behaviour as it happens.
• Separate pre-deployment review from live enforcement Keep model scanning before deployment, but add runtime policy enforcement and monitoring for the production path where abuse actually occurs.
• Build AI detection playbooks into SOC workflows Define triage steps for prompt injection, unsafe tool use, data leakage, and model manipulation so AI incidents enter established response processes.
• Review delegated access to AI-connected systems Check whether the AI can invoke APIs or retrieve data that would be inappropriate if a human user had the same scope, then narrow those paths where necessary.

Key takeaways

• Enterprise AI security now has to govern live model behaviour, tool use, and workflow execution rather than relying on pre-deployment checks alone.
• The key risk is not only model output quality but also the authority AI systems inherit when they can call APIs, reach data, and act inside business processes.
• Practitioners should align runtime telemetry, policy enforcement, and response workflows so AI behaviour can be observed, constrained, and investigated in production.

Key terms

• AI Runtime Security: AI runtime security is the set of controls that inspect, constrain, and respond to model behavior while the application is live. It includes detection, masking, policy enforcement, and response shaping, all aimed at reducing the blast radius of unsafe model interactions.
• MCP Workflow: An MCP workflow is the sequence of steps an AI agent follows when using the Model Context Protocol to request tools, data, or actions. It defines how the agent discovers capabilities, sends structured requests, receives responses, and continues execution. In security terms, it creates a governed path for agent-to-system interaction.
• AI Policy Compliance: AI policy compliance is the practice of governing how AI is used so that interactions stay within legal, regulatory, and internal boundaries. It combines policy, security, and auditability, but the real test is whether the organisation can enforce rules during live AI behaviour, not just document them after the fact.
• Delegated AI Authority: Delegated AI authority is the permission a person gives an assistant to act inside business systems on their behalf. It turns a conversational tool into an execution layer, which means security teams must govern scope, auditability, and revocation with the same seriousness they apply to privileged access.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.