AI runtime security shifts into governance for enterprise AI workloads

At a glance

What this is: This is HiddenLayer’s announcement about bringing AI-native security controls into Databricks Unity AI Gateway for enterprise AI runtime workflows.

Why it matters: It matters because IAM and security teams now have to govern AI interactions as runtime access behaviour, not just model deployment and static permissions.

👉 Read HiddenLayer's announcement on AI runtime security for Databricks AI workflows

Context

Enterprise AI security is moving from model-centric review to runtime governance, because production AI systems now retrieve data, call tools, invoke APIs, and execute actions. In that environment, access control alone does not explain what an AI system actually did, or whether its behaviour stayed within intended boundaries.

For IAM, NHI, and security architects, the key issue is that AI workloads increasingly behave like interconnected identity-bearing systems. That means the control surface extends across prompts, responses, tool calls, MCP integrations, and platform workflows, where visibility and response need to follow the action, not just the account.

Key questions

Q: How should security teams govern AI systems that can call tools and APIs at runtime?

A: Security teams should govern AI systems through runtime visibility, policy enforcement, and response workflows, not just model approval. The control point needs to evaluate prompts, tool calls, data access, and downstream actions as they happen. That is the only way to tell whether an AI system stayed within its intended operating boundary.

Q: Why do AI agents complicate traditional access control and audit models?

A: AI agents complicate access control because the security question is no longer only who was authenticated. It is also what the system decided to do with that access, which tools it selected, and whether the action was influenced by adversarial input. Audit models built for static users often miss that behavioural layer.

Q: What breaks when AI security stops at model scanning?

A: Model scanning helps identify tampering and unsafe dependencies before deployment, but it does not address runtime misuse. Once the system is live, prompt injection, unsafe tool use, and manipulated responses can still drive harmful behaviour. Without runtime controls, the most important security decisions happen after the pre-check has already passed.

Q: How can organisations tell whether AI governance is actually working?

A: AI governance is working when teams can observe AI actions, enforce policy at the point of execution, and produce evidence for audit or incident review. If the organisation can only describe what was approved at deployment, but not what happened in production, governance is incomplete.

Technical breakdown

Runtime AI security is about observing behaviour, not just approving access

Enterprise AI security at runtime means monitoring what a model, agent, or workflow actually does after deployment. That includes prompts, responses, tool calls, API requests, and downstream actions across systems such as Databricks governed workflows and MCP-connected services. Traditional controls can confirm that a user or service was allowed in, but they do not explain whether the AI system was manipulated, whether the output was unsafe, or whether a tool invocation crossed policy boundaries. Runtime visibility turns AI behaviour into security evidence.

Practical implication: security teams need telemetry that covers AI interactions end to end, not just identity authentication events.

AI-specific threat detection targets prompt injection, tool misuse, and model tampering

AI threat detection differs from standard application detection because the attack surface includes the model’s context, instructions, and action selection. Prompt injection can redirect behaviour, model manipulation can alter outputs, and unsafe tool use can turn a benign interaction into a harmful action. In agentic systems, the issue is not only whether access existed, but whether the system was influenced into choosing the wrong action. That makes AI threat intelligence a distinct layer, especially where models, agents, and tools share runtime context.

Practical implication: detection rules should be written for AI abuse patterns, not only for conventional network or endpoint indicators.

Governed AI workflows need policy enforcement close to the execution path

A governance layer is most useful when it sits near the AI runtime path, where it can evaluate tool use, data exposure, and policy violations before an action completes. Databricks Unity AI Gateway is positioned as that control point for models, agents, providers, tools, and frameworks, while HiddenLayer adds security depth around attack detection and response. The architecture matters because AI systems can chain retrieval, reasoning, and tool execution in one session. If controls are separated from that execution path, they become advisory rather than enforceable.

Practical implication: place guardrails where AI work happens, so policy can block or flag unsafe actions before they propagate.

NHI Mgmt Group analysis

AI runtime governance is now a security problem, not just a model management problem. The article shows why controls focused only on model approval leave the real risk untouched: production AI systems now retrieve data, invoke tools, and execute actions across business workflows. That changes the governance question from "is the model approved?" to "what did the model, agent, or MCP-connected workflow actually do?" The practitioner implication is that AI security programmes must treat runtime behaviour as the primary control surface.

Runtime AI security creates a distinct identity problem because the actor is both governed and operational. Models, agents, and tool-connected AI workflows sit inside enterprise access paths, but they also make local decisions about what to call, what to retrieve, and what to execute. That collapses the old separation between identity approval and system behaviour. The implication is that IAM teams need to evaluate AI systems as active runtime participants, not as passive applications with static entitlements.

Model security before deployment is necessary, but it does not address runtime compromise pathways. The article’s emphasis on scanning models for malicious code, unsafe dependencies, and tampering is useful, yet it only covers the pre-production edge. Once AI systems are live, the real exposure shifts to prompts, responses, tool calls, and adversarial manipulation of behaviour. The practitioner implication is that pre-deployment review and runtime defence must be treated as separate control layers.

AI security is becoming a governance discipline that spans data, identity, and operational response. The strongest signal in the article is not a single product capability but the emerging shape of the control plane: visibility, policy, threat intelligence, and response all need to work together across models, agents, and tools. That is a broader programme design issue for IAM, IGA, and security operations. Practitioners should assume AI governance will increasingly sit alongside identity governance rather than inside traditional application security alone.

From our research:

• 69% of organisations now have more machine identities than human ones, according to The Critical Gaps in Machine Identity Management report.
• 66% say their current tooling is not adequate to manage the scale of machine identities they now have, which is why runtime governance gaps keep widening.
• For the broader control context, see Ultimate Guide to NHIs , Key Challenges and Risks for the visibility and lifecycle issues that make machine-scale governance brittle.

What this signals

Model governance will not be enough once AI systems become operational participants in enterprise workflows. HiddenLayer’s announcement reflects a broader shift from approval-based oversight to runtime defence, especially where models and agents can retrieve data and invoke tools. The named concept here is AI runtime governance: the discipline of controlling behaviour while the system is active, not only before it is deployed. Practitioners should expect this to become part of wider identity and access governance rather than a separate niche.

With 66% of organisations saying their current tooling is not adequate to manage machine-identity scale, per The Critical Gaps in Machine Identity Management report, the same pattern is likely to surface in AI workflows unless runtime controls are built in from the start. That means teams should prepare for more behavioural telemetry, more policy points, and tighter linkage between security operations and governance evidence.

The practical next step is to align AI runtime monitoring with existing IAM and IGA processes, especially where AI actions touch sensitive data or production systems. Use NIST Cybersecurity Framework 2.0 to structure the response layer, then decide which AI events should trigger review, containment, or policy refinement.

For practitioners

• Map AI runtime touchpoints across the control stack Inventory where models, agents, prompts, tool calls, and MCP integrations are executed, then identify which control point can actually observe each interaction.
• Separate pre-deployment review from runtime protection Treat model scanning, dependency review, and tamper checks as one control layer, then add runtime monitoring for behaviour, misuse, and unsafe tool use.
• Define policy boundaries for AI actions Write guardrails that limit sensitive data access, external API calls, and tool execution by context, not only by broad application role.
• Integrate AI detections into existing response workflows Route AI abuse signals into the same triage and escalation paths used for other security incidents, so runtime anomalies are actionable instead of isolated.

Key takeaways

• AI security now depends on controlling runtime behaviour, not only approving models before deployment.
• The enterprise risk surface expands when agents, tools, and MCP workflows can execute actions across business systems.
• Practitioners should separate model assurance from runtime governance so detection, policy, and response operate together.

Key terms

• AI Runtime Governance: The control discipline that watches and constrains AI systems while they are actively processing, retrieving, and acting. It combines policy enforcement, telemetry, and response so teams can see what the system actually did, not only what it was allowed to do at deployment time.
• Model Context Protocol: An open protocol that connects AI agents to tools and data sources during execution. In practice, it expands the identity and access problem because the AI system can use external capabilities dynamically, which means governance has to account for tool use as well as model output.
• Agentic AI: AI that can choose actions and sequence them at runtime, rather than only producing outputs on request. For governance teams, the important issue is not the label, but whether the system can independently select tools, time its actions, and create security impact without a human approval gate.
• Runtime Telemetry: Security data collected while an AI system is operating, including prompts, responses, tool invocations, and related workflow events. It gives investigators evidence of behaviour in production and helps distinguish normal use from manipulation, unsafe action, or policy violation.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or NHI governance in your organisation, it is worth exploring.

This post draws on content published by HiddenLayer: HiddenLayer joins Databricks Unity AI Gateway ecosystem to bring AI-native security to enterprise AI workloads. Read the original.