LLM security best practices need runtime governance, not static controls

At a glance

What this is: NHI Mgmt Group's analysis, drawing on WitnessAI's blog post: LLM security is a runtime governance problem because static, keyword-based controls do not adequately govern enterprise AI tools, embedded copilots, models, and autonomous agents.

Why it matters: This matters because IAM, PAM, and NHI teams now have to govern AI interactions, permissions, and tool use as live access decisions rather than after-the-fact review.

Context

Large language model security is no longer just a model-safety issue. Once employees, developers, copilots, and agents all interact with business data and production systems, the control problem becomes one of access, policy, and runtime enforcement.

The governance gap appears when AI activity happens outside the stack that already monitors identities, entitlements, and system behaviour. In practice, that creates a boundary problem for IAM and NHI programmes because the system making the call may be a human, a chatbot, or an agent operating through delegated permissions.

The article treats that operating model as typical of current enterprise AI adoption, not an edge case. That is the point: the risk has moved from experimentation into ordinary business workflows.

Key questions

Q: How should security teams manage permissions for AI agents?

A: Security teams should regularly assess and update the permissions granted to AI agents to ensure they align with their intended scope. Implementing a governance framework that details access levels and usage policies is crucial to mitigate risks. Moreover, continuous monitoring can detect irregular permissions that may increase exposure.

Q: Why do static controls fail for enterprise LLM security?

A: Static controls fail because they are designed to inspect known content, not to govern context, intent, and action. LLMs can blur instruction and data, while agents can trigger tool calls that never appear as obvious malicious text. Effective security has to inspect interactions at runtime and apply policy when the risk is actually present.

Q: What signals show that AI access is outpacing governance?

A: The clearest signals are broad inherited permissions, large volumes of stale guest access, and inconsistent file classification across teams. If remediation is happening only after AI rollout, the programme is reacting to exposure rather than controlling it.

Q: Should organisations block AI tools or enable them safely?

A: Organisations should enable AI safely rather than rely on blanket blocking. Bans often push employees toward personal accounts and unmonitored tools, which reduces visibility and increases risk. A safer model combines approved AI paths, data classification, monitoring, and clear enforcement for prohibited content.

Technical breakdown

Why static controls miss LLM behaviour

Static controls are built to spot known tokens, fixed destinations, or predetermined rules. LLM interactions are different because the same prompt can be benign, risky, or malicious depending on context, role, and downstream action. A keyword filter cannot reliably distinguish a customer-summary request from an attempt to move sensitive data into an unsanctioned model. NIST’s AI guidance reflects this problem by noting that LLM architecture blurs the data and instruction channels, which means the input itself can change system behaviour. That is why traditional packet-centric inspection and regex matching underperform when prompts, tool calls, and agent actions become the security surface.

Practical implication: Shift detection and policy decisions from content matching alone to context-aware runtime controls.

Intent-based classification and graduated enforcement

Intent-based classification looks at the purpose of an AI interaction rather than isolated words. It is designed to catch cases where a prompt appears ordinary but the behaviour crosses policy boundaries, such as an employee pasting sensitive information into a public chatbot or an agent trying to trigger an unauthorised action. Graduated enforcement then turns that context into allow, warn, block, or route decisions. That matters because binary allow-or-block controls either over-restrict legitimate work or push usage back into shadow channels. The architectural point is that policy has to follow risk, data classification, and destination model together, not sit above them as a generic rule set.

Practical implication: Build policy tiers that can route low-risk AI use while intervening only when intent, data, or destination raises exposure.

Runtime guardrails for prompts, responses, and agent actions

Runtime guardrails inspect both the prompt before it reaches the model and the response before it reaches the user. That bidirectional check matters because clean inputs can still produce harmful, sensitive, or policy-breaking outputs, especially in retrieval-augmented or tool-using systems. For AI agents, guardrails also need to cover tool calls, because the security event may be an action rather than a sentence. This is where AI security overlaps with IAM and NHI governance: the system is not just generating text, it is acting through permissions. Continuous enforcement at runtime is the only control layer that can see both conversation and consequence.

Practical implication: Monitor outputs and tool calls with the same seriousness as inputs, because the risky event may occur after the model answers.

Threat narrative

Attacker objective: The objective is to use AI interaction paths to bypass governance, access sensitive data, or trigger unauthorised actions through trusted enterprise workflows.

1. Entry occurs when employees, developers, or agents interact with AI tools through browsers, native apps, embedded copilots, APIs, or MCP connections outside conventional monitoring.
2. Credential or permission exposure follows when AI systems are allowed to act with privileges that exceed the task or when delegated tools can invoke sensitive actions without adequate oversight.
3. Escalation happens when the model, prompt, or agent interaction crosses policy boundaries and reaches data, systems, or workflows the organisation did not intend to expose.
4. Impact is policy failure at runtime, including data leakage, unauthorised actions, unsafe outputs, or agent-driven changes that are difficult to reconstruct after the fact.

Breaches seen in the wild

• CoPhish OAuth phishing via Copilot Studio: Datadog showed Copilot Studio agents on a Microsoft domain can front OAuth consent phishing and forward stolen tokens; no victims reported.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Runtime governance is now the defining control plane for enterprise AI. The article makes a case that LLM risk is not adequately controlled by static content filters, perimeter inspection, or one-time policy design. Once AI systems can read, transform, and act on business data, policy has to be enforced at the moment of interaction. For identity teams, that means the governance question shifts from who logged in to what the system was allowed to do in that session.

AI agents should be governed as non-human identities, not as a side topic in application security. Their permissions, tool access, and delegated authority create the same lifecycle questions that have long applied to service accounts and workload identities. The difference is behavioural autonomy, which makes inventory, approval, and attribution more important, not less. That is why agent governance belongs inside NHI and IAM operating models rather than in a separate AI exception process.

Shadow AI creates a visibility problem that looks operational before it becomes security-critical. Unapproved chat tools, embedded copilots, and API-driven agent flows can sit outside procurement, logging, and policy review until something goes wrong. The named concept here is AI governance drift: the gap between the organisation’s written AI policy and the live runtime paths where AI actually operates. The practitioner conclusion is to treat discovery and enforcement as a single programme, not two disconnected tasks.

Runtime policy must be contextual if it is to preserve business value. Blanket blocking will push users into unsanctioned tools, while unconditional allow creates hidden exposure. Graduated enforcement, bidirectional guardrails, and auditable routing are the controls that make sanctioned AI use possible at scale. That is the practical lesson for security leaders: governance fails when it cannot distinguish acceptable productivity from unacceptable risk.

Continuous evaluation is the missing maturity signal in most AI programmes. Static tests quickly decay because prompts, model behaviour, and attack techniques change together. Organisations need recurring red-team evaluation, category-level measurement, and drift tracking if they want defensible AI governance. The implication for practitioners is clear: if you cannot measure runtime control performance, you do not yet have operational AI security.

From our research library:

• 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job, according to the 2026 Infrastructure Identity Survey.
• 62% of organisations are at least experimenting with AI agents, but only 23% have scaled one into a single business function, according to McKinsey's 2025 State of AI survey.
• Read next: Agentic AI Identity Guide

What this signals

AI governance drift: the gap between written AI policy and live runtime AI behaviour is now a practical control problem, not a future risk. Teams that cannot see prompt flows, tool calls, and agent actions will keep discovering that policy exceptions are actually the normal operating mode.

The most effective programmes will move AI from a shadow-use conversation into the same governance discipline used for privileged access and non-human identities. That means discovery, authorisation, runtime policy, and evidence collection need to operate as one control chain, not as separate initiatives.

For practitioners

• Map all AI interaction paths Inventory chat tools, embedded copilots, agent APIs, native apps, and MCP connections so sanctioned and shadow AI usage is visible in one place.
• Classify AI use by intent and destination Apply policy that considers role, data sensitivity, and model destination together instead of relying on keywords or application allow-lists alone.
• Adopt graduated runtime enforcement Use allow, warn, block, and route decisions so low-risk AI work continues while sensitive prompts or outputs are handled differently.
• Govern AI agents as delegated identities Assign owners, approve tools, constrain MCP servers, and review the privileges of autonomous agents on a defined cadence.
• Build tamper-evident audit trails Log prompts, responses, agent actions, and policy decisions so security, legal, and compliance teams can reconstruct what happened later.

Key takeaways

• LLM security fails when organisations rely on static content inspection for systems that now generate, transform, and act on business data in real time.
• The control gap is not only technical visibility but governance of prompts, outputs, tool calls, and delegated agent actions across sanctioned and shadow AI use.
• Runtime guardrails, intent-based classification, and auditable agent governance are the practical controls that turn AI use into something security teams can govern.

Key terms

• Runtime Governance: Runtime governance is the set of controls that verify what a system or agent is actually doing after deployment. It combines monitoring, authorization checks, and access validation so teams can detect drift, misuse, or excessive privilege in motion rather than assuming build-time policy still holds.
• Intent-based classification: Intent-based classification evaluates what a user or system is trying to do, not just what text or file is present. In AI governance, it distinguishes routine work from risky interaction by reading context, purpose, and sensitivity. That matters when regulated data is handled conversationally rather than through formal file transfer.
• Graduated Enforcement: A policy model that uses more than allow or block. It can warn, route, tokenize, or permit an interaction based on the risk level, which helps organizations preserve productivity while controlling sensitive AI use.
• Agent Governance: Agent governance is the set of policies, controls, and evidence required to manage autonomous software as a non-human identity. It covers consent, tool access, lifecycle review, audit logging, and revocation so that an agent remains bounded as its workflows change.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and agentic AI identity. It helps security practitioners connect identity controls to the runtime decisions now made by AI systems.