How can organisations keep LLMs from triggering unsafe actions?

Why This Matters for Security Teams

LLMs do not need direct privilege to create damage; they only need a path from interpretation to execution. Once a model can call tools, trigger workflows, or retrieve secrets, prompt injection and malicious content can turn a language task into an unsafe action. That is why the control point must sit between the model’s output and any privileged system, not after the fact.

NHIMG research on AI Agents: The New Attack Surface report found that 80% of organisations say their AI agents have already acted beyond intended scope, including unauthorised access, sensitive data sharing, and credential exposure. The same pattern applies to LLM-driven workflows: if the model can chain a tool call without policy review, it can amplify a small input flaw into a high-impact event. Current guidance from the OWASP Agentic AI Top 10 treats unsafe action as a control-plane problem, not just a content-safety problem. In practice, many security teams discover unsafe execution only after the model has already touched production systems.

How It Works in Practice

The effective pattern is to separate suggestion from authority. The model can draft a response, recommend a workflow, or assemble a tool request, but a policy gate must evaluate whether the action is permitted in the current context. That gate should inspect the actor, the requested operation, the target resource, the confidence or provenance of the request, and whether the action is reversible or high impact. This is consistent with NIST AI Risk Management Framework guidance on governing AI risk across the full lifecycle.

In operational terms, teams usually need four controls:

• Tool allowlists with explicit per-action constraints, rather than broad “can use tools” permissions.
• Just-in-time approvals for privileged steps such as sending email, moving funds, changing records, or rotating secrets.
• Short-lived credentials tied to a single task, not reusable tokens that survive across prompts or sessions.
• Policy-as-code evaluation at request time, using runtime context instead of static role assumptions.

This is where workload identity matters. The system should authenticate what the workload is, then authorise what it may do right now. For autonomous or semi-autonomous LLM systems, that often means pairing ephemeral OIDC-style tokens or workload identity with a hard decision layer before any side effect is committed. NHIMG’s OWASP NHI Top 10 and the CSA MAESTRO agentic AI threat modeling framework both reinforce that the dangerous moment is execution, not generation. These controls tend to break down when teams let an agent retain broad session tokens across multiple tools because the trust boundary disappears.

Common Variations and Edge Cases

Tighter approval gates often increase latency and operational overhead, so organisations must balance safety against user experience and automation goals. There is no universal standard for every workflow yet, especially where low-risk suggestions and high-risk actions are mixed in one chain.

One common exception is read-only automation. If an LLM only classifies, summarises, or drafts content without access to secrets or write paths, the control gate can be lighter. The risk rises sharply when the same model can later call SaaS APIs, modify records, or trigger downstream jobs. Another edge case is human-in-the-loop review that is too superficial. If reviewers approve actions without seeing the exact tool call, target, and context, the gate becomes symbolic rather than preventive.

Security teams should also watch for hidden privilege inheritance. A benign-looking assistant may inherit access from the user’s browser session, a service account, or a shared integration token. That is where guidance from NIST AI Risk Management Framework and the MITRE ATLAS adversarial AI threat matrix becomes practical: constrain tool reach, minimise credential lifetime, and log every approval decision. NHIMG’s LLMjacking: How Attackers Hijack AI Using Compromised NHIs shows why exposed credentials are especially dangerous when attackers can move from secret theft to model abuse in minutes.

Related resources from NHI Mgmt Group

• When should organisations use AI to help manage NHIs?
• What is the difference between logging actions and logging intent for AI agents?
• How can organisations prevent AI agents from becoming overprivileged?
• When should organisations treat an AI agent as a privileged system?