How should organisations govern tool use when LLMs can act on instructions?

Why tool use by LLMs must be governed like access, not text generation

Once an LLM can trigger actions in business systems, the meaningful security question is no longer whether the model can produce a plausible instruction. It is whether the application should allow that instruction to become a real operation. That shifts the control point from prompt quality to policy enforcement, scope checking, and contextual approval before execution.

The most important design principle is to separate suggestion from authority. A model can propose a file change, data lookup, ticket update, or API call, but the runtime must decide whether the actor is permitted to do it, whether the target is in scope, and whether the current state justifies the action. That distinction is what prevents a fluent response from becoming an unsafe side effect.

Tool governance also has to account for context that can be manipulated indirectly. Retrieved documents, embedded memory, and prior conversation state can shape what the model tries to do, which means the trust boundary is not just the user prompt. Any system that lets context influence tool invocation needs explicit controls over which inputs can authorize which tools, and under what conditions.

What good tool governance looks like in practice

A sound pattern is to treat every tool call as a policy decision with clear preconditions. The application should validate the requested action against identity, role, object, environment, data sensitivity, and transaction context before the call is issued. If the model asks for something outside policy, the safe response is to deny, degrade, or route to human review rather than trying to make the model “more careful.”

That means tool design matters as much as model design. Keep tools narrow, explicit, and predictable. Prefer discrete operations with bounded parameters over flexible “do anything” endpoints, and require the calling layer to assemble any higher-level workflow from individually governed steps. This is where permission-aware retrieval, scoped connectors, and context filtering reduce blast radius without relying on the model to self-restrain.

Good governance also means defining which decisions can be automated and which cannot. Low-risk, reversible, and well-bounded actions may be suitable for direct execution, while changes with material data, financial, or operational impact should require stronger approval, confirmation, or secondary checks. The practical test is not whether the model is confident, but whether the action is attributable, reversible, and within a tolerated blast radius.

How policy breaks when prompts, memory, or retrieval shape execution

The hardest failures usually come from indirect influence rather than obvious abuse. A retrieved document can smuggle an instruction, a stale memory entry can bias a workflow, or a connector can expose data that causes the model to choose an unsafe tool path. In those cases, the weakness is not just bad prompting, it is missing authorization boundaries between what the model can read and what it may do.

That is why tool use needs to be evaluated as part of the application’s control plane. A system should be able to explain why a call was allowed, what policy rule approved it, and which context elements were considered. If you cannot reconstruct that decision path, you do not have reliable governance, only model behaviour that happens to be correct some of the time.

For a broader treatment of agent behaviour, tool misuse, memory poisoning, and identity abuse, see the Agentic AI Security Guide. For a focused view of memory and context controls, the AI Agent Memory Security Guide shows why cross-session state needs strict isolation and write controls. When tool access depends on upstream identity and credentials, the AI Infrastructure Workload Identity Guide is the right companion reference.

Risk and Threat Considerations

Tool-enabled LLMs expand the attack surface because a successful prompt or context manipulation can become an authenticated action in another system. The main risk is not that the model “gets confused,” but that it is induced to use a legitimate integration in ways the user, policy, or environment never intended.

Failure mechanism: An attacker, malicious document, or poisoned memory steers the model toward a tool call that exceeds intended scope, touches sensitive data, or performs a harmful write operation without a separate authorization check.

Impact: That can produce unauthorized access, data leakage, destructive changes, fraudulent transactions, or lateral movement through connected systems, especially when the tool has broad privileges or weak approval boundaries.

Practitioner Guidance

What to verify: Verify that every tool has an explicit policy decision point before execution, not just after the model has already chosen the action. If the tool can change state, access sensitive records, or reach external systems, the approval logic should be visible, testable, and logged.

Decision rule: If the action would be unsafe for a human operator to perform blindly, do not let the model perform it blindly either. Require a stronger control for anything irreversible, high impact, or sensitive to context manipulation.

Common mistake: Teams often secure the prompt and forget the tool boundary. That is backwards for agentic workflows, because the real security event is the executed action, not the generated recommendation.

Practitioner takeaway: Govern the tool layer as an authorization system with model assistance, not as a chat interface with extras; if you cannot constrain and explain the action, you have not actually controlled the agent.

Related resources from NHI Mgmt Group

• How can organisations govern AI agents that use service accounts and tokens?
• How should teams govern AI agents that use MCP?
• How should security teams govern AI agents that use OAuth access?
• When should organisations use AI to help manage NHIs?