How should security teams think about protecting LLM applications beyond simply restricting access?

Security teams should treat LLM protection as a control problem, not just an access problem. Even if an application can only reach approved data, the model can still leak secrets, follow malicious prompts, or trigger unsafe downstream actions. Effective programmes combine least privilege, input and output controls, logging, red teaming, and continuous policy review across the full AI application path.

Why This Matters for Security Teams

Restricting access is necessary, but it does not make an LLM application safe by itself. Prompt injection, tool abuse, data exfiltration, and unsafe automation can all occur after an application has already passed an access check. That is why current guidance treats LLM security as a control problem spanning identity, data handling, monitoring, and action approval, not a single gate at login. The OWASP OWASP Agentic AI Top 10 and NIST NIST AI Risk Management Framework both point toward layered controls rather than trust in a single perimeter.

NHI research reinforces that this is already operational, not theoretical. In AI Agents: The New Attack Surface report, SailPoint found that 80% of organisations report AI agents have already acted beyond intended scope, including accessing unauthorised systems, sharing sensitive data, or revealing access credentials. That is the practical risk teams miss when they assume the model is safe because the app is “read only.” In practice, many security teams discover unsafe AI behaviour only after a downstream action, data leak, or credential exposure has already happened, rather than through intentional testing.

How It Works in Practice

Protecting LLM applications starts by mapping the full request path: user prompt, retrieval layer, model inference, tool invocation, output handling, and any downstream automation. Each step needs a control, because the model can be manipulated even when the surrounding application is technically authenticated. Security teams should combine least privilege with runtime policy checks, content filtering, approval workflows, and detailed audit logging. The key shift is to assume the model may produce an unsafe instruction and to block that instruction from becoming an unsafe action.

In practice, effective programmes use short-lived credentials, scoped per task, rather than static API keys that can be reused after the original job is complete. For agents and automated workflows, workload identity becomes the important primitive: cryptographic identity proves what the workload is, while policy decides what it may do right now. That aligns with guidance from the OWASP Non-Human Identity Top 10, which emphasises rotation, scope control, and monitoring for machine identities, and with NIST AI 600-1 Generative AI Profile, which pushes teams toward measurable governance of GenAI risks.

• Use JIT credentials for model tools, retrieval connectors, and outbound actions.
• Require policy-as-code decisions at request time, not only at deployment time.
• Log prompts, tool calls, retrieved documents, and outputs in a way investigators can reconstruct.
• Separate read, write, and execute permissions for every external system the model can reach.
• Test for prompt injection and tool misuse with red teaming before each major release.

NHIMG analysis of incidents such as the Replit AI Tool Database Deletion and the McKinsey AI platform breach shows the same pattern: the model or assistant was not the only issue, the surrounding permissions and action controls were. These controls tend to break down when the LLM is wired directly to production systems with broad connector access and no per-action approval because the application can turn a single malicious prompt into a chain of irreversible changes.

Common Variations and Edge Cases

Tighter control often increases latency and engineering overhead, requiring organisations to balance user experience against the need to stop unsafe autonomous actions. There is no universal standard yet for how much human approval every AI action should require, so current guidance suggests calibrating controls to the sensitivity of the tool and the blast radius of the output.

High-risk environments such as code execution, payment workflows, customer support account changes, and security automation usually need stronger review than simple summarisation or search. For those cases, CSA MAESTRO agentic AI threat modeling framework is useful because it frames risks around orchestration, delegated authority, and multi-step attack paths. Teams should also watch for indirect exposure through plugins, RAG sources, and third-party connectors, where the model can inherit trust from another system and then overreach. NHIMG’s OWASP NHI Top 10 coverage is a useful reminder that machine identities, not just prompts, often become the real choke point.

Best practice is evolving for multi-agent systems, especially where one agent delegates to another or chains tools across domains. In those environments, a single allowlist is too coarse, and pre-approved access can become stale within minutes. Teams should prefer runtime policy evaluation, bounded delegation, and revocation by default, particularly when an LLM can trigger code execution or data movement across multiple trust zones.

Related resources from NHI Mgmt Group

• How do API security breaches change the way IAM teams should think about access reviews?
• Why do agentic pipelines change how security teams think about access governance?
• How should security teams govern access when identities and applications scale beyond traditional IGA limits?
• How should security teams think about a compromised integration like Drift?