OWASP Top 10 for LLMs exposes where appsec breaks

At a glance

What this is: This is a StackHawk analysis of the OWASP Top 10 for LLM Applications and how LLM-specific risks change the application security model.

Why it matters: It matters because IAM, PAM, and security teams now have to govern LLMs, AI agents, and the secrets and permissions they touch, not just human users and conventional apps.

By the numbers:

• 17 minutes and as quickly as 9 minutes

👉 Read StackHawk's OWASP Top 10 for LLM security risks analysis

Context

LLM security breaks the assumptions behind traditional application security because the model does not behave deterministically, the context it consumes can alter its behaviour, and the same prompt can produce different outputs across runs. In practice, that means appsec teams are no longer protecting only code paths, but also model instructions, retrieval layers, tool access, and the secrets that support those flows.

The primary governance gap is that many organisations still treat LLMs like standard applications with an added chatbot layer. Once an LLM can read external context, call tools through MCP, or expose sensitive material in responses, identity, privilege, and secret-handling controls become part of the application security problem, not a separate programme.

This is where NHIMG’s identity lens matters: once an LLM can act, it becomes a software entity with runtime permissions, delegated access, and a growing need for explicit identity governance. That makes the intersection with NHI, secrets management, and PAM operational rather than theoretical.

Key questions

Q: What breaks when an LLM can act without strict permission boundaries?

A: The main failure is that the model starts behaving like an over-privileged service account. If it can read, write, send, or execute without scoped approval, prompt injection or bad context can turn a small input problem into unauthorised data access, workflow abuse, or operational disruption.

Q: Why do NHIs complicate zero trust and least privilege efforts?

A: NHIs complicate zero trust because they are numerous, persistent, and often tightly integrated into applications and pipelines. If teams cannot see every identity or keep permissions aligned to actual usage, they cannot consistently prove least privilege. Continuous review and revocation are essential, not optional.

Q: How do security teams know whether an LLM is operating safely?

A: Look for evidence that the model’s read scope, tool scope, and output handling are all bounded and reviewed. If the system can only act on approved data, only invoke approved tools, and logs those actions clearly, governance is working. If any of those are missing, the model is outside control.

Q: Should organisations allow AI agents to use production credentials?

A: Only if those credentials are task-scoped, closely monitored, and revocable without affecting unrelated systems. In most cases, production credentials create unnecessary blast radius. A safer pattern is short-lived access, explicit approvals for non-read actions, and strong separation between agent identity and human privilege.

Technical breakdown

Why LLMs create a different attack surface than traditional apps

Traditional applications are deterministic. Given the same input, they are expected to produce the same output, which makes validation, testing, and policy enforcement comparatively stable. LLMs are probabilistic systems influenced by prompts, model weights, training data, retrieval content, and tool context. That means attackers can manipulate behaviour through instruction conflicts, malicious content in retrieved documents, or malformed outputs that downstream systems trust. The real architectural change is that the model is both a decision engine and a data consumer, so security has to account for control of behaviour as well as confidentiality of information.

Practical implication: Treat model input, retrieval data, and tool output as separate trust boundaries with distinct controls.

How MCP, RAG, and vector stores expand LLM risk

MCP connects AI systems to tools and data sources, while RAG and vector databases supply external context at inference time. Those layers increase usefulness, but they also expand the blast radius of any compromise, because the model can retrieve sensitive documents, surface protected data, or trigger tools with elevated permissions. If access control is weak at the retrieval or tool layer, the model becomes a broker for data exposure rather than a simple interface. The security issue is not only model safety. It is whether the surrounding architecture enforces identity, least privilege, and data scope at every hop.

Practical implication: Lock down retrieval permissions, tool scopes, and data segmentation before expanding LLM access to production systems.

What excessive agency means in LLM security

Excessive agency occurs when an LLM is allowed to take actions with meaningful business impact without adequate constraints or human approval. In practice, that can mean reading mail, sending messages, writing to databases, invoking shell commands, or chaining tool calls in ways the operator did not intend. The risk increases when the model can decide both what to do and when to do it, because a prompt injection or poisoned context can turn a recommendation engine into an action engine. From an identity perspective, that is an NHI governance problem: the model is acting with delegated privilege and needs explicit control of what it may touch.

Practical implication: Use approval gates, step-up authorisation, and task-scoped permissions for any model that can act outside read-only mode.

Threat narrative

Attacker objective: The attacker wants to turn the LLM’s trusted context and delegated permissions into a path for data theft, action abuse, or business disruption.

1. Entry begins when an attacker supplies malicious prompt content, poisoned retrieval material, or manipulated external context that the LLM will ingest.
2. Escalation occurs when the model follows that content into tool use, reveals sensitive data, or executes actions through over-broad downstream permissions.
3. Impact lands as data leakage, unauthorised transactions, corrupted decisions, or abuse of connected systems at scale.

NHI Mgmt Group analysis

LLM security is now an identity governance problem as much as an application security problem. Once a model can read context and invoke tools, it behaves like a software entity with delegated privileges. That shifts the control question from "is the model safe" to "what can this identity do, for how long, and under what conditions." Security teams need to align AppSec with IAM, PAM, and secrets governance because the model cannot be separated from the permissions it uses.

Prompt injection is only the visible symptom of a deeper trust failure. The real issue is that LLM architectures often trust retrieved content, user input, and tool outputs too readily. When an attacker can influence the context window, they can influence the action path. That is why OWASP-style guidance matters, but only if it is paired with enforceable runtime controls and not treated as a checklist.

Least privilege for AI systems must be operational, not theoretical. Many teams still grant broad access to make prototypes work, then keep those permissions in production. The result is an environment where the model can read more, act more, and expose more than intended. The named concept here is delegated model privilege drift: permissions expand quietly as LLM features accumulate, and governance never catches up.

Vector and retrieval layers need the same discipline as privileged data paths. A shared embedding store or permissive document retrieval layer can leak information across tenants, roles, or workflows even when the model itself is well prompted. That means data boundaries, access reviews, and tenant isolation are part of AI security architecture, not just data platform hygiene.

Enterprise AI governance will increasingly be measured by containment, not capability. The organisations that can prove read scopes, tool scopes, and output controls are moving faster than the ones focused only on model quality. For practitioners, the decisive question is whether their governance model can limit harm when the model is manipulated, not whether it can demonstrate useful responses under ideal conditions.

What this signals

LLM adoption is forcing security teams to move from static review models to runtime governance. The more often an AI system can read context, call tools, and interact with production data, the more the programme depends on controls that look like identity enforcement rather than traditional app scanning.

Delegated model privilege drift: this is the operational pattern where LLM capabilities expand faster than access reviews, ownership, and revocation processes. Once that drift starts, the organisation is no longer managing a tool. It is managing a privileged software actor whose permissions must be bounded, monitored, and retired like any other NHI.

Teams should expect AppSec, IAM, and data security to converge around model access paths. The practical priority is to prove which data sources the model can touch, which actions it can take, and which controls stop it from exceeding that scope, especially where MCP or RAG adds more connected systems.

For practitioners

• Classify every LLM as a privileged runtime Inventory each production model, the data it can read, the tools it can call, and the accounts or API keys behind those actions. Then assign explicit ownership and review cadence for those permissions, just as you would for a high-risk service account. Use this as the basis for PAM and NHI oversight.
• Segment prompts, retrieval, and tool execution Do not let the same trust boundary cover system prompts, vector stores, and action endpoints. Separate retrieval permissions from execution permissions, and restrict each layer to the minimum data and functions required for the workflow. This reduces the chance that prompt injection becomes tool abuse.
• Add approval gates for non-read actions Require human approval or step-up authorisation before the model can send messages, modify records, execute code, or trigger external workflows. Reserve autonomous action only for tightly bounded tasks with low blast radius and clear rollback paths.
• Treat secrets in prompts as exposed credentials Scan prompts, connectors, and retrieval content for API keys, tokens, certificates, and passwords. Remove them from prompt text, rotate anything already exposed, and ensure service accounts are not embedded in instructions or documentation that the model can echo back.

Key takeaways

• LLM security fails when teams treat probabilistic systems like deterministic applications with a chatbot on top.
• The main risk is not only model output quality, but delegated access to data, tools, and secrets that the model can misuse or expose.
• Governance now depends on scoping model identity, limiting tool permissions, and proving containment before the system can act autonomously.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Excessive agency: A condition where an AI system is given more operational authority than its task requires. The risk is not just poor output. It is that mistakes, manipulation, or compromise can produce destructive actions at machine speed across the systems the agent can reach.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Delegated Model Privilege Drift: Delegated model privilege drift is the gradual expansion of an AI system’s access as features, connectors, and tools are added over time. The result is a privileged runtime whose permissions no longer match its original purpose or security review.

What's in the full article

StackHawk's full blog covers the operational detail this post intentionally leaves for the source:

• Expanded examples for each OWASP risk category, including prompt injection, system prompt leakage, and vector store weaknesses
• Implementation context for output validation, rate limiting, and access controls in AI-enabled application workflows
• Practical security guidance for teams building LLM features into existing applications and delivery pipelines
• Examples of how AI gateways and model-adjacent controls fit into broader AppSec practice

👉 StackHawk's full post covers the OWASP risk breakdown, examples, and remediation guidance for LLM applications.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and agentic AI identity. It gives security practitioners a practical way to connect identity controls to emerging AI and automation risk.