OWASP Top 10 for LLMs exposes where appsec breaks

At a glance

What this is: StackHawk maps the OWASP Top 10 for Large Language Model Applications to the main places LLM security breaks down, from prompt injection and data leakage to excessive agency and vector and embedding weaknesses.

Why it matters: This matters because IAM, NHI, and AppSec teams now have to govern prompts, outputs, retrieval layers, and tool delegation as security boundaries, not just model behaviour.

Context

LLM security is not just another application security variant. These systems are probabilistic, can change behaviour based on input, and often depend on retrieval, plugins, and tool access that sit outside classic request and response controls.

That breaks assumptions embedded in traditional AppSec, IAM, and even secrets governance. When an LLM can read context, retrieve data, or trigger downstream actions, the security question becomes how to govern decision paths, not only how to validate input fields.

Key questions

Q: What breaks when LLM security is enforced only in the application layer?

A: Alternate routes bypass the controls. Direct API access, service-to-service integrations, and compromised workloads can all reach the model without passing through middleware, input validation, or application-level authorization checks. Once that happens, security teams lose visibility into the request and cannot reliably control the response path either.

Q: How should security teams implement guardrails for LLM applications to reduce prompt injection and excessive agency risks?

A: Start by limiting what the model can do, not just what it can say. Constrain tools, permissions, and external actions to the smallest necessary scope, then validate inputs and outputs before they reach downstream systems. Add human approval for sensitive actions, isolate untrusted content, and red team the application so indirect prompts do not turn into unauthorized execution.

Q: How can security teams tell whether LLM evaluation is actually working?

A: Look for continuous coverage across live traffic, regression testing on new prompts or models, and sampled human review of judge outputs. If evaluation only runs during development, it is not detecting drift, prompt changes, or attack-path changes in production. Working evaluation should change deployment decisions, not just produce dashboards.

Q: Should organisations prioritise retrieval isolation or output filtering first for LLM security?

A: Prioritise retrieval isolation when the model can access sensitive sources or shared vector stores, because exposed context can undermine every later control. Output filtering still matters, but if the model sees the wrong data in the first place, downstream sanitisation cannot fully recover the boundary.

Technical breakdown

Why traditional application security fails against LLM behaviour

Traditional AppSec assumes deterministic software, where a given input should produce a predictable result and where controls can validate requests, responses, and state transitions. LLMs are probabilistic systems that generate outputs from training patterns, context, and runtime prompts, so the same input can yield different behaviour over time. That makes classic schema validation and static business rules insufficient when the model itself is part of the control surface. The architectural shift matters because security must now account for mutable context, external retrieval, and delegated actions, not just the application code around the model.

Practical implication: Treat LLM runtime behaviour as a governed control plane, not as ordinary input processing.

Prompt injection, output handling, and delegated actions

Prompt injection works when attacker-controlled text overrides intended instructions, either directly through a user prompt or indirectly through content the model reads. Improper output handling turns the model into a delivery mechanism for injection, code execution, or exfiltration when its output is passed into a shell, database, browser, or API without validation. Excessive agency is the next step up the stack: once the model can invoke tools or act without approval, a successful prompt manipulation can become an operational action. These are distinct failures, but they chain together quickly.

Practical implication: Apply separate controls for prompt trust, output trust, and tool-authorisation boundaries.

RAG and vector database weaknesses create a new access layer

Retrieval-Augmented Generation adds a retrieval and storage plane that conventional application controls often do not cover well. Vector databases can leak information across tenant boundaries, expose embeddings that can be inverted, or surface poisoned content that shapes model responses. This is not only a data problem but also an access problem, because the retrieval layer determines what the model can see before it answers. In practice, RAG systems extend the identity and entitlement problem into a new substrate where access controls, isolation, and data governance all have to work together.

Practical implication: Review vector stores and retrieval pipelines as access-controlled assets, not as passive data backends.

Threat narrative

Attacker objective: The attacker aims to make the LLM reveal restricted data or perform actions beyond its intended authority.

1. Entry occurs when a user or external document supplies malicious instructions that the LLM ingests as trusted context.
2. Credential access or abuse follows when the model exposes sensitive information, internal prompts, or connected system details through its response path.
3. Escalation happens when excessive agency lets the manipulated model invoke tools, send data, or trigger downstream actions without human approval.
4. Impact is unauthorised disclosure, unsafe system actions, poisoned decisions, or resource abuse across the application stack.

Breaches seen in the wild

• Microsoft Azure OpenAI abuse by Storm-2139: Storm-2139 used API keys leaked by Microsoft customers to hijack Azure OpenAI, bypass safety guardrails and resell access to generate harmful content.
• AI LLM hijack breach: attackers used stolen AWS access keys to hijack Anthropic LLM models on Bedrock.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

LLM security debt is now a governance problem, not a feature problem. The article shows that LLM risk spans prompt injection, retrieval layers, output handling, and delegated tool use, which means the security boundary is no longer the app alone. Traditional AppSec assumes predictable logic, while LLM systems operate on probabilistic behaviour and external context. Practitioners should treat model governance as part of the control stack, not as an experiment layered on top.

Prompt trust and output trust are separate failure modes and must be controlled separately. A model can be manipulated at input time and then used as an injection vehicle at output time, which means a single validation layer will not solve the problem. This is where classic secure coding assumptions break down for AI systems. Security teams need to govern both what the model reads and what downstream systems accept from it.

Vector and embedding weaknesses create a hidden access layer for AI systems. Retrieval is not just about relevance. It is a privileged data path that can cross tenant boundaries, surface poisoned content, and leak information that never should have reached the model context in the first place. This is a governance issue for identity, data access, and AI control planes at the same time, and practitioners need to stop treating vector stores as neutral infrastructure.

Excessive agency is the point where LLM risk becomes non-human identity risk. Once a model can choose tools and act without approval, it behaves like a delegated non-human actor with authority over downstream systems. That brings NHI governance into the centre of AI security, because the question becomes who or what is authorised to act, under what constraints, and with what evidence trail. Practitioners should align AI agent authorisation to the same scrutiny used for other privileged non-human identities.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• The average time to mitigate a leaked secret is 36 hours, highlighting the operational burden of manual remediation processes, according to the 2024 State of Secrets Management Survey.
• Read next: AI Agent Authorisation Guide

What this signals

LLM security now needs governance at the point of decision. Organisations that wait until the model has already generated an output are defending the wrong boundary. The practical shift is to control what the model can read, what it can invoke, and what it is allowed to do with retrieved context before autonomous steps complete.

Non-human identity discipline belongs in AI security programmes. Once an LLM can act through tools and connectors, the model is operating as a delegated non-human actor with a real authority surface. That means authorisation, auditability, and lifecycle control should be applied to AI systems with the same seriousness used for other privileged machine identities.

Retrieval governance is now part of data security, not just AI engineering. Vector stores, embeddings, and connected knowledge bases can leak or reshape sensitive information even when the model itself is not compromised. Security teams should align retrieval controls to the organisation’s broader identity and data access model, not leave them as an engineering convenience.

For practitioners

• Define separate trust boundaries for prompts, outputs, and tool calls Do not treat model input validation as coverage for everything the system does. Apply distinct controls for prompt ingestion, response filtering, and any action the model can trigger in other systems.
• Remove secrets and sensitive terms from system prompts System prompts are not secure storage. Keep credentials, internal process details, and role logic outside the prompt layer and rotate anything already exposed there.
• Restrict model access to downstream tools by task and scope Grant the model only the minimum set of actions needed for a specific workflow, and require approval for any operation that can send data, change state, or touch production systems.
• Audit retrieval and vector databases for tenant isolation Review retrieval pipelines for cross-tenant leakage, poisoned documents, and overbroad access to embeddings or chunks. Treat the retrieval layer as an authorised data path, not an index.
• Test LLM applications adversarially before production Use prompt injection, unsafe output handling, and delegated-action scenarios in testing so security controls are validated against realistic abuse paths rather than assumed coverage.

Key takeaways

• LLM applications fail in places where conventional AppSec assumptions stop holding, especially around prompt trust, output trust, and delegated action paths.
• The article’s core risks are not theoretical edge cases. They include prompt injection, sensitive information disclosure, excessive agency, system prompt leakage, and vector database weaknesses.
• Security teams need separate controls for retrieval, output handling, and tool authorisation if they want to govern LLMs without turning them into uncontrolled business logic.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Excessive agency: A condition where an AI system is given more operational authority than its task requires. The risk is not just poor output. It is that mistakes, manipulation, or compromise can produce destructive actions at machine speed across the systems the agent can reach.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• System Prompt Leakage: System prompt leakage is the exposure of hidden prompt content to users or attackers. The real security problem is usually not the prompt itself, but the secrets, policy logic, and internal architecture details placed inside it. If those details are sensitive, they should live in code or secrets management instead.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps practitioners connect delegated access, auditability, and lifecycle control to the systems they already run.