OWASP LLM top 10 shows where app security breaks down

At a glance

What this is: WorkOS maps the OWASP Top 10 for LLM applications to the security gaps that emerge when probabilistic model behaviour, runtime retrieval, and tool use are folded into application flows.

Why it matters: IAM and appsec teams need to treat LLM features as trust-boundary problems, because authorization, output validation, and data retrieval now have to hold even when the model itself cannot be trusted as a control point.

Context

LLM-enabled applications break a core assumption in classical application security: inputs are not always deterministic, and outputs are no longer safe to treat as passive text. Once a model can read external content, retrieve data, and trigger downstream actions, the security problem shifts from a single application boundary to a chain of model, retrieval, and authorization decisions.

This matters to IAM because the model often sits between the user and the systems that hold data or perform actions. If the application relies on prompt text, shared service accounts, or post-hoc filtering to decide who may see or do what, the identity layer has already lost control before the request is completed.

Key questions

Q: What breaks when LLM security is enforced only in the application layer?

A: Alternate routes bypass the controls. Direct API access, service-to-service integrations, and compromised workloads can all reach the model without passing through middleware, input validation, or application-level authorization checks. Once that happens, security teams lose visibility into the request and cannot reliably control the response path either.

Q: Why do LLM apps need permission-aware retrieval before generation?

A: Because once the model has seen data, the exposure has already happened. Permission-aware retrieval prevents the model from reading content the user is not entitled to see, which is especially important in multi-tenant RAG systems and shared assistants. The right control point is the data access decision, not a warning in the prompt after retrieval has already occurred.

Q: What are the signs that an LLM has been granted more agency than it should have?

A: Common warning signs include unnecessary tools exposed to the model, broad database or API permissions, support workflows that can edit or delete records, and debug features still active in production. Another clue is when routine tasks require admin-level access. If the model can reach infrequently used tables, restricted operations, or sensitive data outside its purpose, the boundary is probably too loose.

Q: How should teams separate LLM output handling from application trust decisions?

A: Treat the model output as untrusted input and validate it before any downstream system acts on it. That means schema checks, context-aware encoding, and blocking outputs that do not match the expected format or permission scope. The application must decide what is allowed, while the model only proposes text or structure for review.

Technical breakdown

Why prompt injection defeats model-only guardrails

Prompt injection works because the model receives instructions and untrusted content through the same channel. Direct attacks manipulate the active prompt, while indirect attacks bury instructions in content the model later ingests, such as emails, PDFs, web pages, or tickets. The result is that the model may follow attacker-supplied instructions even when the developer intended it to act only on trusted system prompts. This is why techniques like RAG or fine-tuning reduce exposure but do not remove the structural problem. Practical implication: put hard boundaries around what the model can act on, and treat model output as untrusted until application controls validate it.

Practical implication: move enforcement out of prompts and into deterministic controls that validate input, constrain output, and gate high-impact actions.

Why sensitive data leaks through retrieval and shared context

LLM applications leak data when the same model context is reused across users, tenants, or tasks without strict permission checks. The risk is not just memorized training data. It is also runtime exposure when a chatbot, code assistant, or RAG system retrieves information that the requester is not authorised to see. If retrieval happens before authorization, the model can surface data the application should never have exposed. Practical implication: enforce permission-aware retrieval at the data layer, not through instructions to the model, and scope every data source to the identity that is actually authorised.

Practical implication: place access control at retrieval time so the model never sees data the requester is not allowed to access.

How excessive agency turns LLMs into action paths

Excessive agency appears when an LLM is given too many tools, too much privilege, or too much autonomy. A model that can search, send, update, or execute with broad rights can be manipulated into taking actions the user never intended. The security issue is not the model's intelligence. It is the combination of capability and authority. When tool access is broad, prompt injection can turn a harmless request into a destructive one by making the model invoke the wrong tool or act with the wrong identity. Practical implication: scope tool credentials to the task and force downstream systems to enforce authorisation independently of the model.

Practical implication: separate tool capability from authority so the model cannot convert persuasion into privileged action.

Breaches seen in the wild

• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.
• Moltbook AI agent keys breach: Moltbook breach exposed 1.5M AI agent keys.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 150+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

LLM security failures are trust-boundary failures, not model-quality failures: The OWASP Top 10 for LLM applications is really a map of where application teams have been outsourcing security decisions to a probabilistic system. Once a model reads content, retrieves data, or triggers tools, the boundary must be enforced by deterministic controls outside the model. The implication is that security architecture, not prompt craftsmanship, becomes the primary defence line.

Authorization belongs before retrieval, not after generation: Many LLM incidents happen because the application lets the model see data first and checks permissions later, if at all. That assumption is incompatible with multi-tenant or role-scoped environments, where the model may surface sensitive content before any downstream filter can intervene. The implication is that retrieval paths need identity context at decision time, not just audit trails after the fact.

Excessive agency is the closest LLM analogue to privilege creep: When the model has tool access that exceeds the task, prompt manipulation can convert a simple request into an unintended action chain. This is not a new identity problem, but it is a sharper version of one: authority and intent diverge inside the same runtime session. The implication is that app teams need scoped, task-bound authority rather than broad service access.

Identity controls regain importance precisely because the model cannot be the control: The article shows that authentication, RBAC, and fine-grained authorization become more important, not less, when LLMs are introduced. Model behavior is not stable enough to carry security logic, so the governance model has to move back to the application and identity layers. The implication is that LLM adoption strengthens the case for deterministic policy enforcement outside the model.

LLM prompt leakage is a symptom of misplaced security logic: If a system prompt contains credentials, authorization rules, or backend details, the application has already collapsed security logic into a user-visible channel. That practice assumes the prompt can remain private, which is an unsafe premise. The implication is that teams should stop treating prompts as policy stores and re-establish those controls in code and identity systems.

From our research library:

• AI-related credential leaks surged 81.5% year-over-year in 2025, with the surrounding AI infrastructure leaking 5x faster than core LLM providers, according to the State of Secrets Sprawl 2026.
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap, according to the State of Secrets in AppSec.
• Read next: Permission-Aware RAG Guide

What this signals

Permission-aware retrieval is the governing concept for LLM apps: if the model can see data before identity and entitlement checks run, the security model has already failed. That means retrieval controls must be designed like access controls, not like post-processing filters, and they need to evaluate the calling identity at decision time.

LLM features also expose the limits of prompt-centric governance. Prompts can guide behaviour, but they cannot reliably enforce authorisation, prevent cross-tenant disclosure, or stop a model from being manipulated into tool misuse. Teams that move security logic back into the application and identity layer will be better positioned to contain both model error and deliberate abuse.

For practitioners

• Harden retrieval at the identity layer Enforce tenant and document permissions before the model receives any context, using the requester’s identity rather than a shared service account.
• Remove security rules from prompts Move roles, access rules, and approval logic into deterministic application code so prompt leakage cannot expose the authorisation model.
• Scope tool access to the task Give each LLM tool the narrowest credentials and actions required for the job, and avoid persistent privileges that outlive the session.
• Validate every model output before execution Check LLM output against strict schemas before it reaches a browser, database, shell, or email system, and block anything unexpected.
• Log identity, retrieval, and tool calls together Correlate each prompt, data lookup, and downstream action to the session and user identity so unexpected behaviour is traceable.

Key takeaways

• LLM applications break classical app security assumptions because model behaviour is probabilistic and can be manipulated through both direct and indirect prompt injection.
• The most important controls sit outside the model, especially permission-aware retrieval, output validation, and scoped tool authority.
• Teams that keep policy in prompts instead of identity and application code will struggle to bound the blast radius of LLM misuse.

Key terms

• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads, causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• Permission-Aware Retrieval: Permission-aware retrieval is the practice of enforcing access control before data reaches the model. It ensures the model only sees documents, records, or context that the requesting identity is allowed to access. In multi-tenant or sensitive environments, this control is more important than post-processing filters.
• Excessive agency: A condition where an AI system is given more operational authority than its task requires. The risk is not just poor output. It is that mistakes, manipulation, or compromise can produce destructive actions at machine speed across the systems the agent can reach.
• Model Output Validation: Model output validation is the control layer that checks an LLM's response before another system uses it. It applies schemas, encoding, and policy checks so the model cannot directly trigger unsafe browser, database, shell, or workflow behaviour.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.