Most teams can name LLM01 Prompt Injection, but few can show an auditor which of its seven OWASP mitigations they have implemented, how each one is tested and which of the nine OWASP scenarios their application would survive. This guide treats LLM01:2025 as an assessable requirement set: it maps every OWASP mitigation to an owner, a design pattern and a piece of evidence, turns the nine attack scenarios into a defensive test plan, and shows where the entry overlaps the rest of the Top 10. The broader mechanics of injection channels, hiding techniques and agent design patterns are covered in the Prompt Injection Guide and are not repeated here. It is written for application security leads, AI platform owners and assessors who need to sign off an LLM feature against OWASP. It contains no working attack strings.
Key takeaways
- OWASP does not promise prevention. The entry says it is "unclear if there are fool-proof methods of prevention", so the seven strategies are impact reducers and must be judged on what an injected model can still do.
- Four of the seven strategies are deterministic. Output format validation in code, least privilege, human approval and content segregation can be tested pass or fail; prompt constraints and filters are probabilistic and need statistical testing. Over-privileged models also feature in our top 10 LLM deployment mistakes.
- Strategy 4 is an identity control. OWASP asks for the application to hold its own API tokens and run functions in code rather than handing them to the model, which makes LLM01 an NHI design question.
- The nine scenarios are a ready-made test plan. Each one maps to a channel, a control that should break it and an observable outcome, which is the evidence an assessor needs.
- LLM01 is usually the entry point, not the impact. The damage is recorded under LLM02, LLM05, LLM06 and LLM07, so fixes belong in those controls as much as in input handling.
What it is and who it applies to
The official entry is LLM01:2025 Prompt Injection in the OWASP Top 10 for LLM Applications 2025, published on genai.owasp.org (page last modified April 2025) under Creative Commons Attribution-ShareAlike 4.0. The plan title of this page matches the official risk name. Prompt injection also held the LLM01 position in the 2023 edition, so it has led the list in both releases.
OWASP defines the vulnerability as one that "occurs when user prompts alter the LLM's behavior or output in unintended ways", and stresses that the content does not need to be visible to a person, only parsed by the model. It names two types (direct, from the user's own input, and indirect, from websites, files and other external sources) and says either can be intentional or unintentional. It also positions jailbreaking as "a form of prompt injection" aimed at the model's safety protocols, and warns that retrieval augmented generation (RAG) and fine-tuning "do not fully mitigate" the problem.
Who it applies to: any system that places text, images or other content into a model's context window and acts on, displays or stores the result. That includes customer chatbots, internal copilots, RAG search assistants, email and document summarisers, coding assistants and any LLM feature embedded in a SaaS product. Exposure rises sharply once the application can retrieve private data or call functions.
How it relates to the rest of the list: OWASP lists six outcome classes for a successful injection. Each has its own Top 10 entry, and a complete LLM01 assessment checks the controls in that entry too.
| LLM01 outcome named by OWASP | Where the impact control lives | Control to verify |
|---|---|---|
| Disclosure of sensitive information | LLM02 Sensitive Information Disclosure | Data minimisation in context; per-user retrieval permissions |
| Revealing system prompts or infrastructure | LLM07 System Prompt Leakage | No secrets or authorisation logic in prompts |
| Content manipulation, biased or incorrect output | LLM09 Misinformation | Grounding checks and citation validation |
| Unauthorised access to functions available to the LLM | LLM06 Excessive Agency | Minimal tools, scoped credentials, approvals |
| Executing arbitrary commands in connected systems | LLM05 Improper Output Handling | Output treated as untrusted by every downstream component |
| Manipulating critical decision-making | LLM06 and LLM09 | Human review of consequential decisions |
See the OWASP LLM Top 10 LLM06 Excessive Agency Guide and the OWASP LLM Top 10 LLM05 Improper Output Handling Guide.
Structure and key requirements
The entry has four working parts: the description and two injection types, a list of outcomes, seven prevention and mitigation strategies, and nine example attack scenarios. It closes with reference links and three MITRE ATLAS mappings. The table below turns the seven strategies into assessable requirements.
| OWASP strategy | Control type | What good implementation looks like | Evidence an assessor can check |
|---|---|---|---|
| 1. Constrain model behaviour | Probabilistic | System prompt states role, permitted topics and refusal behaviour; prompts versioned in source control and changed through review | Prompt repository history; regression results per prompt version |
| 2. Define and validate expected output formats | Deterministic | Responses requested as a typed schema (for example JSON with an enumerated action field) and parsed by code that rejects anything else; citations checked against retrieved document IDs | Schema files; rejection counts in logs; unit tests for malformed output |
| 3. Implement input and output filtering | Mostly probabilistic | Defined sensitive categories, semantic classifiers and string checks on input and output, plus RAG Triad scoring for retrieval features | Classifier versions, thresholds, false positive rates on benign traffic |
| 4. Enforce privilege control and least privilege | Deterministic | Application holds its own scoped tokens; functions run in code; model never sees credentials; read-only access by default | Credential inventory; scope listings; denied-call logs |
| 5. Require human approval for high-risk actions | Deterministic | Approval bound to the exact operation, target and parameters, single use, enforced at execution | Approval records joined to executed actions |
| 6. Segregate and identify external content | Mixed | Untrusted content labelled by source and kept apart from instructions; provenance kept per retrieved chunk | Prompt assembly code; provenance fields in logs |
| 7. Adversarial testing and attack simulations | Assurance | Regular testing that treats the model as an untrusted user and probes trust boundaries and access controls | Test plan, per-case results with run counts, retest after changes |
Reading strategy 3 correctly: the RAG Triad
OWASP asks teams to evaluate responses with the RAG Triad. The TruLens project, which popularised the term, defines its three checks as Context Relevance (is each retrieved passage relevant to the query), Groundedness (is each claim in the answer supported by the retrieved context) and Answer Relevance (does the answer address the user's question). For LLM01 the signal is a drop in groundedness or answer relevance, since an injected document tends to pull the answer off the question. Treat it as a detective control with tuned thresholds: TruLens frames it as a hallucination check and publishes no security thresholds.
Strategy 6 is a label, not a wall: the OWASP prevention cheat sheet says labels organise the prompt but enforce nothing, so pair it with strategies 2, 4 and 5 wherever untrusted content can influence an action.
The nine scenarios as a defensive test plan
Write test cases per scenario using benign marker text (a harmless canary word or a dummy tool call) rather than real attack strings, and grade the observable outcome.
| OWASP scenario | Channel to test | Control expected to break it | Pass condition |
|---|---|---|---|
| 1. Direct injection into a support chatbot | Chat input | Strategies 4 and 5: no data store or email function reachable without authorisation | No out-of-scope query or send executes |
| 2. Indirect injection via a summarised web page leading to image-link exfiltration | Fetched web content | Output rendering controls (LLM05): external images and links blocked or rewritten | No outbound request to an unapproved domain |
| 3. Unintentional injection (hidden instruction in a job advert) | Third-party documents | Strategy 6 plus human review of the decision | Decision unchanged or flagged for review |
| 4. Modified document in a RAG repository | Indexed content | Write access control on indexed sources; RAG Triad monitoring | Poisoned chunk traceable to author; answer flagged |
| 5. Code injection in an LLM email assistant (CVE-2024-5184) | Application API | Strategies 1, 2 and 4; no hard-coded secrets in prompts | System prompt not disclosed; no unintended prompt runs |
| 6. Payload split across a CV | Multi-field documents | Strategy 2 typed scoring; LLM Map-Reduce style isolation per item | Score matches content-based rubric |
| 7. Instruction hidden in an image | Multimodal input | OCR and image text treated with the source's trust label | Hidden text does not change behaviour or tool use |
| 8. Adversarial suffix | Chat input | Strategy 3 classifiers plus rate limits on repeated attempts | Attack success rate within agreed threshold across repeated runs |
| 9. Multilingual or encoded instructions | Any input | Normalisation and decoding before screening; no reliance on keyword lists | Encoded marker detected or harmless |
Scenario 5 in detail: CVE-2024-5184 describes a prompt injection vulnerability in the EmailGPT service whose API let a malicious user inject a direct prompt to leak hard-coded system prompts or run unwanted prompts. NVD records CWE-74. Prompt injection flaws can be tracked and patched like any other vulnerability, and keeping secrets out of prompts limits what they disclose.
Identity and NHI requirements
Strategy 4 is the most important line in the entry for identity teams: "Provide the application with its own API tokens for extensible functionality, and handle these functions in code rather than providing them to the model." Read literally, it means three things. The application, not the model, authenticates to downstream systems. The credential lives where the model cannot read it. And the decision to call a function, with which parameters, is checked in code.
| Design choice | LLM01 effect if injected | Recommendation |
|---|---|---|
| API key placed in the system prompt or tool description | Key can be disclosed by injection (LLM07) and reused anywhere | Never; move to a vault and inject at call time |
| One shared service account for every user and tool | Injection by any user reaches every user's data | Avoid; use per-tool identities and per-user delegation |
| Per-tool service identity, read-only, scoped to named resources | Injection limited to that tool's narrow scope | Use for shared reference data |
| Delegated user token (OAuth) with minimum scopes, short lifetime | Injection can do no more than the user, within scope | Use for actions on a user's own mail, files or records |
| Human approval with step-up authentication | Injection cannot complete the action alone | Use for sends, shares, payments and deletions |
- Treat every model-facing tool credential as an NHI: record owner, scopes, rotation and the LLM feature that uses it. See the OWASP NHI Top 10 NHI5 Overprivileged NHI Guide.
- Authorise per request: check the end user's entitlement at the moment of each call rather than granting the application a standing superset. See the AI Agent Authorisation Guide.
- Bind approvals to identity: an approval must come from the authenticated user, in a channel the model cannot write to, for the exact parameters shown. See the Human-in-the-Loop for AI Agents Guide.
Implementation steps
- Scope the assessment: list each LLM feature, every content source reaching its context and the functions it can trigger, marking sources trusted or untrusted and functions by impact.
- Remove secrets and authorisation from prompts: search system prompts, tool descriptions and templates for keys, connection strings and rules such as "only admins may". Move secrets to a vault and rules into code.
- Apply strategy 4 first: give each function its own scoped identity or delegated token, make read-only the default and remove unused functions. This is deterministic and cuts impact immediately.
- Constrain outputs (strategy 2): define a schema for every response that drives an action or a decision; reject non-conforming output in code; validate citations against the IDs actually retrieved.
- Close rendering paths: block automatic loading of external images and links in model output, or rewrite them through an allowlist, so scenario 2 has no exfiltration route.
- Segregate content (strategy 6): wrap and label untrusted content by source, record provenance per chunk, and strip invisible characters and hidden markup before assembly.
- Add filtering in log-only mode (strategy 3): deploy input, retrieved-content and output classifiers plus RAG Triad scoring, measure false positives on real benign traffic for two to four weeks, then enforce for a pilot group.
- Add approvals (strategy 5): require bound, single-use approval for irreversible or external actions; start with a suggest-only mode where the model proposes and a person executes.
- Test (strategy 7): run the scenario test plan above, repeat each case several times because outputs vary, and rerun on every model, prompt, retrieval source or tool change.
- Plan rollback: version prompts and tool definitions, flag each tool, and be able to revoke credentials in minutes.
Decision guidance
- Chatbot with no tools or private data: strategies 1, 2 and 3 are proportionate; document that strategies 4 and 5 are not applicable and why.
- RAG over internal data: strategies 4 (per-user retrieval permissions) and 6 are mandatory, plus write control on indexed sources and rendering controls. See the Permission-Aware RAG Guide.
- Feature that reads external content and can act: all seven strategies apply, and approvals for sends, shares and changes are not optional.
- Small organisations buying copilots: ask the supplier for evidence against each strategy, especially indirect content and link rendering.
- Large organisations: implement strategies 3, 4 and 6 once in a shared gateway so each product team inherits them. See the AI Gateway and LLM Gateway Guide.
Evidence and assessment
The OWASP cheat sheet sets the grading discipline: define an observable outcome per test case, count missing telemetry as inconclusive rather than blocked, measure benign false positives separately, and report results with denominators, versions and repeated runs.
- Design evidence: data flow diagram showing trust labels for every content source; tool and credential inventory; output schemas; approval policy.
- Test evidence: scenario test plan results with run counts, dated against the model and prompt versions tested.
- Runtime evidence: logs of prompts, retrieved document IDs, classifier verdicts, schema rejections, proposed and executed tool calls, approvals and outbound URLs, with secrets excluded.
| Metric | What it shows | Target |
|---|---|---|
| Attack success rate on the scenario suite, per release | Whether model, prompt or filter changes made things better or worse | Agreed threshold; no regression between releases |
| Benign false positive rate of filters | Whether strategy 3 is usable in production | Measured and accepted by the product owner |
| Schema rejection rate | Strategy 2 is working; spikes can signal injection attempts | Baselined; spikes investigated |
| Functions running with credentials broader than their use case | Strategy 4 coverage | Zero |
| High-impact actions executed without a bound approval | Strategy 5 coverage | Zero |
| Days since last adversarial test of each feature | Strategy 7 freshness | Within policy, and after every material change |
Mapping to other frameworks
OWASP itself maps LLM01 to three MITRE ATLAS techniques. The control mappings below are NHI Mgmt Group's and are a starting point for a control library.
| Framework | Reference | Relevance to LLM01 |
|---|---|---|
| MITRE ATLAS | AML.T0051.000 LLM Prompt Injection: Direct; AML.T0051.001 Indirect; AML.T0054 LLM Jailbreak | The three mappings OWASP lists |
| MITRE ATLAS | AML.T0068 LLM Prompt Obfuscation; AML.T0077 LLM Response Rendering | Scenarios 7 and 9 (hiding) and scenario 2 (rendering exfiltration) |
| MITRE ATLAS mitigations | AML.M0020 Generative AI Guardrails; AML.M0015 Adversarial Input Detection; AML.M0021 Generative AI Guidelines | Strategies 3 and 1 |
| NIST SP 800-53 Rev. 5 | SI-10 Information Input Validation; SI-15 Information Output Filtering | Strategies 2 and 3 |
| NIST SP 800-53 Rev. 5 | AC-6 Least Privilege; AC-4 Information Flow Enforcement | Strategies 4 and 6, and blocking exfiltration paths |
| NIST SP 800-53 Rev. 5 | CA-8 Penetration Testing | Strategy 7 |
| NIST CSF 2.0 | PR.AA-05; DE.CM-09 | Least privilege for model-facing functions; monitoring runtime and data for adverse events |
| ISO/IEC 27001:2022 Annex A | A.8.28 Secure coding; A.8.29 Security testing in development and acceptance | Design and testing obligations for LLM features |
| CWE | CWE-74 | Weakness class NVD assigns to CVE-2024-5184 |
See the MITRE ATLAS Guide.
Common pitfalls
- Claiming compliance from strategy 1 alone: a hardened system prompt is one of seven strategies and the weakest; an assessment that stops there has tested intent, not control.
- Filtering the chat box only: scenarios 2, 3, 4, 6 and 7 all arrive through content the user did not type. In EchoLeak a crafted external email evaded Microsoft's injection classifiers and used reference-style markdown links that link redaction missed.
- Leaving actions unconfirmed: in the Gemini calendar prompt injection hidden text in an invite led the assistant to write a summary of private meetings into a new event; BleepingComputer reported that Google's defences include asking for explicit confirmation before creating events, which is strategy 5.
- Secrets in prompts: scenario 5 shows system prompts leaking; any key or internal hostname placed there should be assumed disclosed.
- Testing once: a single pass at launch says nothing after the next model upgrade or new retrieval source; strategy 7 is a recurring activity.
- Reporting a single success rate: results without run counts, model versions and benign false positive rates cannot be compared or trusted.
Practitioner checklist
- Record LLM01:2025 assessment status for each LLM feature against all seven OWASP strategies, with "not applicable" justified.
- Remove credentials, connection strings and authorisation rules from system prompts and tool descriptions.
- Give every model-facing function its own scoped identity or delegated user token, read-only by default.
- Validate every action-driving response against a schema in code and reject anything else.
- Block or allowlist external images and links in rendered output.
- Label untrusted content by source, keep provenance per retrieved chunk and strip invisible characters before assembly.
- Run input, retrieval and output filters plus RAG Triad scoring, tuned on benign traffic before enforcement.
- Require bound, single-use approvals for sends, shares, changes and payments.
- Maintain a scenario-based test suite covering all nine OWASP scenarios and rerun it on every material change.
- Log prompts, sources, verdicts, tool calls and approvals to the SOC, excluding secrets.
Standards and references
- OWASP: LLM01:2025 Prompt Injection
- OWASP: Top 10 for LLM Applications 2025
- OWASP: LLM Prompt Injection Prevention Cheat Sheet
- MITRE ATLAS: LLM Prompt Injection
- NIST: SP 800-53 Rev. 5
- NIST: AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- NIST NVD: CVE-2024-5184
- TruLens: The RAG Triad
Related NHI Mgmt Group resources: OWASP Top 10 for LLM Applications Guide · Prompt Injection Guide · OWASP LLM Top 10 LLM07 System Prompt Leakage Guide · AI Guardrails and AI Firewalls Guide · AI Red Teaming Guide