OWASP LLM05 Improper Output Handling Guide

Applications that pass large language model output straight into a browser, a database, a shell or another API are handing every person who can influence the prompt a route into those systems. LLM05:2025 Improper Output Handling is the OWASP Top 10 for LLM Applications entry for this failure: insufficient validation, sanitisation and handling of model output before it reaches downstream components. This guide goes deeper than the overview page. It covers the sink-by-sink controls, the identity of the components that consume output, a phased rollout, how to test and evidence the controls, and how LLM05 maps to ASVS 5.0, NIST SP 800-53 and MITRE ATLAS. It is written for application security engineers, developers building LLM features and the reviewers who sign them off.

Key takeaways

Model output is untrusted input. OWASP's first mitigation is to treat the model "as any other user" and validate its responses before they reach backend functions, because anyone who shapes the prompt, directly or through retrieved content, shapes the output.
The damage is classic web and injection bugs. OWASP lists XSS, CSRF, SSRF, privilege escalation and remote code execution as the results, so the fixes are the ones you already know: context-aware encoding, parameterised queries, allowlists and no eval.
Markdown rendering is an exfiltration channel. EchoLeak and ForcedLeak both moved data out by getting an image URL into rendered output, so Markdown and image handling deserve as much attention as script tags.
Privilege multiplies impact. The same unvalidated output is a nuisance in a read-only chat window and a breach when the consuming component runs with a powerful database or cloud identity.
Prove it with tests, not prompts. System prompt instructions such as "never output HTML" are not a control; unit tests that push hostile markup, SQL and path strings through every output sink are.

What it is and who it applies to

OWASP defines Improper Output Handling as "insufficient validation, sanitization, and handling of the outputs generated by large language models before they are passed downstream to other components and systems". Because model output can be controlled through prompt input, OWASP describes the effect as "similar to providing users indirect access to additional functionality". In the 2023/24 edition the same risk was LLM02 Insecure Output Handling.

OWASP separates LLM05 from neighbouring entries, and the distinction matters for ownership:

Entry	Question it asks	Typical owner
LLM01 Prompt Injection	Can someone change what the model is trying to do?	AI platform and application team
LLM05 Improper Output Handling	Is whatever the model produced safe for the component that consumes it?	Application developers, AppSec
LLM06 Excessive Agency	What can the system do with that output, and with whose permissions?	Architects, IAM, platform owners
LLM09 Misinformation	Is the content accurate enough to rely on?	Product owners, users

LLM05 applies to any system where generated text becomes code, markup, a query, a file path, a URL, a template or a command. That includes chat interfaces that render Markdown, text-to-SQL and analytics assistants, code assistants, email and document generators, agents that pass one model's output to a tool or to another model, and retrieval pipelines that summarise external web pages. Prompt injection is the usual trigger, but OWASP's framing does not depend on an attacker: a hallucinated query that drops a table is still an output handling failure. See the OWASP Top 10 for LLM Applications Guide for the whole list and the Prompt Injection Guide for the input side.

Structure and key requirements

OWASP gives five common examples, six aggravating conditions and seven mitigations. The practical way to apply them is per sink: for each place model output lands, decide the encoding, validation and privilege rule.

Sink	Failure OWASP describes	Primary control	ASVS 5.0 requirement
Browser (HTML, JavaScript)	Generated JavaScript or Markdown interpreted by the browser, giving XSS	Context-aware output encoding; sanitise any HTML with a maintained library; strict CSP	V1.2.1, V1.3.1, V3.4.3
Markdown renderer	Images and links that fetch attacker URLs carrying data	Disable or sanitise scriptable Markdown; allowlist image hosts; strip reference-style links	V1.3.5
Database	Generated SQL executed without parameterisation	Parameterised queries; a read-only, narrowly scoped database identity; query allowlists for text-to-SQL	V1.2.4
Shell, exec, eval	Output entered into a system shell, exec or eval, giving RCE	Never execute output directly; parameterised OS calls; sandbox where code execution is the feature	V1.2.5, V1.3.2
File system	Output used to build file paths, giving path traversal	Canonicalise and check against a base directory; map output to IDs not paths	V1.3.3
Outbound HTTP	Output used as a URL, giving SSRF or data exfiltration	Allowlist of protocols, domains and ports; egress proxy	V1.3.6
Email and templates	Output placed in email templates without escaping, enabling phishing or XSS	Escape per template context; never build templates from output; sanitise for SMTP	V1.3.7, V1.3.11
Another model or tool	Output passed to a privileged extension without validation	Schema validation of structured output; quarantine untrusted content	V1.3.3

The aggravating conditions OWASP lists are worth turning into review questions: does the application grant the LLM privileges beyond those of end users, is it exposed to indirect prompt injection, do third-party extensions validate their inputs, is encoding applied per context (HTML, JavaScript, SQL), are outputs logged and monitored, and is there rate limiting or anomaly detection on LLM use?

Real cases that show the pattern

Code execution: CVE-2023-29374 affected LangChain through 0.0.131; NVD records that the LLMMathChain chain allowed prompt injection to execute arbitrary code through Python's exec method (CVSS 3.1 9.8, CWE-74). CVE-2024-5565 in the Vanna library is similar: with visualize set to True, the default, prompt injection could make the generated visualisation code run arbitrary Python (CWE-94).
Rendered image exfiltration: in the EchoLeak Microsoft 365 Copilot case (CVE-2025-32711), reference-style Markdown images survived Copilot's link filtering and trusted Microsoft services were used to get past its Content Security Policy. In ForcedLeak, Agentforce output placed CRM data in an image URL on an expired domain that was still on the CSP allowlist.
Cross-tool forgery: in May 2023 Embrace The Red showed ChatGPT URL-encoding an email summary obtained through one plugin and loading an attacker URL with a browsing plugin. Output from one step was trusted as an instruction by the next.

Identity and NHI requirements

Output handling is usually discussed as an encoding problem, but the impact is set by the identity of whatever consumes the output. OWASP's first aggravating condition is an application that "grants the LLM privileges beyond what is intended for end users". In practice that privilege sits in a non-human identity: the database login used by a text-to-SQL feature, the cloud role of the code-execution sandbox, the API key a summariser uses to fetch URLs, or the OAuth token an email generator sends with.

One identity per output sink: give the SQL executor, the code sandbox and the fetcher separate service identities so a weakness in one sink cannot use another sink's rights. See the Service Account Security Guide.
Least privilege on the consumer: a reporting assistant's database identity needs SELECT on named views, not the schema owner. A dropped table from a generated query (OWASP scenario 3) is impossible if the login cannot DROP. See the OWASP NHI Top 10 NHI5 Overprivileged NHI Guide.
User context where data is personal: when output drives a query on behalf of a signed-in user, run it with that user's delegated token or row-level security, so the model cannot return rows the user could not see.
No secrets in the execution environment: code interpreters and sandboxes should start with no ambient credentials, no metadata service access and no mounted keys. See the AI Agent Sandboxing and Code Execution Guide.
Egress identity: outbound fetches should leave through a proxy that authenticates the calling workload and enforces a domain allowlist, which is also the control that would have stopped the image-based exfiltration above.

Implementation steps

Inventory sinks (weeks 1-2): trace every code path where model output, including tool results and retrieved content echoed by the model, is rendered, executed, stored, queried or sent. Record the sink type, the identity it runs as and the user population that can reach it.
Remove the worst sinks first: delete any direct exec, eval, shell or raw SQL execution of output. Where code execution is the feature, move it into an isolated sandbox with no network by default and no credentials.
Constrain output shape: where the application expects data rather than prose, require structured output (JSON with a schema) and reject anything that does not validate. For text-to-SQL, validate generated statements against an allowlist of statement types, tables and columns before execution, and bind values as parameters.
Encode at the point of use: HTML-encode in HTML bodies, attribute-encode in attributes, JavaScript-encode in scripts, and never insert output into templates as template source. Use a maintained HTML sanitiser where rich text is required.
Lock down Markdown and images: render Markdown with raw HTML disabled, strip or rewrite image and link URLs to an allowlist, block reference-style link definitions you do not need, and consider showing links as text until the user clicks.
Set CSP and egress controls: deploy a strict Content Security Policy for LLM-rendered pages (tight img-src and connect-src), review the allowlist for expired or third-party-controlled domains, and route server-side fetches through an egress proxy.
Quarantine untrusted content: where a model processes emails, web pages or documents, keep that output away from components that can act. Simon Willison's Dual LLM pattern passes untrusted results by reference ($VAR1) so the privileged model never reads them; its author notes it is not fully reliable, so pair it with the controls above.
Test in report-only first, then enforce: run CSP in Content-Security-Policy-Report-Only, run new output validators in log-only mode against production traffic for one to two weeks, review false positives, then switch to blocking. Keep the previous renderer behind a feature flag for rollback.
Add regression tests to CI: fixed test cases that make the model, or a stub of it, return script tags, Markdown images to external hosts, SQL with stacked statements, ../ paths and internal URLs, and assert each sink encodes or rejects them.

Choosing a control by sink

Situation	Prefer	Trade-off
Output only shown to the same user	Plain-text rendering or sanitised Markdown with no external images	Loses rich formatting; lowest risk
Output shown to other users (shared chats, tickets, emails)	Full sanitisation plus CSP; treat as stored XSS risk	More engineering; mandatory because victims did not write the prompt
Text-to-SQL for analysts	Read-only replica, allowlisted views, statement validation	Some questions cannot be answered; safer than free SQL on primary
Code generation that is executed	Ephemeral sandbox, no secrets, no default network	Latency and cost; the only defensible option for executing output
Small team, one LLM feature	Plain text, no tools, a hosted model with built-in output filters as a second layer	Less flexible; fewer sinks to secure

Evidence and assessment

OWASP's last mitigation is "robust logging and monitoring systems to detect unusual patterns in LLM outputs". Assessors should expect to see both the preventive controls and the telemetry.

Design evidence: the sink inventory, the identity each sink runs as, and the encoding or validation rule per sink.
Test evidence: CI results for the hostile-output regression suite, SAST rules that flag model output flowing into exec, eval, subprocess, raw SQL or template engines, and DAST or manual test reports against rendered output.
Runtime evidence: CSP violation reports, blocked egress requests, schema validation failures and sanitiser removals, logged with session and user identifiers.

Metric	What good looks like
Sinks with a documented and tested control	100 percent of inventoried sinks
Direct exec, eval or raw SQL calls on model output in code	Zero, enforced by SAST in the pipeline
CSP violation reports from LLM-rendered pages	Trended weekly; every new external host investigated
Structured output validation failure rate	Baselined; sudden rises reviewed as possible injection
Egress requests from LLM components to non-allowlisted hosts	Zero allowed; all blocked attempts reviewed
Output sink identities with write or admin rights not justified in review	Zero

Mapping to other frameworks

Framework	Reference	Relevance to LLM05
OWASP ASVS 5.0	V1.2.1, V1.2.4, V1.2.5, V1.3.2, V1.3.5, V1.3.6, V1.3.7, V3.4.3	Context encoding, parameterised queries and OS calls, no eval, Markdown sanitisation, SSRF allowlists, template injection, CSP
NIST SP 800-53 Rev. 5	SI-10 Information Input Validation; SI-15 Information Output Filtering	Validate output as input to the next component; filter output for anomalous content
NIST SP 800-53 Rev. 5	AC-4 Information Flow Enforcement; SC-18 Mobile Code	Egress and image URL control; control of generated script content
NIST SP 800-53 Rev. 5	SA-11 Developer Testing and Evaluation; AU-12, SI-4	Hostile-output test suites; logging and monitoring of output
NIST CSF 2.0	PR.PS-06; DE.CM-09	Secure development practice; monitoring software and runtime for adverse events
MITRE ATLAS	AML.T0051 LLM Prompt Injection; AML.T0077 LLM Response Rendering; AML.T0067 LLM Trusted Output Components Manipulation	The trigger, the rendering exfiltration path, and manipulation of output users trust
MITRE ATLAS	AML.T0086 Exfiltration via AI Agent Tool Invocation	Where output becomes a tool call that sends data out
OWASP Agentic Top 10	ASI05 Unexpected Code Execution (RCE)	The agentic form of executing generated code
ISO/IEC 27001:2022 Annex A	A.8.26 Application security requirements; A.8.28 Secure coding	Where output handling rules and encoding standards are required

See the OWASP ASVS Guide, the Cross-Site Scripting (XSS) Prevention Guide, the SQL Injection Prevention Guide and the SSRF Prevention Guide for the underlying controls, and the OWASP Agentic Top 10 ASI05 Guide for agents that execute code.

Common pitfalls

Trusting the system prompt: instructing the model not to emit HTML or SQL keywords. Injected content overrides instructions; only code at the sink enforces anything.
Filtering links but not images: stripping inline links while reference-style images, auto-loading previews or allowlisted proxies remain, which is exactly the gap EchoLeak used.
A stale CSP allowlist: listing third-party or expired domains in img-src. ForcedLeak's channel was an allowlisted domain anyone could buy for about five dollars.
Executing "helpful" code: math, chart and data-analysis features that run generated Python in the application process, as in the LangChain and Vanna CVEs.
A powerful identity behind text-to-SQL: connecting the query executor as the schema owner, so a hallucinated or injected DELETE succeeds.
Treating internal consumers as safe: passing output to an internal admin API or another model without validation because "it is not user-facing", which is OWASP's first attack scenario.

Practitioner checklist

Maintain an inventory of every sink that consumes model output, with the identity each runs as.
Remove all direct exec, eval, shell and raw SQL use of model output, and block reintroduction with SAST rules.
Require schema-validated structured output wherever the application expects data.
Apply context-aware encoding at every render point and a maintained sanitiser for rich text.
Render Markdown with raw HTML off and image and link hosts restricted to an allowlist.
Deploy a strict CSP on LLM-rendered pages and review its allowlist for third-party or expired domains quarterly.
Run generated code only in a credential-free, network-restricted sandbox.
Give each sink a least-privilege service identity, read-only where possible.
Keep a hostile-output regression suite in CI and roll out new validators in report-only mode first.
Log output validation failures, CSP violations and blocked egress with session identifiers, and alert on spikes.

OWASP LLM Top 10 LLM05 Improper Output Handling Guide: Treating Model Output as Untrusted