The renderer stops treating the model’s output as inert text and turns it into browser-executable content. That creates stored XSS when the output is saved, replayed, or shared with another user. The core failure is not the model itself. It is the missing trust boundary between generated text and active DOM construction.
Why Raw HTML Changes the Meaning of Model Output
Markdown is usually treated as text-to-formatting. Once raw HTML is enabled, the renderer also accepts active browser markup, which means model output can stop being inert content and start becoming executable page structure. That is the trust boundary shift that matters: the system is no longer just displaying text, it is assembling DOM from untrusted input.
That change affects any place where generated content is persisted, replayed, or shared. A payload that looks harmless in a chat response can become dangerous later when it is rendered inside a comment, document, knowledge base, or audit trail without escaping or sanitisation.
How Stored XSS Emerges from Trusted Rendering
The core failure is not that the model “knows HTML.” The failure is that the application lets generated output cross from a text channel into a browser execution channel without an explicit sanitisation step. If the renderer allows tags, event handlers, inline script-like behaviour, or dangerous URL schemes, the model can introduce markup that the browser interprets as active content.
That is why stored XSS is the common outcome in this pattern. The harmful content may be saved once, then served many times, so the blast radius is not limited to the first render. Any user who later opens the saved content can inherit the attack surface.
What Controls Actually Break, and What Still Works
When raw HTML is enabled, the broken control is the assumption that generated Markdown is safely passive. Escaping alone is not enough if the HTML pass is still permitted downstream. The application needs a hard boundary between generated prose and any markup that can alter browser behaviour.
Useful controls include output encoding, HTML sanitisation, a restrictive allowed-tags policy, and context-aware rendering rules that distinguish safe formatting from active content. If the content must support rich text, the system should treat HTML as an explicitly governed feature, not an automatic side effect of rendering.
Developers should also assume that attackers do not need a direct injection point in the template if they can influence the model output itself. Prompt injection, unsafe post-processing, and copy-paste into shared systems all become delivery paths when the renderer trusts the markup.
Risk and Threat Considerations
raw html rendering creates a durable cross-user exposure because one unsafe model response can be reused across sessions, documents, or viewers. The risk is highest when generated content is stored or syndicated, because the attack then survives beyond the original interaction and reaches users who never saw the prompt that produced it.
Failure mechanism: the application treats model output as safe text during generation but later feeds the same content into a browser context that interprets HTML, so untrusted markup becomes active page content.
Impact: attackers can achieve stored XSS, hijack user sessions, manipulate page content, or trigger actions in the victim’s browser, especially when the rendered page is viewed by privileged users or reused in multiple contexts.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP ASVS, NIST SP 800-53 Rev 5 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP ASVS | V1 — Encoding and Sanitization | Covers safe handling of untrusted output before browser rendering. |
| V16 — Security Logging and Error Handling | Supports detection and investigation of suspicious rendered payloads and failures. | |
| Recommendation — Encode and sanitise model output before it reaches any HTML renderer. Log rendering errors and sanitize failures to spot hostile content attempts. | ||
| NIST SP 800-53 Rev 5 | SI-10 — Information Input Validation | Applies because model output becomes untrusted input once it is rendered as HTML. |
| AC-3 — Access Enforcement | Relevant where rendered content can trigger actions or expose protected UI to users. | |
| Recommendation — Validate and constrain generated content before it is accepted by the renderer. Enforce least-privilege access to any page or workflow that displays generated content. | ||
| CIS Controls v8 | 16 — Application Software Security | Directly supports secure handling of untrusted generated content in web applications. |
| Recommendation — Harden rendering paths so untrusted Markdown cannot become executable HTML. | ||
Practitioner Guidance
What to verify: Confirm whether the rendering path ever allows raw HTML after generation, including preview panes, export views, and replay surfaces. The important test is not whether the original model output looked safe, but whether any later step turns that output into browser-interpreted markup.
Decision rule: If untrusted content must be rendered with formatting, allow only a tightly defined subset of safe tags and attributes, and strip everything else before storage or display. If the application cannot enforce that boundary consistently, treat the feature as unsafe by default.
Common mistake: Teams often secure the prompt pipeline but forget the display pipeline. That leaves a gap where the model is not the exploit, the renderer is. The fix is to govern the output sink with the same care used for any other untrusted web input.
Practitioner takeaway: Raw HTML support turns a text-generation problem into a web security problem, so the right control objective is not “make model output harmless,” but “prevent untrusted output from becoming active browser content.”
Related resources from NHI Mgmt Group
- What breaks when AI-generated Markdown is rendered without sanitization?
- What breaks when AI pentesting relies on raw frontier-model prompting?
- What breaks when organisations rely on raw public code datasets for model fine-tuning?
- What breaks when an agent only applies sandboxing to the first command in a model-generated shell string?