Join our Newsletter — 33% off our NHI Course

Model-generated Markdown and stored XSS: are your renderers safe?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 16626
Topic starter  

TL;DR: Two customer-facing AI chat implementations exposed the same stored XSS by rendering model-produced Markdown with raw HTML enabled and no sanitizer, allowing attacker-controlled markup to execute in another user’s browser, according to Escape. The lesson is that every model output surface needs the same render-chain scrutiny as public input, because unsafe HTML handling compounds fast.

Editorial analysis by NHI Mgmt Group, based on content published by Escape: “Escape found the same XSS in two AI chatboxes. The vulnerability was in the Markdown renderer.”.

Key questions

Q: What breaks when model-generated Markdown is rendered with raw HTML enabled?

A: The renderer stops treating the model’s output as inert text and turns it into browser-executable content.

Q: Why does stored XSS in AI chat matter more when privileged users read transcripts?

A: Because the payload executes in the browser of whoever opens the conversation, not only in the attacker’s browser.

Q: What do security teams get wrong about sanitising model output?

A: They often sanitise too early or trust a single library to cover every downstream render step.

Practitioner guidance

  • Remove raw HTML from model-rendered Markdown Disable HTML passthrough in chat and assistant output unless there is a documented business requirement that survives security review.
  • Sanitize after every transformation stage Apply HTML sanitization after any Markdown, rehype, syntax-highlighting, or embed plugin that can change the tree before it reaches the browser.
  • Inventory every model-output render path Map where generated text appears in chat, tickets, dashboards, emails, exports, and support tooling so you can test each renderer separately.

Bottom line: Model-generated Markdown can become stored XSS when raw HTML rendering and missing sanitization turn chat output into executable browser content.

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 17057
 

Generated text is now a first-class attack surface: When a model writes content that the browser later interprets as markup, the trust boundary shifts from user input to model output. That changes how teams should think about application security reviews, because the issue is not whether the user typed a script tag. It is whether any generated text can survive the render chain as active HTML. Practitioners should treat output rendering as part of the security model, not as presentation logic.

A question worth separating out:

Q: How should teams govern AI chat features that render Markdown?

A: Treat them as application security features with explicit render governance, not as harmless UI polish. That means reviewing which elements and attributes are allowed, disabling raw HTML where possible, sandboxing embedded documents, and testing every place the model output is re-used across the product.

👉 Read our full editorial: Model-generated Markdown can become stored XSS in chat features



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.