Model-generated Markdown can become stored XSS in chat features

At a glance

What this is: This is an analysis of how model-generated Markdown became stored XSS when chat renderers enabled raw HTML without sanitization.

Why it matters: It matters because IAM and application security teams need to treat AI output surfaces as shared attack surfaces, especially where privileged users, support consoles, or transcript replay can magnify impact.

👉 Read Escape's analysis of stored XSS in AI chat Markdown rendering

Context

Model output becomes a security problem when it is rendered back into a web application without strict controls. In this case, the weakness was not the model itself but the downstream browser rendering path, where Markdown, raw HTML, and iframe handling combined into a stored XSS condition. For teams building AI chat, support, or agent workflows, the core governance issue is that generated text can carry executable content if the frontend treats it as trusted markup.

That creates a boundary problem for identity and access control as well as application security. If attacker-controlled model output can execute in a support console, admin transcript view, or ticketing workflow, the browser session and any same-origin data become part of the blast radius. This is a typical modern AI application failure pattern, not an edge case.

Key questions

Q: What breaks when AI-generated Markdown is rendered without sanitization?

A: The browser treats attacker-controlled markup as trusted HTML, which can turn a chat transcript into stored XSS. That means scripts, iframes, and attribute-based payloads may execute in another user’s session, especially when the content is replayed in support tools or admin views. The failure is in the render path, not the model output itself.

Q: Why do model output surfaces create more risk than ordinary user comments?

A: Model output often re-enters the application through first-party APIs, making it look internal and safe even when the content is untrusted. It can also be reused across multiple surfaces, which broadens exposure and makes privilege boundaries harder to see. The result is a provenance gap that security teams frequently underestimate.

Q: How can security teams know whether Markdown rendering is actually safe?

A: They need to test the full render chain, not just the source library configuration. That means checking whether raw HTML is enabled, whether sanitization occurs after parsing, whether iframe and URL attributes are controlled, and whether the same output is reused elsewhere. Safe rendering must be verified with live payloads on the real origin.

Q: Who is accountable when an AI browser exposes sensitive data or makes a bad decision?

A: The organisation remains accountable for the access path it allowed. Security, IAM, and data-governance teams should jointly define approval boundaries, logging requirements, and content restrictions before deployment. If the browser can act across regulated systems, then its governance must be explicit before use, not after failure.

Technical breakdown

Why raw HTML in Markdown becomes executable

Markdown renderers often preserve raw HTML unless a sanitization step strips it later. When a frontend enables raw HTML and then converts it into real DOM elements, the browser no longer sees text. It sees markup that can include active content such as iframes, event handlers, and documents embedded through attributes like srcdoc. The risk is not Markdown itself but the decision to trust rendered model output as safe HTML. In AI chat, that trust is especially dangerous because the content may be generated dynamically and replayed across multiple surfaces.

Practical implication: remove raw HTML rendering unless there is a documented, enforced sanitization step after parsing.

Why srcdoc and similar attributes are dangerous

Some HTML attributes carry more than a URL. srcdoc on an iframe contains an entire document, which means a filter that only checks link destinations can miss executable markup. If the iframe is not sandboxed, that embedded document can inherit the parent origin and access sensitive browser-resident data. This is why allowlisting tags alone is insufficient. Security teams need attribute-aware controls that understand whether a field carries text, a URL, or a nested document.

Practical implication: inspect and restrict high-risk attributes, not just tags, and sandbox any iframe that must remain.

Why AI chat expands the attack surface beyond the chat widget

AI chat is rarely a single page. The same model output can flow into transcript views, support tooling, ticket comments, email templates, PDF exports, or dashboard widgets. Each downstream renderer may use a different library or plugin chain, which multiplies the number of places where unsafe HTML can reappear. That makes the problem architectural, not local. A secure chat widget does not make the rest of the application safe if output is reused elsewhere without the same controls.

Practical implication: inventory every render path that can consume model output and apply the same security baseline to all of them.

Threat narrative

Attacker objective: The attacker aims to execute script in a victim’s browser through a rendered chat transcript and pivot into sensitive application context.

1. Entry occurs when attacker-controlled content reaches a model prompt or a shared chat transcript that the application later re-renders.
2. Escalation happens when the frontend enables raw HTML in Markdown and passes the rendered content through without sanitization.
3. Impact occurs when the payload executes in another user’s browser, exposing same-origin data and privileged session context.

Breaches seen in the wild

• Moltbook AI agent keys breach — Moltbook breach exposed 1.5M AI agent keys.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Model output must be treated as untrusted browser input: The security mistake here is assuming that generated text is less dangerous than user input. Once a model can emit Markdown that later becomes HTML, it belongs in the same trust category as any other untrusted content. Application security teams should align this with NIST-CSF protective controls and, where render pipelines span tools and workflows, treat it as a governance issue rather than a UI bug.

Rendered AI output creates a provenance gap: Human-authored content has attribution, intent, and an accountability trail that most teams can reason about. Model-generated content often arrives over first-party APIs and is therefore easy to mistake for trusted internal output. That provenance gap is the named failure mode here: the application trusts the source channel instead of the actual content. Practitioners should map rendered AI output to the same review and containment logic they use for other untrusted inputs.

Markdown sanitization debt is now an AI governance problem: The combination of raw HTML, iframe handling, and multiple downstream renderers creates security debt that compounds across features. This is where application security and identity governance meet, because support consoles, admin views, and transcript replay often elevate the audience that will see the payload. Teams should assume that any workflow exposing model output to privileged viewers increases blast radius unless the render chain is tightly controlled.

Cross-surface reuse is the real risk multiplier: A single unsafe renderer in a customer chat can be copied into tickets, dashboards, exports, and agent workflows, multiplying exposure with every reuse. That makes feature sprawl a control problem, not just a development pattern. Practitioners need a central policy for how model output is parsed, sanitized, and displayed across the application estate.

What this signals

Model output is becoming a new class of trusted-looking untrusted input: As AI features spread into chats, agents, and workflow tools, security teams need a control model that assumes generated text can be malicious. The best analogue is not a chatbot problem but an application trust problem, which means render policy, sandboxing, and output governance must be reviewed together.

Render-chain security should be part of AI application governance: When a model can write content that later reaches privileged users, the security question is no longer only whether the model is accurate. It is whether the application can safely display what the model says without creating a browser exploit. That shifts review toward output containment, not just prompt safety.

For practitioners

• Remove raw HTML rendering from model output Disable rehype-raw or equivalent raw HTML passthrough for all AI-generated Markdown unless there is a documented business case and a post-parse sanitizer. This is the simplest way to eliminate executable markup from chat transcripts and similar surfaces.
• Sanitize after parsing, not before Place sanitization after Markdown and HTML parsing so the filter sees the final DOM tree, not raw tokens that can be reinterpreted later. Validate the order in code review and with payload-based testing on the real application origin.
• Audit every downstream render path Inventory where model output is reused, including support consoles, ticket comments, email templates, PDF generation, and analytics dashboards. Apply the same markup policy everywhere, because a secure chat widget does not protect a weak export or admin view.
• Restrict high-risk HTML attributes Treat iframe srcdoc, form action, image src, and any on* event handler as high-risk fields that require explicit allowlisting or removal. Tag-only filtering is not enough when the dangerous payload lives inside attributes rather than elements.

Key takeaways

• AI chat can become a stored XSS vector when generated Markdown is rendered as HTML without sanitization.
• The real failure is architectural reuse of untrusted model output across multiple render paths, not a single library choice.
• Teams should remove raw HTML, verify sanitization order, and test every downstream surface that displays model output.

Key terms

• Stored Xss: Stored XSS happens when an application saves malicious script in backend content such as comments, profiles, or posts, then serves it to other users later. Because the payload persists, one successful injection can trigger repeated execution across many authenticated sessions.
• Rehype-raw: Rehype-raw is a Markdown processing step that reparses raw HTML into real elements instead of leaving it as escaped text. It is useful for formatting, but it also reintroduces executable markup risk unless a sanitizer runs after parsing and attributes are tightly controlled.
• Srcdoc: Srcdoc is an iframe attribute that contains an entire HTML document rather than a URL. Security controls that only inspect link destinations often miss it, which makes it a common bypass target when renderers allow embedded HTML without sandboxing or attribute allowlists.
• Provenance Trust Gap: A provenance trust gap exists when a system trusts the source of stored data but fails to distinguish whether the content itself is safe to execute. In AI workflows, this gap appears when retrieved records or documents can influence agent actions without validation of origin, intent, or trust level.

What's in the full article

Escape's full analysis covers the implementation detail this post intentionally leaves for the source:

• The exact bundle inspection path used to trace the Markdown renderer and identify the unsafe plugin chain.
• The step-by-step exploit reasoning that turns model-emitted Markdown into browser-executable content.
• The renderer hardening guidance for React, Markdown, and iframe handling in the customer environment.
• The practical payload verification approach Escape used to confirm the issue on the real origin.

👉 The full Escape article covers the exploit walk-through, bundle inspection, and hardening steps in detail.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for teams that need to connect access governance to AI and identity risk.