Google Gemini XSS exposes the limits of AI code execution controls

At a glance

What this is: Appknox describes a Gemini XSS vulnerability that let malicious HTML and JavaScript execute automatically in an AI preview flow, showing that generated output can become an execution path.

Why it matters: This matters because AI coding tools are collapsing the line between creation and execution, so application security teams must govern runtime behaviour as well as code quality.

Context

AI code generators are no longer just producing text or snippets for later review. When a platform renders output in a live preview or canvas, the security problem shifts from static code risk to execution control, because untrusted content can behave like trusted application logic.

That change creates a direct intersection with application security and, in some environments, AI governance. The core issue is not simply whether the generated code contains a flaw, but whether the system allows that output to execute before meaningful validation, sandboxing, or operator review has occurred.

Key questions

Q: What breaks when AI-generated code is allowed to render automatically?

A: The main failure is that generation and execution become indistinguishable. Once an AI system can render code immediately, validation moves too late in the flow, and malicious output can execute before a human ever reviews it. That turns a convenience feature into a security boundary that must be actively governed.

Q: Why do live previews create more risk than static code review in AI tools?

A: Live previews create risk because they execute content in context, while static review only inspects code before runtime. If the preview engine interprets HTML, JavaScript, or other active content automatically, attackers can exploit parsing quirks and sandbox gaps that never appear in a file-based review process.

Q: What signals show that AI runtime controls are failing?

A: Warning signs include unexplained tool usage, access to data outside the expected workflow, repeated policy overrides, and behavioural drift across sessions. If teams cannot trace why an action happened, who authorised it, and what context the system used, runtime controls are too weak to trust.

Q: How should teams govern AI systems that can take actions as well as generate outputs?

A: Treat the agent as a governed actor, not just a model output stream. Require action-level logging, tool-call traceability, authorization boundaries, and approval gates before the system can write to records or invoke downstream tools. If an AI system can change state, its authority must be scoped, monitored, and revocable like any other privileged non-human identity.

Technical breakdown

Why automatic rendering turns generated output into an execution path

In a normal development workflow, generated code should remain inert until a developer chooses to run it. In live AI canvases and preview panes, the platform itself may render HTML, interpret JavaScript, and invoke browser behaviour immediately. That collapses the control boundary between input and execution. Once that happens, malicious content does not need a separate delivery step, because the preview surface becomes the execution surface. The risk is therefore not just injected code, but system behaviour that treats untrusted output as if it were already approved.

Practical implication: Security teams need to treat preview and canvas components as execution environments, not passive display layers.

How sanitisation gaps and parsing quirks enable AI code execution

The article points to multiple failure modes working together: non-standard parsing contexts such as SVG, legacy execution attributes, entity encoding, and incomplete sanitisation. Each layer may appear to filter unsafe content, but browsers and preview engines do not always interpret input in the same way. That mismatch matters because validation that checks one representation can miss the executed representation. This is a classic control-alignment problem, where different layers enforce different rules and attackers look for the gaps between them.

Practical implication: Review sanitisation against the browser’s final parse state, not just the string that entered the system.

Why sandboxing is only effective when execution boundaries are strict

Sandboxing is meant to contain whatever the rendering layer allows through, but it only works when the boundary is tightly configured. If a preview iframe can still open pop-ups, navigate away, or interact with broader browser features, the sandbox stops being a reliable containment control. In AI-assisted code environments, that matters because even a small sandbox weakness can convert an injected payload into session theft, redirects, or other downstream actions. The control failure is not sandboxing in principle, but sandboxing that assumes the rendered content is already trustworthy.

Practical implication: Test preview sandboxes as hostile execution contexts and remove any capability that is not essential.

Threat narrative

Attacker objective: The attacker aims to make the AI system execute malicious code automatically inside a trusted workflow so the preview environment can be used for exploitation.

1. Entry occurs when malicious HTML or JavaScript is introduced into the Gemini preview flow through generated or pasted content.
2. Credential or control abuse follows when the platform automatically renders the payload and the browser executes it without meaningful user intent.
3. Impact occurs when the payload can trigger redirects, malicious execution, or other actions inside a trusted preview environment.

Breaches seen in the wild

• Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Zero-click execution is now an application security problem, not a niche AI bug. When AI tools render and execute code automatically, the security model moves from static review to runtime control. That changes what AppSec has to protect, because the dangerous event is not code being written, but code being executed before review. Practitioners should reframe AI code generation as an execution governance problem.

Parsing mismatches are the real weakness behind many AI preview exploits. The Gemini case shows how SVG parsing, entity encoding, and legacy execution paths can bypass controls that look adequate in isolation. This is a familiar security pattern: the browser, the sanitizer, and the preview engine each evaluate risk differently, which creates a gap an attacker can exploit. Practitioners should assume attackers will target the boundary between validation logic and browser interpretation.

Runtime trust is the named concept this category now exposes. AI workflows often assign trust to output because it was produced by the system itself, but system-generated does not mean safe. That assumption is especially dangerous in code canvases, live previews, and low-code builders where execution is immediate. Practitioners should govern trust at render time, not after deployment.

Preview sandboxes must be treated as hostile execution zones. A sandbox that allows pop-ups, redirects, or other escape paths is not containment in any meaningful sense. The article shows that weak sandboxing is enough to turn a rendering feature into an attack path. Security teams should measure preview isolation by what it blocks under adversarial input, not by how smooth the developer experience feels.

AI-generated code now belongs inside the broader application security and AI governance stack. The article is not really about one XSS bug. It is about a class of systems that blur output, input, and execution, and that makes ownership unclear across AppSec, platform engineering, and AI governance. Practitioners should assign explicit control ownership for AI execution surfaces before those surfaces become normalised in development workflows.

What this signals

Runtime trust is becoming the new control boundary. As AI systems move from content generation into content execution, the security question shifts to whether generated output can be contained at the moment it is rendered. That means developers and security teams need controls that operate inside the preview or canvas, not just before release.

AI-assisted development also widens the ownership problem. AppSec, platform engineering, and product teams all touch the execution surface, but none of them can safely assume another team has already enforced the boundary. Practitioners should map who owns rendering, who owns sanitisation, and who can disable auto-execution before misuse becomes normal.

AI-generated output must be treated as untrusted until the browser has finished interpreting it. That single discipline closes off a large class of preview-based exploits, because it stops teams from trusting the source representation instead of the executed representation. For many programmes, that will require new test cases, stricter sandboxes, and explicit runtime approval for active content.

For practitioners

• Harden preview and canvas execution surfaces Treat any live preview, code canvas, or auto-rendering pane as an execution environment. Disable unnecessary browser capabilities, constrain navigation, and remove any permission that is not required for the feature to function.
• Validate against the rendered form, not the source string Test sanitisation after parsing, encoding, and browser interpretation, because a payload that looks safe in source form may execute differently once the browser resolves tags, attributes, and entities.
• Separate generation from execution Require an explicit approval step before AI-generated HTML, JavaScript, or other active content can render in a trusted context. Do not allow automatic execution simply because the content came from an internal assistant.
• Add hostile-input test cases to AppSec pipelines Include SVG-based payloads, encoded attribute values, and preview-escape scenarios in regression tests so teams can verify that sanitisation and sandboxing still hold under realistic abuse patterns.
• Define ownership for AI execution controls Assign clear responsibility for the controls that govern AI-generated output at runtime, including platform engineering, AppSec, and product security. The control boundary should be owned before the feature becomes broadly used.

Key takeaways

• AI code generators can create execution risk when live previews render output before review, which means the control point has moved from static code inspection to runtime governance.
• The Gemini XSS case shows that parsing quirks, sanitisation gaps, and permissive sandboxing can combine into zero-click execution in an AI workflow.
• Teams that govern AI-generated output as hostile until it is explicitly approved and safely contained are better positioned to limit preview-driven exploitation.

Key terms

• Zero-Click Execution: A condition where malicious code runs without the victim taking a deliberate action such as clicking or confirming. In AI environments, this often happens when generated output is auto-rendered in a live preview and the security boundary is assumed to be a review step when it is really an execution step.
• Runtime Trust: Runtime trust is the idea that access should remain valid only while current context justifies it. Instead of trusting a setup decision indefinitely, teams continuously re-evaluate whether a workload or agent still deserves privilege. This approach is especially important for AI agents that can change behaviour mid-task.
• Template Sandbox: A template sandbox is a control intended to limit what template expressions can access or execute. It tries to block dangerous objects, file system access, and arbitrary code execution. In security testing, a sandbox should be treated as a risk-reduction measure, not a guarantee, because escapes and alternate paths may still exist.
• Sanitisation Gap: A mismatch between what a security filter thinks it has removed and what the browser actually executes. These gaps often appear when content is parsed differently across layers, allowing encoded or non-standard payloads to survive validation.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity, and machine identity security. It helps security practitioners connect runtime trust decisions to broader identity governance in modern environments.