Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

Google Gemini XSS: are your AI code execution controls ready?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 20377
Topic starter  

TL;DR: AI-generated code can execute before humans review it, and Appknox says a Gemini XSS flaw showed how auto-rendering, weak sanitisation, and permissive sandboxing can turn trusted output into zero-click execution. The broader lesson is that application security now has to govern runtime behaviour, not just pre-release code, according to Appknox.

NHIMG editorial — based on content published by Appknox: Gemini XSS Vulnerability: When AI Executes Malicious Code

By the numbers:

Questions worth separating out

Q: How should security teams govern AI coding assistants that can execute commands?

A: Treat them as delegated non-human identities with bounded execution authority.

Q: Why do AI-generated code changes increase application security risk?

A: AI-generated code can increase risk because it accelerates output faster than review, testing, and secret hygiene can keep up.

Q: What do teams get wrong about sandboxing autonomous AI agents?

A: Teams often confuse containment with trust.

Practitioner guidance

  • Separate generation from execution Require a distinct review step before any AI-generated HTML, JavaScript, or embedded content is rendered in a live preview or browser context.
  • Harden preview sandbox settings Disable unnecessary iframe capabilities such as pop-ups, redirects, and cross-window actions, and test the sandbox against encoded and nested payloads.
  • Treat AI execution features as privileged Assign explicit owners, access rules, and audit logging to tools that can execute code automatically, just as you would for a sensitive automation identity.

What's in the full article

Appknox's full blog post covers the operational detail this post intentionally leaves for the source:

  • The full exploit chain showing how SVG parsing, entity obfuscation, and xlink:href were combined to bypass sanitisation.
  • A clearer breakdown of the preview sandbox weaknesses that allowed malicious actions to escape intended boundaries.
  • The article's practical prevention guidance for teams evaluating AI code execution and preview isolation controls.
  • The source also expands on why traditional AppSec tooling struggles when generation and execution happen in the same workflow.

👉 Read Appknox's analysis of the Gemini XSS vulnerability and AI code execution risk →

Google Gemini XSS: are your AI code execution controls ready?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 4 months ago
Posts: 19968
 

Zero-click execution is the new control failure in AI-assisted development: the risk is no longer only whether code is malicious, but whether the system is allowed to execute it before scrutiny. Traditional AppSec assumes human review sits between creation and runtime. AI preview workflows collapse that boundary, so governance must focus on execution permission, not just code quality. Practitioners should treat auto-render features as a privileged pathway and control them accordingly.

A question worth separating out:

Q: What should organisations do when GenAI is embedded in code and workflows?

A: Apply secure SDLC and third-party dependency controls to AI outputs, including review for bugs, secrets, prompt-injection artifacts, and unsafe API use. AI-assisted code is not exempt from normal engineering governance. The practical standard is the same as any external input: inspect, test, approve, and monitor before it reaches production.

👉 Read our full editorial: Google Gemini XSS exposes the limits of AI code execution controls



   
ReplyQuote
Share: