Gemini Enterprise prompt injection exposes a collapsed trust boundary

At a glance

What this is: GeminiJack is a zero-click indirect prompt injection flaw in Gemini Enterprise and Vertex AI Search that lets poisoned content execute as trusted AI instructions and exfiltrate connected data.

Why it matters: It matters because AI search and assistant integrations can turn routine collaboration data into a data-exfiltration path, forcing IAM, NHI, and AI governance teams to re-evaluate trust boundaries and blast radius.

👉 Read Noma Security's analysis of the GeminiJack indirect prompt injection flaw

Context

Gemini Enterprise prompt injection is a trust-boundary problem, not just a model behaviour problem. In this case, untrusted content indexed into enterprise search could be treated as instructions, which means the search layer becomes part of the execution surface instead of a passive retrieval layer.

For IAM and NHI teams, the key issue is delegated access. Once Workspace content, search indexing, and response generation share the same trust path, the organisation has to govern what the AI can reach, what it can infer, and what it can emit. That changes both data-access design and control ownership across identity and AI operations.

The article's scenario is typical of modern federated-search deployments, where productivity gains are created by wider reach across systems but the security model assumes a clean separation between content and command. GeminiJack shows that assumption is too weak when retrieved content can steer execution.

Key questions

Q: How should security teams reduce indirect prompt injection risk in AI systems?

A: Security teams should limit what AI systems can read, separate untrusted content from privileged actions, and apply least privilege to every connected agent. The strongest posture combines content filtering, allowlisted sources, short-lived sessions, and explicit approval for sensitive actions. If any one of those layers is missing, the attack path remains open.

Q: Why do federated AI search integrations increase enterprise risk?

A: Because they expand the number of systems one query can touch and make a single poisoned item influence multiple repositories. The risk is not only broader disclosure, but also a larger execution surface where hidden instructions can steer data collection across mail, files, and calendars.

Q: What do organisations get wrong about zero-click AI data leaks?

A: They assume the absence of a phishing click or malware means the event is low risk. In reality, if a model can act on untrusted retrieved content and reach sensitive systems through normal search behaviour, the leak can be silent, fast, and hard to detect.

Q: Who is accountable when AI search exposes sensitive enterprise data?

A: Accountability sits with the teams that approved the data connections, retrieval scope, and response handling, not just the users who queried the system. Governance should cover access design, provenance controls, and operational monitoring across identity, search, and AI platform owners.

Technical breakdown

Indirect prompt injection in enterprise search

Indirect prompt injection occurs when malicious instructions are embedded in content that a model retrieves later, such as a document, email, or calendar item. The model does not need to be directly prompted by the attacker at runtime. Instead, retrieval-augmented generation loads the poisoned content into context, and the model may treat the embedded text as instruction-bearing if the architecture does not preserve a hard boundary between data and control. In enterprise search, that is especially dangerous because indexed collaboration content inherits organisational trust by default.

Practical implication: separate untrusted retrieved content from executable instructions and review how search indexing changes the effective trust model.

Why federated search expands the blast radius

Federated search makes one query reach multiple systems, so a single compromised retrieval path can expose Gmail, Calendar, Docs, and other connected sources. The risk is not just disclosure, but command propagation: once a model can access multiple repositories under the same identity context, poisoned instructions can chain across those sources and assemble sensitive material into a single response. The architectural flaw is that retrieval convenience also concentrates privilege. That creates a blast radius much larger than the original document or event.

Practical implication: map the maximum data reachable from one AI query and segment high-value repositories before enabling broad federation.

Exfiltration through normal AI response channels

The most concerning part of GeminiJack is that exfiltration can happen through a legitimate-looking response path rather than malware or obvious outbound traffic. When hidden instructions cause the model to build an image URL or similar request, sensitive data can leave the environment as part of standard AI output handling. Traditional DLP and perimeter monitoring often miss this because the traffic resembles ordinary application behaviour. That means the control problem shifts from blocking files to governing what the model is allowed to retrieve, combine, and emit.

Practical implication: monitor AI output for anomalous response construction and restrict external content handling in generated answers.

Threat narrative

Attacker objective: The attacker wants to exfiltrate high-value enterprise data from connected collaboration systems without user interaction or obvious forensic evidence.

1. Entry occurs when an attacker shares a document, sends an email, or creates a calendar event containing hidden instructions that look legitimate to the search layer.
2. Escalation occurs when Gemini Enterprise retrieves the poisoned content, loads it into context, and treats attacker instructions as valid commands across connected data sources.
3. Impact occurs when the model compiles sensitive information from Gmail, Calendar, and Docs and sends it out through an outbound request that blends into normal AI traffic.

Breaches seen in the wild

• Gemini CLI Breach — Silent Code Execution — Vulnerability in Gemini CLI allows silent code execution via prompt injection in AI developer tool.
• Gemini AI Breach — Google Calendar Prompt Injection — Gemini AI assistant prompt injection attack leaks sensitive Google Calendar data.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Prompt injection becomes an identity problem when the model inherits trust from retrieved content. GeminiJack shows that the boundary between data and instruction is no longer reliable once collaboration content is indexed into AI search. The governance failure is not only model vulnerability, but the assumption that retrieved content can be safely treated as organisational knowledge without revalidating intent. Practitioners need to treat retrieval paths as privileged decision surfaces, not passive lookup layers.

Blast-radius mapping is now a core control for AI search deployments. The article demonstrates that one poisoned document can reach Gmail, Calendar, and Docs when federation is broad enough. Identity blast radius: the real control question is not whether a user can access a source, but how far one AI query can propagate that access across systems and content types. Teams should use that lens to narrow exposure before adding more integrations.

Zero-trust for AI agents is really zero-trust for retrieved context. The moment the model can act on content it did not author, the usual trust hierarchy collapses. That means least privilege must be applied to retrieval scope, response construction, and outbound side effects together. The practical conclusion is that AI governance cannot stop at prompt hygiene; it has to govern what content is allowed to influence execution.

Content provenance is the missing control plane for enterprise AI search. The article shows that externally supplied material can become operationally trusted once indexed. That is a governance gap, not just a detection gap, because provenance determines whether content should be allowed to shape downstream AI behaviour at all. Practitioners should treat provenance tagging, source reputation, and retrieval segmentation as first-class identity controls.

From our research:

• 92% of organisations expose NHIs to third parties, raising concerns about supply chain security, according to Ultimate Guide to NHIs.
• 79% of organisations have experienced secrets leaks, and 77% of those incidents resulted in tangible damage, according to NHI Mgmt Group research.
• For a lifecycle lens, see NHI Lifecycle Management Guide for how provisioning, rotation, and offboarding reduce exposure windows.

What this signals

Identity blast radius: AI search programmes are now only as safe as the narrowest data connection they inherit. When retrieval reaches across collaboration tools, the programme needs explicit controls for provenance, segmentation, and response-side containment, not just prompt hardening.

With 92% of organisations exposing NHIs to third parties, according to our Ultimate Guide to NHIs, the same governance pattern is now visible in AI search: broad connectivity without enough trust discrimination. The implication is that identity teams must own the reachable-data question before AI tools become default interfaces.

This is where the retrieval trust boundary becomes a practical programme concept. If indexed content can shape execution, then the control objective shifts from access approval alone to content provenance, source segmentation, and output monitoring across AI systems that hold delegated enterprise reach.

For practitioners

• Audit AI query blast radius Inventory which repositories Gemini Enterprise or similar systems can reach, then map what a single query can expose across Gmail, Calendar, Docs, and external sources. Use the result to remove unnecessary high-value connections before enabling broad federation.
• Separate trusted instructions from retrieved content Require your AI layer to preserve a hard boundary between system instructions and untrusted indexed material, including documents, email, and calendar events. Block models from promoting retrieved text into command-bearing context without validation.
• Restrict response-side exfiltration paths Disable or tightly control response features that can embed external requests, such as auto-loading remote content or image beacons, and monitor for anomalous outbound patterns in generated answers.
• Tag and segment high-risk content sources Mark externally contributed documents and shared calendar items as untrusted by default, then route them through stricter review or reduced retrieval scope when they can influence AI search results.

Key takeaways

• GeminiJack shows that AI search can collapse the line between content and command, turning indexed collaboration data into an execution path.
• The impact is broad because one poisoned item can reach multiple enterprise repositories and exfiltrate sensitive data without obvious user interaction.
• Practitioners need blast-radius mapping, provenance controls, and strict separation between retrieved content and executable instructions before expanding AI federation.

Key terms

• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Identity Blast Radius: The amount of damage a compromised identity can cause across systems, data, and infrastructure. In NHI environments, it is shaped by permissions, network reach, and administrative capability rather than by the credential alone. Reducing blast radius is a containment strategy that limits lateral movement and data exposure.
• Content Provenance: Content provenance is the practice of tracking where input came from and how trusted it should be before an AI system uses it. For agents, it helps separate instructions from retrieved or external data so malicious content is less likely to be treated as operational guidance.

What's in the full report

Noma Security's full blog covers the operational detail this post intentionally leaves for the source:

• A step-by-step attack chain showing how a poisoned document, email, or calendar event becomes executable AI context.
• The exact exfiltration path used to move data from Gmail, Calendar, and Docs into an outbound request.
• The architectural changes Google made after the issue was addressed, including the separation of Vertex AI Search from Gemini Enterprise.
• The practical defence model for AI search, including blast-radius mapping and provenance handling.

👉 The full Noma Security post covers the attack path, data exposure scope, and containment guidance.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.