Join our Newsletter — 33% off our NHI Course

Google Gemini and indirect prompt injection: are your controls keeping up?

 

(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 16497
Topic starter  

TL;DR: Unosecur says private calendar data was exposed through indirect prompt injection in Google Gemini because hidden instructions inside a calendar invite were treated as actionable context, allowing the system to summarize sensitive details into a new event. The deeper failure is that access control alone cannot govern whether an AI service should act on untrusted language in a given context.

Editorial analysis by NHI Mgmt Group, based on content published by Unosecur: “AI Prompt Injection & Identity Security Gaps”.

Key questions

Q: What breaks when a scheduled AI agent reads untrusted content and can also write to production systems?

A: The control boundary breaks at the moment the routine can act on external input without a human gate.

Q: Why do AI-assisted IAM workflows still need strict authorization controls?

A: Because an agent can interpret context and initiate actions, but it cannot be trusted to decide privilege on its own.

Q: What are the signs that prompt injection is becoming a governance problem?

A: Warning signs include rising jailbreak success rates, tools being called in ways the business did not intend, and changes in behaviour after prompt or connector updates.

Practitioner guidance

  • Map AI read-write chains to explicit context boundaries Document every AI service that can read untrusted content and write into another system, then define where context must be dropped before output is allowed to propagate.
  • Separate permission checks from intent checks Add policy decisions that evaluate whether an AI action makes sense in context, even when the underlying IAM grant is technically valid.
  • Classify calendar, email, and document inputs as hostile context Treat externally supplied metadata as untrusted until it has been isolated from model instructions and downstream write paths.

Bottom line: The Gemini incident exposed a governance failure in which an AI service acted within permission but outside safe context.

What's in the full article

Unosecur's full blog covers the operational detail this post intentionally leaves for the source:

  • Step-by-step identity monitoring recommendations for AI services that read and write across multiple applications
  • Detailed checklist for reducing blast radius when AI outputs can become visible outside the original system
  • Examples of access reviews and audit trails used to investigate suspicious AI-driven identity behaviour
  • Unosecur's own guidance on visibility, governance, monitoring, and response for AI services

👉 Read Unosecur's analysis of the Google Gemini indirect prompt injection incident →

Explore further

View Full Forum →  |  NHI Foundation Course →  |  Our Services →  |  Read the full analysis →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 5 months ago
Posts: 16988
 

AI identity governance fails when teams only ask whether an identity is authorised. Gemini had permission to read and write calendar data, yet the incident still leaked private information because the system acted on hidden instructions inside untrusted context. That shows the governance problem is not pure access control, but whether an AI service should execute a given action in the current context. Practitioners need to treat intent governance as a separate control plane.

A question worth separating out:

Q: How should security teams govern personal AI assistants that act on behalf of employees?

A: Treat each assistant as a distinct non-human actor with its own identity, policy scope, and audit trail. Human delegation alone is not enough when the assistant can move across email, documents, calendars, and internal systems. Governance should bind the sponsor, the executor, and the target resource so access reviews and investigations can separate request from action.

👉 Read our full editorial: Google Gemini incident exposes the AI identity governance gap



   
ReplyQuote
Share:

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.