Notification-based Gemini prompt injection exposes AI assistant risk

At a glance

What this is: SafeBreach shows that notification-based prompt injection can steer Gemini into unauthorized actions by abusing trusted messaging channels and assistant context.

Why it matters: This matters because AI assistants now sit inside identity and trust paths, so governance must cover message intake, tool delegation and persistent memory, not just model output.

Context

Notification-based indirect prompt injection is a trust-boundary problem, not just a content-safety problem. The attacker leverages inbound messages, notification summaries and assistant context to smuggle instructions into an AI system that is expected to behave as a helper, not as a decision engine.

The Gemini case shows why agentic AI governance cannot stop at prompt filtering. When a message channel, a notification reader and a tool-using assistant are chained together, the security question becomes who or what is allowed to influence action, memory and delegation across those boundaries.

Key questions

Q: What breaks when notification content is allowed to shape AI assistant actions?

A: The trust boundary breaks. Notification content can be mistaken for legitimate conversation context, which lets attackers steer output, trigger tools and seed persistence without direct model access. The safer pattern is to treat message intake as untrusted input until policy checks, provenance checks and intent validation have all passed.

Q: Why do notification-based prompt injections create a governance problem for AI assistants?

A: Because the assistant is not only interpreting text. It is converting untrusted notifications into context that can shape decisions, trigger tools, and persist across sessions. That turns a messaging channel into an authorisation path, which breaks the assumption that only the user can create legitimate intent.

Q: How can security teams detect context poisoning in AI agents?

A: Look for behavioural drift, especially unexpected tools, new databases, altered output destinations, and role changes that do not match the agent’s approved mission. Detection works best when teams baseline normal task patterns and compare runtime activity to that baseline continuously.

Q: Should organisations restrict memory and scheduled actions in agentic assistants?

A: Yes, unless those features are explicitly governed. Memory and recurring tasks create persistence, which means a single successful injection can survive the original conversation and affect future sessions. Organisations should treat them like high-risk delegated privileges and require approval, scope limits and revocation paths.

Technical breakdown

How notification-based prompt injection reaches the assistant context

The attack begins when a notification-reading component ingests untrusted message content and folds it into the assistant’s working context. That content may be hidden in plain sight, obfuscated, muted, or formatted to look like a benign system message. Once the assistant treats the notification stream as part of the conversation, the attacker no longer needs direct access to the model. The real failure is context contamination across channels that were never supposed to be equivalent. In an agentic system, that is enough to reshape subsequent outputs, recommendations and tool decisions.

Practical implication: Treat notification ingestion as an attack surface and separate untrusted message content from assistant decision context.

Why fake context alignment bypasses delayed tool invocation

Delayed tool invocation defenses try to stop assistants from acting on hidden instructions until a later user confirmation. SafeBreach’s fake context alignment technique works by making the security layer believe a legitimate permission exchange occurred, while the user experiences something else entirely. The model sees an apparent user consent path, so tool execution is re-enabled. This is a governance failure rooted in cross-layer inconsistency: the security check reasons over one interpretation of the dialogue, while the user sees another. That mismatch is exactly what attackers exploit.

Practical implication: Design tool-approval controls around verifiable intent, not just conversational consistency.

How long-term memory and recurring actions turn a one-off injection into persistence

Once the attacker can shape assistant context convincingly enough, persistent features become reachable too. Long-term memory writes and scheduled recurring actions convert a transient prompt injection into a durable control plane inside the assistant. That matters because persistence shifts the problem from a single bad answer to ongoing influence over future sessions, devices and workflows. In governance terms, memory is not just storage and schedules are not just convenience features. They are runtime permissions that need the same scrutiny as delegated access.

Practical implication: Put explicit policy and approval controls around assistant memory writes and recurring task creation.

Threat narrative

Attacker objective: The attacker aims to turn trusted notifications into a durable influence channel that can drive unauthorized assistant actions and persistent compromise.

1. Entry begins when malicious instructions are delivered through notification-bearing channels such as WhatsApp, Slack or SMS and absorbed into the assistant’s context.
2. Credential or trust abuse follows when the assistant is induced to speak as if a trusted contact or system message were authentic, which alters user perception and system state.
3. Escalation occurs when fake context alignment makes the assistant believe the user approved a tool action, enabling device control, URL opening, app launch or memory persistence.
4. Impact is unauthorized execution, including social engineering, smart-device control, live video streaming, recurring actions and long-term memory poisoning.

Breaches seen in the wild

• Gemini AI Breach — Google Calendar Prompt Injection: Gemini AI assistant prompt injection attack leaks sensitive Google Calendar data.
• Gemini CLI prompt injection flaw 2025: Tracebit showed a poisoned README could make Gemini CLI run hidden commands and exfiltrate developer secrets; Google fixed it in 0.1.14.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Notification channels have become an identity boundary for AI assistants. Once assistants ingest chat and alert content as if it were conversational truth, the trust model collapses. The relevant control question is no longer only what the model says, but who or what is allowed to speak into its decision context. Practitioners should treat message intake as a governed trust boundary.

Fake context alignment is a more precise failure mode than generic prompt injection. The problem is not simply that the model is manipulated, but that the security layer and the user experience can be made to disagree about what was authorised. That gap is especially dangerous in agentic AI because tool invocation depends on inferred intent, not just content inspection. Practitioners need controls that validate intent across layers, not just text.

Memory and recurring tasks are de facto NHI-style privileges inside assistants. They persist across sessions, influence future behaviour and can outlive the triggering conversation. That makes them governance objects, not convenience features. In practice, any system that can remember, schedule or delegate action should be managed as a high-risk non-human control surface.

Context trust gap: this article surfaces a named concept for AI assistant governance where the assistant, the user and the security layer each interpret the same interaction differently. That gap is what lets a benign-looking notification become an execution path. The control response is to narrow which channels can influence tool use and to log cross-channel transitions for review.

Notification-driven assistants widen the attack surface beyond the model itself. The article shows that the trust problem spans messaging apps, device notifications, voice interaction and downstream app control. That means AI governance has to join up identity, platform policy and workflow permissions. Practitioners should audit the whole path from inbound message to outbound action, not just the prompt.

From our research library:

• Roughly 1 in 3 phishing payloads are delivered outside email, through channels such as social media, search ads and messaging apps.

What this signals

AI assistant governance now has to cover message provenance, not just model safety. When notifications from chat apps can alter context, the practical control is to make the path from message to action explicit, logged and reviewable. That is especially important where assistants can call tools, remember preferences or trigger follow-on workflows.

Persistence inside an assistant should be treated as delegated privilege. Long-term memory and recurring actions are not neutral convenience features. They create durable influence that can survive the original injection, so the programme question becomes who can write them, who can read them and how they are revoked.

Context trust gap: this article shows how easily a benign-looking notification can be transformed into an execution path when the assistant, user and control layer disagree about intent. Practitioners should look for any place where one channel can silently authorise another.

For practitioners

• Separate notification ingestion from assistant decision-making Classify incoming message content as untrusted until it is explicitly normalised and policy-checked before it can influence tool use or memory.
• Constrain tool execution to verified user intent Require a stronger approval signal for device actions, app launches and external links than conversational agreement or implied consent.
• Gate long-term memory writes and recurring tasks Apply review and approval controls to any feature that persists beyond the current session, including saved memory and scheduled automations.
• Monitor for cross-channel context shifts Alert when a notification, message or summary changes assistant state in ways that later enable tool invocation, contact impersonation or persistence.
• Reduce trusted-contact impersonation risk Limit how assistants can attribute content to real contacts and prevent generated messages from being presented as authentic without provenance.

Key takeaways

• Notification-driven prompt injection turns everyday messaging into a control problem for AI assistants, not merely a content filtering problem.
• The research shows that trusted channels, context poisoning and tool delegation can combine to produce unauthorized actions, social engineering and persistence.
• Governance needs to extend to memory, recurring tasks and provenance-aware approvals if organisations want to reduce assistant abuse.

Key terms

• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Fake Context Alignment: A manipulation pattern where the assistant is tricked into believing a user has authorised an action while the user sees only a benign interaction. The attack aligns the model’s internal state with attacker intent, creating a policy bypass that can lead to tool use or persistent state changes.
• Assistant Memory Persistence: Assistant memory persistence is the ability of an AI assistant to retain information or behaviour across sessions and later reuse it during future interactions. In security terms, that creates a durable influence channel, so writes to memory should be governed like persistent privilege rather than convenience storage.
• Notification Ingestion Boundary: The notification ingestion boundary is the point where message data from external apps enters an assistant’s working context. If that boundary is weak, untrusted content can shape reasoning, output and tool decisions, so provenance, filtering and authorization need to happen before the content becomes actionable.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, agentic AI identity and secrets management. It helps practitioners build the control model needed for delegated access, persistence and trust boundaries.