Indirect prompt injection increases risk because the attacker does not need to compromise the model itself. They only need to place instructions inside ordinary content that the assistant will process. Email, documents, calendar invites, and web pages can carry hidden commands that humans overlook, while the assistant may execute them if guardrails and action controls are weak.
Why This Matters for Security Teams
indirect prompt injection is especially dangerous in enterprise inboxes because email is already a trusted business channel, and AI assistants are often granted broad read, summarize, and act capabilities. That creates a path for hostile instructions to arrive as ordinary business content, then be interpreted by the assistant as if they were legitimate user intent. The control problem is not just malicious text, but the combination of trust, automation, and weak action boundaries. The OWASP Agentic AI Top 10 treats this as a core risk because agentic systems can be manipulated through untrusted input that influences downstream decisions.
Security teams often underestimate how quickly this becomes an identity and privilege problem. If an assistant can search mail, extract secrets, draft replies, create calendar events, or trigger workflow actions, then a single injected prompt can shift from content manipulation to operational misuse. The risk rises further when the assistant has access to delegated accounts, shared mailboxes, CRM data, or internal knowledge repositories. In practice, many security teams encounter this only after an assistant has already exposed sensitive data or executed an unintended action, rather than through intentional abuse testing.
How It Works in Practice
Indirect prompt injection works by hiding instructions inside content that the assistant is expected to process. The content may be obvious to a human reviewer, such as a phishing email, or subtle, such as a document footer, HTML comment, invoice note, or calendar description. The assistant then blends those instructions with the user’s request unless the system separates untrusted content from trusted directives and constrains what the assistant is allowed to do.
Current guidance suggests three practical layers of defense. First, treat all external content as untrusted input, even when it enters through a legitimate mailbox. Second, constrain tool use so the assistant cannot freely send mail, approve actions, or retrieve high-value data without explicit user confirmation. Third, validate outputs before execution, especially when the assistant is generating actions rather than summaries. NIST’s NIST Cybersecurity Framework 2.0 is useful here for framing governance, protection, detection, and response around AI-enabled workflows, while NIST SP 800-53 Rev 5 Security and Privacy Controls helps translate that into access control, system monitoring, and approval gates.
- Separate system prompts, user prompts, and retrieved content so instructions do not blend together.
- Apply least privilege to inbox access, mailbox delegation, and connected tools.
- Require human approval for external side effects such as sending mail, creating tickets, or sharing files.
- Log prompt inputs, tool calls, and model outputs for investigation and rollback.
- Test with adversarial mail and document samples during red team exercises.
These controls tend to break down when assistants are embedded into legacy email workflows with broad delegated access and no tool-level approval boundary.
Common Variations and Edge Cases
Tighter approval controls often increase user friction and workflow latency, requiring organisations to balance automation value against the risk of silent misuse. Best practice is evolving here, and there is no universal standard for how much autonomy an enterprise inbox assistant should have by default.
The hardest edge cases appear when content is both legitimate and hostile, such as vendor emails, legal attachments, or threaded conversations that mix business instructions with hidden model instructions. Risk also rises when retrieval systems pull in old messages or archived documents that were never written with machine consumption in mind. In those cases, the assistant may treat stale or irrelevant text as current authority. That is why output validation matters as much as input screening.
Identity becomes relevant when the assistant acts on behalf of a person or service account. If the system cannot reliably distinguish the human user’s intent from instructions embedded in retrieved content, then delegated authority can be misused without direct credential theft. The operational answer is to keep assistant permissions narrower than the user’s full mailbox privilege wherever possible, and to apply change control for any action that crosses a trust boundary. This risk is amplified in high-volume environments such as executive mailboxes, customer support desks, and shared service accounts.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 address the attack and risk surface, while NIST AI RMF, NIST CSF 2.0, NIST SP 800-63 and NIST AI 600-1 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | A2 | Indirect prompt injection is a primary agentic AI attack path. |
| NIST AI RMF | AI risk management covers governance for manipulated model behaviour. | |
| NIST CSF 2.0 | PR.AC-4 | Least privilege limits what an assistant can do after injection. |
| NIST SP 800-63 | Delegated actions depend on trustworthy user and session identity. | |
| NIST AI 600-1 | GenAI systems need safeguards against untrusted instructions in inputs. |
Classify mailbox content as untrusted and gate every tool action behind explicit policy checks.
