Man-in-the-Prompt Attack

What a man-in-the-prompt attack does

A man-in-the-prompt attack sits inside the instruction path of an AI system. Instead of merely supplying malicious content, the attacker changes what the model treats as authoritative guidance, so the system may follow the injected instructions over the user’s original intent. That makes it an integrity problem for both prompting and downstream agent behaviour.

The core danger is that the model can appear to be responding normally while its priorities, constraints, or action choices have been silently rewritten. That can happen through retrieved documents, tool output, hidden text, or other context that the system ingests during runtime.

Where the attack enters the prompt chain

Man-in-the-prompt attacks usually exploit a trust boundary inside the AI application pipeline. The system may assume retrieved text, file contents, browser-rendered pages, or tool responses are safe enough to influence reasoning, even when those sources are not trustworthy.

This matters because the attack does not need to defeat the model directly. It only needs to place instructions where the model will read them, then rely on the model’s instruction-following behaviour to reinterpret the conversation or task.

For a broader defensive view of prompt injection patterns, Prompt Injection Guide covers direct, indirect, multimodal and tool-poisoning variants.

Why it is dangerous in AI agents

The attack becomes more serious when the AI system can take actions, call tools, or make access decisions. A poisoned prompt can redirect the agent toward unsafe tool use, disclosure, or policy bypass while still looking like a legitimate task completion path.

That is why the same pattern often overlaps with agent goal hijack: the attacker is not just changing the text the model sees, but steering the objective the system pursues. In practice, the model may preserve fluency while losing alignment with the user’s real request.

When an agent is in scope, the relevant defensive lens is often goal integrity and action gating, which is the focus of OWASP Agentic Top 10 ASI01 Guide.

Attackers can also abuse hidden or delayed instructions embedded in data that the system later reuses. That makes the issue especially hard to spot because the malicious influence may be separated in time from the eventual harmful action.

How organisations reduce exposure

Exposure checks start with identifying every place the system ingests untrusted text before it reasons or acts. That includes retrieval layers, browser content, uploaded files, tool outputs, and any feature that mixes user input with external context.

Defences should preserve a clear hierarchy between system instructions, user intent, and untrusted retrieved material. The practical aim is not to trust all context equally, but to make the model less willing to obey instructions that arrive from lower-trust sources.

For organisations that already use search, retrieval, or tool-augmented workflows, the key question is whether the AI application can distinguish content from instruction. That is where the Man-in-the-Middle Attacks Guide is useful as an analogy for interception and trust-boundary thinking, and the MITRE ATT&CK T1557 Adversary-in-the-Middle page is a useful detection reference for relay and poisoning patterns.

Detection and recovery after suspected prompt compromise

Detection is often indirect because the model may not crash or error. Instead, you look for unusual instruction adherence, tool calls that do not match the user request, data exfiltration behaviour, or responses that suddenly adopt language from untrusted content.

After suspected exploitation, review the prompt chain, retrieved sources, tool outputs, and agent actions around the event window. The goal is to determine whether the system merely consumed bad content or actually executed attacker-shaped instructions.

On the recovery side, organisations should assume that any compromised session or task context may need to be rebuilt from a clean state. For AI systems that call external services or handle sensitive material, the compromise review should extend to any credentials, tokens, sessions, or service permissions that may have been used while the prompt was poisoned.

Related resources from NHI Mgmt Group

• Prompt Attack
• Who is accountable when a prompt bombing attack succeeds?
• Who is accountable when a man-in-the-middle attack succeeds through weak authentication?
• Who is accountable when an AI model exposes data after a prompt attack?