Prompted phishing is phishing that uses a crafted prompt, message, or interaction to trick a user or AI system into revealing data or taking an unsafe action. It often targets chatbots, copilots, or other AI assistants by steering them to expose secrets, trust fraudulent instructions, or bypass controls.
How prompted phishing works
Prompted phishing uses a crafted prompt, message, or interaction to manipulate a person or AI system into revealing data or taking an unsafe action. The core trick is not technical exploitation alone, but the abuse of language, trust, and workflow context.
This matters because the target may treat the instruction as routine, urgent, or internally generated. When the recipient is a chatbot or copilot, the prompt can be shaped to elicit secrets, policy exceptions, or actions that the system should not normally perform.
Prompted phishing is often effective because it blends into legitimate conversation flows. A single misleading instruction can appear harmless on its own, yet still steer the target toward credential disclosure, unsafe approval, or a broken control path.
Where prompted phishing shows up
Prompted phishing appears in classic phishing messages, but also in chat-based help desks, AI assistants, and workflow tools that accept natural language input. In those environments, the attacker may use urgency, authority cues, or false context to get the target to comply.
It is especially dangerous when an assistant has access to connected tools, sensitive knowledge, or delegated actions. If the system is willing to follow a prompt without verifying the requester, the interaction can become a shortcut around normal checks.
For a broader view of common phishing patterns and layered defenses, see the Phishing Guide.
When the attacker is trying to abuse trust rather than exploit code, the Social Engineering Guide helps place the technique in a wider manipulation context.
Why prompted phishing is dangerous
The main danger is that the attack can turn a normal conversation into a disclosure or action channel. Secrets, tokens, session material, and internal instructions are all at risk if the target is persuaded to reveal or reuse them.
AI systems add another layer of exposure because they may synthesize responses that sound authoritative even when the input is malicious. If the prompt can override intended boundaries, the result may be data leakage, policy bypass, or an unsafe downstream action taken in good faith.
Prompted phishing can also chain into broader compromise. A stolen token, exposed API key, or coerced approval may be enough to move from message-level manipulation to account abuse, repository access, or further social engineering.
Real-world examples of prompt-led abuse often end in token theft or secret exposure, as seen in CoPhish OAuth phishing via Copilot Studio.
How organisations reduce exposure
Exposure starts with knowing which systems can be reached by prompts, which data they can surface, and which actions they can trigger. Organisations should treat any assistant or workflow that can reveal data or execute actions as part of the attack surface, not just a productivity feature.
Controls should focus on restricting what the system can disclose, requiring confirmation for sensitive actions, and making trusted sources explicit. Verification matters when the prompt may be impersonating a person, a service, or an internal workflow.
Where the assistant can access credentials or developer material, the organisation should assume that a successful prompt attack may become a secret-exposure event. That is why guidance on phishing, access paths, and trust abuse should be read alongside cases such as Dropbox GitHub breach 2022 and Ledger Connect Kit npm compromise 2023, where phishing and access misuse led to material secret and access loss.
Risk and Threat Considerations
Prompted phishing is risky because it can bypass technical controls by targeting the judgment layer: the user, the assistant, or both. Once the interaction is framed as legitimate, the target may voluntarily reveal data or approve an unsafe step that would otherwise be blocked.
Failure mechanism: The attacker crafts a prompt that appears routine or authoritative, then uses that framing to elicit secrets, override caution, or induce an unsafe action inside a chat or assistant workflow.
Impact: The result can be secret disclosure, account or session abuse, fraudulent approvals, or follow-on compromise through tools and integrations that trust the compromised interaction.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5, NIST SP 800-63 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | IA-5 — Authenticator Management | Prompted phishing often aims at secrets and tokens that authenticate access. |
| AC-6 — Least Privilege | The term centers on unsafe actions that should not be available from a single prompt. | |
| SI-4 — System Monitoring | Detection of suspicious prompt-led disclosure or action requests depends on monitoring. | |
| Recommendation — Manage and rotate authenticators so a coerced disclosure cannot remain useful. Restrict assistant and user permissions to the minimum needed for each task. Monitor assistant interactions for unusual disclosure requests and unsafe action patterns. | ||
| NIST SP 800-63 | Phishing-Resistant Authenticators | The subject involves phishing pressure on authentication and trust decisions. |
| Recommendation — Prefer phishing-resistant authentication for systems that gate sensitive actions. | ||
| CIS Controls v8 | CIS-6 — Access Control Management | Prompted phishing seeks unauthorized access or action through manipulated requests. |
| CIS-8 — Audit Log Management | Investigating prompt-based abuse requires reliable logs of prompts and outcomes. | |
| Recommendation — Limit and review access paths that a coerced prompt could abuse. Log assistant prompts, disclosures, and tool actions for later review. | ||
Practitioner Guidance
What to watch for: Treat any assistant behavior that asks for secrets, urges urgency, or shifts to unfamiliar instructions as a review point. The warning sign is not only the content of the prompt, but whether the request causes the system to step outside its normal role.
Governance implication: Define which prompts, actions, and disclosures require verification, human approval, or refusal. Assistant owners should be explicit about what the system is never allowed to reveal or execute from free-form user input.
Practitioner takeaway: Prompted phishing is best managed as a trust-boundary problem, not just a content-filter problem.