A phishing report button only reduces risk if every report is triaged quickly, the verdict reaches the user, and confirmed threats are pulled from every other mailbox. Most security operations teams find the opposite: a reporting mailbox full of marketing mail, simulation emails and real phish mixed together, triaged by hand days later, with users who never hear back and stop reporting. This guide covers how to set up the report path so evidence arrives intact, how to cluster and auto-resolve the bulk of reports, how to escalate the real ones into incident handling, and how to close the feedback loop. It uses Microsoft Defender for Office 365 settings as the worked example because they are well documented, but the design applies to any mail platform or third-party reporting add-in. It is written for SOC leads, email security engineers and detection engineers. General alert queue practice is in the Alert Triage Guide; this page stays on user reported messages.
Key takeaways
- Preserve the original message. Reports must arrive as the unmodified original (EML or MSG) with headers intact; forwarded copies lose the headers that triage and remediation depend on.
- Protect the reporting mailbox from your own filters. Exempt it from anti-spam, safe links rewriting, DLP and simulation handling, or reported phish will be quarantined or altered before analysts see it.
- Cluster before you triage. One campaign can produce hundreds of reports; grouping by sender, URL, attachment hash and subject turns them into one decision and one remediation.
- Close the loop with the reporter. A verdict email (phishing, spam, no threat found) is what keeps users reporting, and it is the cheapest awareness training there is.
- Measure time to remediate, not time to read. The metric that matters is how quickly a confirmed phish is removed from every mailbox that received it, and how many users clicked before then.
How the report path works
A user reported message flows through four stages: the user reports it, the platform delivers it to a reporting mailbox and optionally to the mail vendor, the SOC or an automated workflow reaches a verdict, and the verdict drives both remediation and user feedback. Each stage has a design decision.
| Stage | Decision | Good practice |
|---|---|---|
| Report | Built-in client button or third-party add-in | Use one button only; two competing buttons split evidence and confuse users |
| Deliver | Reporting mailbox, mail vendor, or both | Both, where data handling rules allow, so vendor analysis and automation run alongside your own |
| Verdict | Manual, automated, or automated with analyst approval | Automate clustering and known-good dismissal; keep humans on new campaigns and anything targeted |
| Act | Remediate, block, notify, escalate | Purge all copies, block indicators, tell the reporter, open an incident if anyone clicked or replied |
Worked example: Microsoft Defender for Office 365 user reported settings
- Destinations: the user reported settings offer "Microsoft and my reporting mailbox", "My reporting mailbox only" and "Microsoft only" (the last only for the built-in Report button). US Government clouds (GCC, GCC High, DoD) are limited to the reporting mailbox only.
- Reporting mailbox: must be an Exchange Online mailbox in the organisation, not a distribution group or external address. Microsoft says to configure it as a SecOps mailbox in the advanced delivery policy and exclude it from DLP.
- Built-in button features: optional confirmation before reporting, a success pop-up, and customised pop-up text in up to seven languages; third-party add-ins do not get these pop-ups.
- Results email: body and footer can be customised for Phishing, Junk and No threats found outcomes, with your own sender address and logo.
- Configuration objects: in Exchange Online PowerShell the settings live in DefaultReportSubmissionPolicy and DefaultReportSubmissionRule (Get-ReportSubmissionPolicy, Get-ReportSubmissionRule), which makes them easy to export as configuration evidence.
Other platforms have equivalents: a client report button, a destination address or API, and an admin view of reports. Whatever the platform, record the configuration and check it after every tenant change.
Getting evidence that survives triage
Triage quality depends on what arrives in the reporting mailbox. Three rules cover most problems:
- Attach, do not forward: Microsoft's format requirements for non-Microsoft reporting tools say the original message must be attached unmodified as an uncompressed .EML or .MSG file, and that messages with several attached messages are discarded. Forwarding strips the headers analysts need.
- Keep the identifying headers: for Defender, the headers to preserve include X-Microsoft-Antispam-Message-Info, Message-Id, X-Ms-Exchange-Organization-Network-Message-Id and X-Ms-Exchange-Crosstenant-Id. The network message ID is what lets you find every other copy of the same message for remediation.
- Classify at the source: third-party tools tag the report type in the subject line (for Defender, "1|" or "Junk:", "2|" or "Not junk:", "3|" or "Phishing:"); messages with no prefix are treated as phishing. Make sure your add-in sets the prefix, or junk reports will inflate the phish queue.
Do not let the reporting mailbox become a hazard in its own right. Analysts should open reported content only in an isolated analysis environment, URL detonation and attachment analysis should run in a sandbox, and access to the mailbox should be limited to the SOC role group, because it holds live malicious content and sometimes sensitive mail reported by mistake.
Triage at scale: clustering, automation and verdicts
In most organisations the majority of reports are not malicious: newsletters, legitimate vendor mail, internal announcements and authorised phishing simulations. The goal is to resolve those automatically and spend analyst time on the rest.
- Separate simulations: recognise authorised simulation messages by header or sending infrastructure and route them to the awareness programme, not the SOC queue. Defender's advanced delivery policy is where both SecOps mailboxes and third-party simulations are declared.
- Cluster: group reports by sender address and domain, envelope sender, URLs and their final landing domain, attachment hashes, and normalised subject. Triage the cluster, not the individual report.
- Enrich: add authentication results (SPF, DKIM, DMARC), sender age and reputation, URL and file verdicts, the number of recipients of the same message, and whether any recipient clicked, replied or opened an attachment.
- Auto-resolve known patterns: bulk and marketing mail from authenticated senders, and messages identical to a cluster already marked clean, can be closed with a "no threat found" or "junk" verdict and a notice to the user.
- Escalate the rest: new clusters, targeted messages to executives or finance, credential harvesting pages, payment change requests and anything where a user interacted go to an analyst.
- Record the verdict: phishing, spam or junk, clean, or simulation, so the data feeds metrics and detection tuning.
Platform automation, worked example
With Defender for Office 365 Plan 2, choosing "Microsoft and my reporting mailbox" lets automated investigation and response (AIR) run on user reports. Microsoft documents that AIR investigates the message and surrounding evidence, closes the investigation where no threat is found or threats were already remediated (which it notes is common for user submissions), and otherwise queues remediation actions for SecOps approval in the Action center. The exception is malicious URL and file similarity clusters, where AIR approves its own pending actions. Microsoft also warns that the built-in tuning rule "Auto-Resolve - Email reported by user as malware or phish" can suppress the user report alert, and that suppressed alerts are not classified by its Security Copilot Phishing Triage Agent. Whichever tooling you use, confirm that automation is actually receiving the alerts you think it is.
Choosing the triage model
| Model | Fits | Trade-off |
|---|---|---|
| Manual mailbox review | Small organisations with low report volume | Slow; no clustering; depends on one person |
| Mail platform automation (for example AIR) | Organisations licensed for it and standardised on one platform | Good remediation reach; less control over logic; verify what is auto-approved |
| SOAR playbook on the reporting mailbox | SOCs with existing SOAR and mixed tooling | Full control and custom enrichment; you maintain the playbook and parsers |
| Managed service or MDR | Teams without 24x7 coverage | Coverage out of hours; needs clear remediation authority and feedback timelines in the contract |
Remediation and escalation
A confirmed phish is a containment problem across the whole mail estate, not a single message.
- Purge all copies: use the network message ID, sender and URL indicators to find every delivered copy and move it to quarantine or delete it.
- Block indicators: add sender, domain, URL and file blocks with an expiry. In Defender, admin submissions can create block entries, which default to 30 days; review whether that matches your intent.
- Check interaction: query URL click logs, proxy and DNS logs, and endpoint telemetry for the landing domain and attachment hash. Any click on a credential harvesting page triggers credential reset and session revocation for that user.
- Escalate to incident handling: confirmed credential entry, a reply to a payment or gift card request, or malware execution moves the case into the Phishing Incident Response Playbook.
- Submit misses to the vendor: a phish that reached inboxes is a filtering miss; submit it so the vendor's detections improve, and tune your own rules from the cluster's features.
For techniques common in reported messages, ATT&CK lists T1566 Phishing with sub-techniques T1566.001 Spearphishing Attachment and T1566.002 Spearphishing Link, and T1204.002 Malicious File for the user execution step. Tagging verdicts with these IDs makes reporting comparable with other detections.
Where identity fits
Most reported phish worth escalating is after credentials, tokens or consent. Triage should therefore ask identity questions, not only mail questions: did the user enter a password on the landing page, approve an MFA prompt, grant an OAuth consent, or enter a device code? Each has a different response. Credential entry needs a password reset plus revocation of refresh tokens and sessions; an illicit OAuth grant needs the app consent removed and its service principal reviewed, as covered in the Consent Phishing and Illicit OAuth Grants Guide; device code lures are covered in the Device Code Phishing Guide. The long-term fix for credential phish is the move to phishing-resistant authentication in the Phishing-Resistant MFA Guide.
The automation itself is a non-human identity. The SOAR or mail automation account that searches and purges mailboxes holds powerful permissions across every mailbox. Scope it to the minimum roles needed, store its credentials in a vault, monitor its actions, and review it like any other privileged service account.
Closing the feedback loop
User feedback is what keeps the reporting rate high. Effective feedback has three parts:
- Immediate acknowledgement: a pop-up or email confirming the report was received.
- A verdict: a short message saying whether the message was phishing, spam or safe, with one sentence on why. Defender's "Mark as and notify" action sends templated verdicts of No threats found, Phishing or Spam.
- Recognition: periodic thanks for reporters who caught real campaigns early, shared with their managers, links reporting to the Phishing Simulation Programme Guide metrics.
Keep verdict messages short and avoid blaming language for false reports: a user who reports a safe message is behaving correctly.
Detection and measurement
| Metric | How to measure | What good looks like |
|---|---|---|
| Time to first report for a campaign | Delivery time of first copy to first user report | Minutes, and falling |
| Time to remediate | First report to purge of all copies | Within your SLA, for example under one hour for confirmed phish |
| Interaction before remediation | Users who clicked, replied or entered credentials before purge | Falling as time to remediate falls |
| Auto-resolution rate | Reports closed by automation with no analyst touch | High for bulk and clean mail, with sampled quality checks |
| Verdict feedback rate | Reports where the user received a verdict | Close to 100% |
| Real report rate | Users who report real phish divided by users who received it | Rising; compare with simulation report rates |
Monitor the pipeline itself: alert if the reporting mailbox stops receiving reports, if the report submission rule is disabled, or if the queue age exceeds the SLA.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | RS.MA-02 Incident reports are triaged and validated; DE.AE-02, DE.AE-06; RS.AN-03 | Triage, analysis and sharing of user reports |
| NIST SP 800-53 Rev. 5 | IR-6 Incident Reporting; IR-4 Incident Handling; SI-8 Spam Protection; AT-2 Literacy Training and Awareness | Reporting channel, handling, filtering and user training |
| NIST SP 800-61 Rev. 3 | CSF 2.0 Community Profile for incident response | Places triage within detection and response |
| CIS Controls v8.1 | Control 9 Email and Web Browser Protections; Control 14 Security Awareness (14.6 recognising and reporting incidents); Control 17 Incident Response Management | Filtering, user reporting behaviour and response |
| MITRE ATT&CK v20 | T1566, T1566.001, T1566.002, T1204.002 | Tagging verdicts by technique |
Common failure patterns
- Reporting mailbox filtered by your own controls: reported phish quarantined, links rewritten or DLP-blocked before analysts see it, because the mailbox was never exempted.
- Two report buttons: a vendor add-in and the built-in button both deployed, sending reports to different places. Microsoft explicitly advises against running both at once.
- Simulations in the SOC queue: authorised simulation emails flooding triage, or worse, real reports being treated as simulation and closed.
- No verdict to users: reporting rates fall within months when reports disappear into silence.
- Email-only reporting: SMS and voice lures have no report path. The Twilio Breach 2022 began with SMS messages to employees posing as IT, and our analysis recommends making reporting easy for SMS and voice phishing as well as email.
- Single-message remediation: deleting the reported copy while hundreds of identical messages stay in other inboxes.
Practitioner checklist
- Deploy one report button across desktop, web and mobile clients, and document its configuration.
- Exempt the reporting mailbox from filtering, link rewriting, DLP and simulation handling.
- Require reports to arrive as unmodified EML or MSG attachments with headers intact.
- Route authorised simulation reports to the awareness team automatically.
- Cluster reports by sender, URL, attachment hash and subject before analyst review.
- Purge every copy of a confirmed phish and check click and sign-in logs for interaction.
- Escalate credential entry, consent grants and replies into the phishing incident playbook.
- Send every reporter a verdict, and recognise early reporters of real campaigns.
- Scope and monitor the automation account that searches and purges mailboxes.
- Report time to remediate and interaction before remediation monthly.
Standards and references
- Microsoft Learn: User reported settings in Defender for Office 365
- Microsoft Learn: Use the Submissions page to submit suspected spam, phish, URLs, legitimate email and attachments
- Microsoft Learn: Automated investigation and response in Defender for Office 365
- NIST: SP 800-61 Rev. 3 Incident Response Recommendations and Considerations for Cybersecurity Risk Management
- MITRE ATT&CK: T1566 Phishing
- CISA, NSA, FBI and MS-ISAC: Phishing Guidance: Stopping the Attack Cycle at Phase One
Related NHI Mgmt Group resources: Alert Triage Guide · Phishing Incident Response Playbook · Phishing Simulation Programme Guide · Phishing Guide