Reduce open relay risk by restricting unauthenticated sending, allowing relay only for trusted users or networks, and testing that external recipients cannot be used for unrestricted forwarding. Require authentication where appropriate, enforce recipient controls, and review server settings after changes. Regularly verify the mail system does not accept third-party delivery it should reject.
How to stop a mail server from becoming an open relay
An open relay is usually a mail configuration problem, not a mail content problem. The practical fix is to make relay decisions explicit: only authenticated senders, or trusted networks and systems, should be allowed to hand mail to third-party recipients. Any path that can forward mail for arbitrary external recipients should be tested and closed.
The key control is recipient and sender discrimination. If a server accepts unauthenticated submission and then forwards to any destination, it needs tighter policy on who may submit mail, which recipients are allowed, and which interfaces are exposed for relay. That usually means separating authenticated submission from server-to-server delivery and reviewing relay rules after every configuration change.
Operationally, the safest posture is to confirm the system rejects mail that should not be relayed, especially messages submitted from outside the trusted boundary. That includes checking whether the mail service treats external recipients differently from internal recipients, and whether any fallback or legacy setting silently reopens relay behaviour after an upgrade or migration.
Where open relay exposure usually comes from
Open relay risk tends to appear when mail transport settings are broader than intended. Common failure points are permissive relay rules, weak recipient validation, insecure submission ports, and administrative exceptions that were added for convenience and never removed. A server may also become an unintended relay when different mail paths use inconsistent authentication or policy.
Another frequent source of exposure is boundary confusion. Mail systems often have one policy for trusted internal users, another for authenticated external submission, and a third for inbound mail from the internet. If those paths are not cleanly separated, the system can accidentally allow third-party delivery that should have been rejected. That is why relay testing after change is as important as the initial hardening.
Email Security Guide is the most useful deeper reference when you need the broader email architecture around MX routing, SPF, DKIM, DMARC, TLS, and remediation controls that sit around relay policy.
What to verify after you change relay rules
Verification matters because open relay is a behavioural condition, not just a configuration label. After tightening policy, test that unauthenticated users cannot submit mail for external delivery, that trusted senders are still able to send through the intended path, and that recipient restrictions behave consistently across all mail submission interfaces.
It is also worth verifying that server logs and mail queue behaviour match the policy you intended. A clean configuration can still be undermined by an overlooked connector, inherited setting, or alternate listener. If mail is still being accepted for arbitrary third-party recipients, the control is not working, even if the primary submission route looks correct.
When mail security is being reviewed alongside wider email protection, the same control set that governs authenticated sending, routing, and remediation is also where you check for adjacent gaps such as spoofing resistance and enforcement consistency. For that broader control plane, CISA cyber threat advisories remains a practical external reference point for email-abuse patterns and defensive response context.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST SP 800-53 Rev 5 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AC-3 — Access Enforcement | Relay permission is an access decision that must be enforced by policy. |
| IA-2 — Identification and Authentication (Organizational Users) | Authenticated submission is a core mitigation against unauthorised relay use. | |
| CM-6 — Configuration Settings | Open relay is typically created or removed through mail server configuration. | |
| Recommendation — Enforce relay rules so only authorised senders can submit messages for external delivery. Require authenticated access before allowing users to submit mail through the relay path. Baseline and review mail transport settings after each change to prevent unintended relay exposure. | ||
| CIS Controls v8 | CIS-5 — Account Management | Controlling who can submit mail directly reduces unauthorised relay access. |
| CIS-16 — Application Software Security | Mail server behaviour depends on secure configuration and validation of allowed flows. | |
| Recommendation — Limit mail submission to approved accounts and remove unnecessary send-capable access. Validate mail service configurations so external delivery cannot occur through unintended paths. | ||
Practitioner Guidance
What to verify: Treat relay testing as part of change validation, not a one-time hardening task. Confirm both positive cases, trusted submission still works, and negative cases, unauthenticated external relay does not.
Common mistake: Teams often lock down one mail submission path and assume the rest of the server follows the same rule set. In practice, legacy listeners, smart-host settings, and inherited relay exceptions are common ways an open relay reappears.
Decision rule: If the system can forward to arbitrary external recipients without proving the sender is allowed to do so, prioritise relay restriction before tuning spam or content filtering. Open relay is a boundary failure first, a filtering problem second.
Practitioner takeaway: The safest mail configuration is the one that makes relay eligibility explicit, testable, and narrow, so only intended users and systems can hand mail across the boundary.