An email security gateway decides which mail reaches your people, often signs outbound mail as your domain and may hold rights to read and delete messages in every mailbox, so an attacker who controls it can let phishing through, send trusted mail as you and quietly collect email. The gateway stores quarantined messages that include malware and sensitive content, keeps allow and block lists that override detection, holds DKIM private keys and TLS certificates, and connects to the mail platform through connectors and API applications. This guide is for email administrators, security architects and SOC leads who run a secure email gateway or an API-connected email security service. It covers admin and quarantine roles, policy and allow-list integrity, signing keys and certificates, the service accounts and API permissions the gateway uses, outbound relay controls, the gateway's own logs and the lifecycle. It is vendor-neutral: confirm exact setting names in your vendor's documentation. For routing, sender authentication and filtering design, see the Email Security Guide.
Key takeaways
- Allow lists are the most dangerous setting on the gateway. A sender or domain allow entry overrides detection; restrict who can add one, give each an expiry and alert on every change.
- Quarantine release is a privilege. Releasing a held message can deliver malware; separate the people who review quarantine from those who can release high-risk verdicts.
- Treat DKIM keys as signing credentials. A stolen DKIM private key lets someone send mail that passes DKIM as your domain; use 2048-bit keys (our recommendation, in line with M3AAWG guidance), rotate them and never reuse selectors, as M3AAWG recommends.
- Scope the API application. Post-delivery remediation often needs rights over every mailbox; give it only the permissions it uses and monitor that identity like an administrator.
- Keep the gateway's audit log outside the gateway. Policy, allow-list, quarantine release and admin changes belong in a store email administrators cannot alter.
Why the gateway is a high-value target
The gateway is trusted by design: the mail platform accepts what it delivers, recipients trust what passes it, and receiving domains trust what it signs. The NCSC secure system administration principles (trusted management devices, protected admin interfaces, tiered administration, privileged access management and logged admin activity) apply to its console. NIST SP 800-177 Rev. 1, Trustworthy Email, covers the sender authentication and transport security the gateway usually performs.
The component table reflects a common architecture and is NHI Mgmt Group's view, not a sourced description of any one product; your platform may name, merge or split these components differently.
| Component | What it holds or can do | If an attacker controls it | Primary control |
|---|---|---|---|
| Admin console | Policies, allow and block lists, routing, roles, integrations | Allows their phishing domain, weakens policies, adds forwarding or journaling | SSO with phishing-resistant MFA, least-privilege roles, two-person rule for allow lists |
| Quarantine | Held malware, phishing and sensitive mail; release controls | Releases malicious mail, or reads sensitive held messages | Role-limited access, release approval for high-risk verdicts, logging |
| DKIM private keys | Signing keys for your domains | Sends mail that passes DKIM as your domain | 2048-bit keys, rotation, restricted export, unique selectors |
| TLS certificates and keys | Certificates for SMTP TLS and the admin interface | Intercepts or impersonates mail transport | Protected key storage, renewal tracking |
| API application in the mail platform | Read, move or delete messages across mailboxes | Collects or deletes email at scale (T1114) | Minimum permissions, certificate credential, monitoring |
| Connectors and relay | Trusted path into the mail platform; outbound smart host | Bypasses filtering or relays spam and phishing as you | Certificate-bound connectors, authenticated and restricted relay |
Admin and quarantine roles
- Role design: separate at least platform administrator (routing, integrations, roles), policy administrator (filtering policies and lists), quarantine reviewer (view and release low-risk verdicts), quarantine approver (release malware and high-confidence phishing verdicts), and read-only auditor. Keep full administrators to a handful.
- End-user release: if users can release their own quarantined mail, limit self-release to spam and bulk verdicts; require an administrator for malware and high-confidence phishing.
- Message content access: quarantine and message trace can show message bodies and attachments. Restrict content view to the roles that need it, and log every view and download.
- Authentication: sign in through your identity provider with SSO and phishing-resistant MFA. The CISA and FBI Secure by Demand guide asks vendors for standards-based SSO at no additional cost, MFA by default and no default passwords; use these as procurement questions. Keep one documented break-glass account. See the Break-Glass Account Guide.
- Admin devices: administer from privileged access workstations, never from the mailbox you read phishing in. See the Privileged Access Workstations Guide.
- Review: review roles every quarter and on every leaver or role change (our recommendation).
Policy and allow-list integrity
Attackers who reach a gateway console rarely switch it off. They add an allow entry for their sending domain, lower a policy threshold, or add a routing rule that copies mail elsewhere.
| Setting | Risk | Control |
|---|---|---|
| Sender, domain or IP allow list | Overrides spam, phishing and sometimes malware verdicts | Ticket, owner and expiry for each entry (90 days, our default); never allow your own domains; two-person approval |
| Policy thresholds and actions | Lowered spam or phishing action from quarantine to deliver | Policy as code or exported baseline; alert on change |
| Routing, forwarding and journaling rules | Copies of mail sent to an outside address (T1114.003) | Restrict to platform administrators; alert on any new external destination |
| Attachment and URL rewriting exceptions | Disables sandboxing or time-of-click checks for a sender | Same controls as allow lists |
| Outbound relay permissions | Unauthorised systems send as your domains | Authenticated relay limited to named sources; alert on new sources |
- Baseline: export the full configuration weekly and compare it with the last approved version; alert on drift.
- Clean up: on first review, remove allow entries with no owner and any entry covering your own domains or large free mail providers.
- Test: after every policy change, send test messages (a standard antivirus test file and a test phishing URL from your simulation platform) and confirm the expected verdicts.
Signing keys and certificates
If the gateway signs outbound mail, it holds the DKIM private keys for your domains. The M3AAWG DKIM Key Rotation Best Common Practices (revised March 2019) describe 2048-bit RSA keys as immune to cracking in today's computing environment and recommend rotation at least every six months, unique selector names that are never reused, keeping the old public key published for at least seven days after rotation before clearing its value, and never sharing keys across organisations. Where a third party signs for you, the guidance describes options including the customer generating keys, the vendor generating them and supplying the public key, or CNAME records pointing to vendor-hosted keys.
- Who generates the key: prefer the gateway generating the key and exposing only the public key, so the private key never travels. If you must import a key, move it only over an encrypted channel.
- Per-domain keys: use separate keys and selectors per domain and per sending service, so a leak affects one stream.
- Pre-publish the next key: M3AAWG suggests pre-publishing the next public key so an emergency rotation is fast.
- TLS: track certificates for SMTP and the admin interface in your certificate inventory and renew before expiry; if you publish MTA-STS, a broken certificate will stop inbound mail from senders that enforce it. See the SMTP Security, STARTTLS and MTA-STS Guide.
- Suspected key leak: rotate the DKIM key immediately under a new selector and clear the old record. See the DMARC, SPF and DKIM Guide and the Leaked TLS Private Key Response.
Service accounts, API applications and connectors
| Identity | Needs | Avoid | Credential pattern |
|---|---|---|---|
| Mail platform API application (remediation, scanning) | Read and move messages for remediation; read audit and rule data if documented | Tenant-wide administrator roles; send-as or full mailbox rights beyond what features use | Certificate credential, no client secret; scope to mailboxes where the platform supports it |
| Inbound connector into the mail platform | Deliver filtered mail | Trusting the gateway's shared IP ranges, which other customers also use | Bind to the gateway's TLS certificate name where supported; restrict to your domains |
| Outbound relay account | Send outbound mail through the gateway | Open relay or relay for any authenticated user | Per-system relay credentials or certificates, limited sender addresses |
| Directory sync for recipient validation | Read recipient addresses and groups | Write rights or privileged group membership | Read-only service account or app, managed credential |
| SIEM and SOAR integration | Pull logs; SOAR may search and remove messages | One key with admin rights for both | Separate read and act keys, each scoped and expiring |
- Monitor the API application: alert on credentials added to it (T1098.001), on use from unexpected addresses, and on bulk message reads or deletes outside remediation jobs. See the Consent Phishing and Illicit OAuth Grants Guide for reviewing high-privilege app grants.
- Lock the bypass path: the mail platform should accept inbound mail only through the gateway; this is covered in the Email Security Guide.
- Inventory: record every gateway identity with owner, permissions and credential expiry in your NHI inventory. See the Service Account Security Guide.
The gateway's own logs
| Event to capture | Why it matters | Alert? |
|---|---|---|
| Role change, new local user, SSO setting change | Persistence in the console | Always |
| Allow-list or exception added or widened | Detection overridden for a sender | Always, with approver |
| Quarantine release of malware or high-confidence phishing | Malicious delivery | Always |
| New routing, forwarding or journaling destination | Mail copied out (T1114.003) | Always |
| DKIM key generated, exported or deleted; certificate replaced | Signing or transport trust changed | Always |
| Message content viewed or downloaded by an administrator | Privacy and insider risk | Above a set volume |
Forward the gateway audit log to a SIEM or archive that email administrators cannot alter (NIST SP 800-53 AU-9(2)), keep it for at least six months as NCSC logging guidance advises, and check your contract: Secure by Demand asks SaaS providers to retain security logs for at least six months at no extra charge.
Hardening and lifecycle
| Stage | Security actions |
|---|---|
| Buy | Ask the Secure by Demand questions (SSO, MFA by default, no default passwords, security logs in the base product, vulnerability disclosure policy, SBOM). Ask which mailbox permissions each feature needs, where quarantined mail is stored and for how long, and how vendor staff access message content. See the Email Security Buyer's Guide and Email Security RFP Template. |
| Deploy | Enable SSO and audit log export first; generate DKIM keys on the gateway; bind inbound connectors to the gateway's certificate; restrict relay to named systems; start new policies in monitor mode where offered. |
| Run | For appliances and virtual gateways, apply vendor security updates promptly, subscribe to the vendor's security advisories page and allow admin interfaces only from a management network. For all deployments, review roles, allow lists and API permissions every quarter and rotate DKIM keys at least every six months. |
| Retire | Repoint MX and connectors first; revoke the API application consent and delete its credentials; remove relay credentials; retire DKIM selectors by clearing their values; obtain written confirmation that quarantine and logs held by the vendor are deleted after any export you need. |
Roll back a policy change by restoring the last approved configuration export; roll back a permission reduction by re-granting only the specific permission a failing feature reports as missing.
Detection and measurement
- Allow-list entries past expiry or without an owner (target: zero).
- Releases of malware or high-confidence phishing per month, each with an approver and case reference.
- Days since last DKIM key rotation per signing domain (target: under 183, in line with M3AAWG).
- Percentage of gateway identities using certificate credentials rather than secrets.
- Configuration drift events detected against the approved baseline, and time to explain each.
- Number of full gateway administrators (target: five or fewer, our suggestion).
Framework mapping
| Control area | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | ATT&CK threat addressed |
|---|---|---|---|
| Admin and quarantine roles | AC-2, AC-6, AC-6(5) | PR.AA-05 | T1078 Valid Accounts |
| Policy and allow-list change control | CM-3, CM-6 | PR.PS-04 | T1685 Disable or Modify Tools |
| DKIM keys and TLS certificates | SC-12, SC-8, IA-5 | PR.AA-01 | T1552.004 Private Keys |
| API application and connectors | AC-6, IA-5 | PR.AA-05 | T1114 Email Collection, T1098.001 Additional Cloud Credentials |
| Audit trail | AU-6, AU-9(2), AU-11, AC-6(9) | PR.PS-04, DE.CM-01 | T1114.003 Email Forwarding Rule |
| Vendor and supply chain | SA-9, SR-3 | GV.SC-07 | T1199 Trusted Relationship |
Common failure patterns
- Allow-listing your own domain: added to fix an internal notification, it lets spoofed mail claiming to be you skip checks.
- Everyone can release: help desk staff release malware verdicts because a user insisted the attachment was expected.
- Old DKIM keys: a short key set up years ago is still signing, or a retired selector still publishes a valid public key.
- Over-permissioned remediation app: the API application holds full mailbox and send rights it never uses, protected by a client secret with no expiry.
- Open relay paths: any internal host can relay through the gateway, so a compromised printer or application sends phishing as you.
- Hidden journaling rule: a copy-to-external rule added in the console is not reviewed because nobody exports the configuration.
Practitioner checklist
- Enforce SSO with phishing-resistant MFA for the console and keep one tested break-glass account.
- Separate platform, policy, quarantine reviewer, quarantine approver and auditor roles.
- Require approval, an owner and an expiry for every allow-list entry and exception, and alert on changes.
- Limit self-release to low-risk verdicts and log every content view and download.
- Generate DKIM keys on the gateway at 2048 bits, rotate at least every six months and never reuse selectors.
- Give the API application only the permissions its features use, with a certificate credential.
- Restrict outbound relay to named systems and alert on new routing or journaling destinations.
- Export the audit log off the gateway and compare configuration with an approved baseline weekly.
- Keep a decommissioning checklist covering MX, connectors, API consent, relay credentials and DKIM selectors.
Standards and references
- NIST: SP 800-177 Rev. 1 Trustworthy Email
- M3AAWG: DKIM Key Rotation Best Common Practices
- NCSC: Secure system administration
- NCSC: Introduction to logging for security purposes
- CISA and FBI: Secure by Demand Guide
- NIST: SP 800-53 Rev. 5 Security and Privacy Controls
- NIST: Cybersecurity Framework 2.0
- MITRE ATT&CK: T1114 Email Collection
Related NHI Mgmt Group resources: Email Security Guide · Email Security Buyer's Guide · DMARC, SPF and DKIM Guide · Securing the PAM Vault Itself · Securing the identity threat detection and response (ITDR) tool itself