Defenders who cannot read a DNS message cannot tell a beaconing implant from a busy CDN, a poisoned cache from a slow resolver, or a tunnel from a TXT-heavy mail check. Almost every intrusion touches DNS, and DNS logs are among the cheapest and richest telemetry an organisation has, but only if analysts understand what normal looks like at the protocol level. This guide explains the parts of DNS that matter for security work: the message format, how resolution and delegation actually happen, record types, TTLs and caching (including negative caching and serve-stale), transport over UDP and TCP, and how each of these shows up in detection of command and control, tunnelling and domain abuse. It is for SOC analysts, detection engineers, network engineers and incident responders. Control choices such as protective DNS and DNSSEC are covered in the DNS Security Guide; this page is the protocol foundation underneath it.
Key takeaways
- Resolution is a chain of trust you do not see. A stub asks a recursive resolver, which follows referrals from the root to the authoritative server; whoever controls any link, or the resolver itself, controls the answer.
- The response code and flags carry as much signal as the name. NXDOMAIN bursts suggest domain generation algorithms; NODATA, truncation and the AA, RD and RA bits tell you who answered and how.
- TTL is a control and a clue. It sets how long a cached answer, good or malicious, persists; very low TTLs on rarely seen domains are a classic fast flux and C2 indicator, and negative answers are cached too (RFC 2308).
- Log at the recursive resolver, not only at the edge. Resolver logs tie each query to an internal client; edge logs see only the resolver.
- DNS is two transports. RFC 7766 makes TCP support mandatory for all general-purpose implementations; blocking TCP 53 breaks large responses and hides nothing.
The DNS message: what every analyst should be able to read
RFC 1035 defines one message format for queries and responses: a fixed 12-byte header followed by four sections (question, answer, authority, additional). Most security tooling flattens this into a log line, so it is worth knowing which fields the line came from.
| Field (RFC 1035 unless noted) | Meaning | Why defenders care |
|---|---|---|
| ID (16 bits) | Matches a response to its query | Only 65,536 values: off-path spoofing depends on guessing ID plus source port, which is why port randomisation matters |
| QR, OPCODE | Query or response; standard query is OPCODE 0 | Non-zero opcodes (for example dynamic update, OPCODE 5 from RFC 2136) in client traffic deserve attention |
| AA | Answer is authoritative | A response to a client from your recursive resolver should not normally carry AA unless the resolver also hosts the zone |
| TC | Message truncated | Client should retry over TCP; frequent TC can mean large TXT or DNSSEC answers, or tunnelling payloads |
| RD, RA | Recursion desired, recursion available | An internet-facing server answering RD queries for anyone is an open resolver |
| RCODE | 0 NOERROR, 1 FORMERR, 2 SERVFAIL, 3 NXDOMAIN (Name Error), 4 NOTIMP, 5 REFUSED | NXDOMAIN rates drive DGA detection; SERVFAIL spikes can indicate DNSSEC validation failures or upstream attack |
| OPT pseudo-record, type 41 (RFC 6891, EDNS) | Advertises UDP payload size, the DNSSEC OK (DO) bit and extended RCODEs | Large advertised payloads are normal; unusual EDNS options can fingerprint tools |
Name limits: labels are at most 63 octets and full names at most 255 octets (RFC 1035). Tunnelling tools pack data into labels, so queries with many labels close to 63 characters and high character entropy are a strong signal. NODATA is not an RCODE: RFC 9499 defines it as NOERROR with an empty answer section, usually with an SOA in the authority section, meaning the name exists but not with that record type.
How resolution actually works
RFC 9499 (BCP 219, March 2024) is the current terminology reference, and its roles map directly onto where you can observe and control DNS.
- Stub resolver on the endpoint sends a query with RD set to its configured resolver. It does not do the work itself.
- Forwarder (often a branch router, a domain controller or a security agent) may sit in between and pass the query on.
- Recursive resolver checks its cache. On a miss, it queries a root server, receives a referral (NS records for the top-level domain, plus glue address records where needed), queries the TLD servers, receives another referral, and finally queries the zone's authoritative server, which answers with AA set.
- The recursive resolver caches every RRset it learned for its TTL and returns the answer to the client.
Delegation is simply an NS RRset in the parent zone pointing at the child's servers. The zone apex is where the child's SOA and authoritative NS RRset live. A mismatch between parent and child NS sets, or delegation to a name server name in a domain that has lapsed, is a hijack opportunity: whoever registers that name controls the zone. See the Authoritative DNS and Registrar Account Security Guide.
- QNAME minimisation (RFC 9156, Standards Track, November 2021): a resolver sends upstream only the labels needed to find the next delegation, so root and TLD servers no longer see the full name. It does nothing to hide queries from the recursive resolver itself.
- Open resolvers: RFC 9499 defines an open resolver as a full-service resolver that answers queries from almost any client. Internet-facing open resolvers are abused for reflection and amplification. Restrict recursion to your own address ranges.
- Where to observe: the stub sees per-process queries (endpoint telemetry such as Sysmon Event ID 22 on Windows records DNS queries per process), the recursive resolver sees per-client queries, and the network edge sees only resolver-to-internet traffic. Detection engineering usually needs the resolver view joined to the endpoint view.
Record types that matter in investigations
| Type (code) | Purpose | Security relevance |
|---|---|---|
| A (1), AAAA (28) | IPv4 and IPv6 addresses | Resolution of C2 and phishing infrastructure; rapidly changing answer sets indicate fast flux (T1568.001) |
| NS (2) | Delegation to authoritative servers | Unexpected NS changes on your domains signal hijack (see the DNS Hijack Playbook) |
| CNAME (5) | Alias to another name | Dangling CNAMEs to deprovisioned cloud resources enable subdomain takeover |
| SOA (6) | Zone authority: MNAME, RNAME, SERIAL, REFRESH, RETRY, EXPIRE, MINIMUM | SERIAL changes show zone edits; MINIMUM sets negative caching |
| PTR (12) | Reverse mapping address to name | Internal PTR sweeps are reconnaissance (T1590.002 covers DNS information gathering) |
| MX (15) | Mail exchangers with preference | Mail routing; MX changes can redirect inbound mail |
| TXT (16) | Free text | SPF, DKIM, domain verification tokens, and also the most common carrier for DNS tunnelling and C2 payloads |
| SRV (33) | Service location | Active Directory locates domain controllers through SRV records; external SRV lookups for internal services indicate leakage |
| OPT (41) | EDNS pseudo-record | Not data; controls payload size and DNSSEC behaviour |
| DNSKEY, DS, RRSIG, NSEC, NSEC3 | DNSSEC | Validation failures surface as SERVFAIL; NSEC chains can allow zone walking |
| HTTPS (65), SVCB (64) | Service binding | Carry encrypted ClientHello keys and ALPN hints; affect what TLS inspection can see |
| NULL (10), ANY (255) | Legacy or meta query types | Rare in normal client traffic; NULL is favoured by some tunnelling tools and ANY by amplification abuse |
TTL and caching: how long an answer lives
Every resource record carries a TTL in seconds, a 32-bit field (RFC 1035). Recursive resolvers cache each RRset for its TTL and serve it to every client that asks, so a single poisoned or malicious record affects everyone behind that resolver until it expires.
- Positive caching: the TTL is chosen by the zone owner. Legitimate CDNs and load balancers often use short TTLs (tens of seconds to a few minutes); stable infrastructure uses hours or days. A short TTL is not malicious by itself; a short TTL on a newly registered domain that one internal host resolves every 60 seconds is.
- Negative caching (RFC 2308): NXDOMAIN and NODATA answers are cached for the lower of the SOA MINIMUM field and the SOA record's own TTL. RFC 2308 reports one to three hours working well and values over a day being problematic. If you sinkhole or delete a name, negative caches elsewhere determine how quickly clients notice.
- Serve-stale (RFC 8767, March 2020): resolvers may answer with expired data when authoritative servers are unreachable, with a recommended stale TTL of 30 seconds and a suggested maximum stale period of one to three days. RFC 8767 also describes capping accepted TTLs at seven days. It improves resilience during outages, but it also means a taken-down malicious domain can keep resolving from caches for a while.
- Flushing during incidents: after you change a record in response to a hijack, clients may keep the old answer until TTLs expire. Lower TTLs ahead of planned changes, and flush your own resolvers' caches as part of response.
| Observation | Benign explanations | Suspicious when |
|---|---|---|
| TTL under 300 seconds | CDN, global load balancing, failover | Domain is newly registered, rarely queried and resolves to changing hosting ranges |
| High NXDOMAIN rate from one host | Misconfigured search suffix, typo in software config | Names look algorithmic (random characters, consistent length, many TLDs): T1568.002 |
| Many unique subdomains of one domain | Telemetry and CDN beacons, anti-virus cloud lookups | Long high-entropy labels, TXT or NULL queries, steady volume: T1071.004 or T1572 tunnelling |
| Regular query interval to one name | Update checks, health monitors | Low-popularity domain, jitter pattern typical of implants (see the Beaconing and C2 Detection Guide) |
Transport: UDP, TCP and what changes with encryption
- UDP 53: the default. RFC 1035 limited UDP messages to 512 bytes; EDNS (RFC 6891, STD 75) lets clients advertise larger sizes, with 4096 octets suggested as a starting point and smaller fallbacks around 1280 to 1410 bytes where fragments are dropped.
- TCP 53: used for zone transfers and whenever a response is truncated. RFC 7766 (March 2016) states that all general-purpose DNS implementations must support both UDP and TCP. Firewalls that block TCP 53 cause intermittent failures for large DNSSEC and TXT answers.
- Zone transfers (AXFR, IXFR): run over TCP and hand over a whole zone. Restrict them to named secondaries and authenticate them, because an open transfer maps your network for an attacker.
- Encrypted transports: DNS over TLS (port 853), DNS over HTTPS (port 443) and DNS over QUIC hide queries from on-path observers, including your own network monitoring. Enterprises need to decide which resolver endpoints are allowed. See the Encrypted DNS Enterprise Guide.
Turning protocol knowledge into detection
Good DNS detection starts with complete, well-structured logs. Capture at least: timestamp, client address (and host or process where possible), query name, query type, response code, answer data, TTL, and which resolver answered. The DNS Logging and Detection Guide covers pipelines; the protocol-based analytics below are the core set.
| Analytic | Protocol fields used | ATT&CK technique |
|---|---|---|
| Algorithmically generated names and NXDOMAIN bursts | QNAME entropy, RCODE 3 rate per client | T1568.002 Domain Generation Algorithms |
| Fast flux | Many distinct A or AAAA answers, short TTLs, diverse networks | T1568.001 Fast Flux DNS |
| DNS tunnelling and C2 | Label length near 63, many unique subdomains per parent, TXT or NULL qtype, response sizes, TC bits | T1071.004 Application Layer Protocol: DNS; T1572 Protocol Tunneling |
| Data exfiltration over DNS | Total bytes in query names per client per domain | T1048.003 Exfiltration Over Unencrypted Non-C2 Protocol |
| Resolver bypass | Clients sending port 53, 853 or known DoH traffic to non-approved resolvers | Supports detection of all of the above |
| Hijack of your own zones | NS, MX, A changes on owned domains; SOA SERIAL jumps | T1584.002 Compromise Infrastructure: DNS Server |
| Cache poisoning attempts | Mismatched IDs, bursts of responses for one name, unexpected AA | CAPEC-142 DNS Cache Poisoning |
Implementation steps:
- Force all clients to use approved recursive resolvers and block outbound 53 and 853 from everything else.
- Enable query and response logging on those resolvers, including RCODE and answers, and forward to the SIEM.
- Join resolver logs with endpoint DNS telemetry to attribute queries to processes.
- Build a baseline of domain popularity across your estate; most C2 domains are seen by one or very few hosts.
- Start the analytics above in alert-only mode, tune allow-lists for CDNs, security agents and telemetry services, then promote to response actions such as sinkholing.
- Rollback: keep sinkhole and blocking changes in resolver policy (for example response policy zones) so they can be withdrawn within one TTL.
Metrics:
- Percentage of endpoints whose DNS traffic reaches only approved resolvers.
- Percentage of resolver queries logged with response code and answer data.
- Mean time to detect a test tunnelling or DGA simulation in purple-team exercises.
- False positive rate of DGA and tunnelling analytics after tuning.
- Time from an unauthorised NS or MX change on an owned domain to alert.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev. 5 | SC-20 Secure Name/Address Resolution Service (Authoritative Source); SC-21 (Recursive or Caching Resolver); SC-22 Architecture and Provisioning for Name/Address Resolution Service; SI-4 System Monitoring; AU-12 Audit Record Generation | Resolver architecture, validation and logging |
| NIST CSF 2.0 | DE.CM-01 Networks and network services are monitored | DNS logging and analytics |
| MITRE ATT&CK | T1071.004, T1568.001, T1568.002, T1572, T1048.003, T1584.002, T1583.001 Acquire Infrastructure: Domains, T1590.002 | Adversary use of DNS for C2, exfiltration, infrastructure and reconnaissance |
| MITRE D3FEND | D3-DNSTA DNS Traffic Analysis; D3-DNSDL DNS Denylisting; D3-DNSAL DNS Allowlisting | Defensive techniques built on resolver data |
| MITRE CAPEC | CAPEC-142 DNS Cache Poisoning | Attack on the caching behaviour described above |
Common failure patterns
- Logging only at the firewall: every query appears to come from the resolver, so the infected host cannot be identified. Log at the resolver and the endpoint.
- Blocking TCP 53: large responses fail intermittently, teams add exceptions in a hurry, and the resulting rules are wider than needed.
- Treating short TTLs as malicious: analytics that alert on TTL alone drown in CDN traffic. Combine TTL with domain age, popularity and answer diversity.
- Forgetting negative and stale caches during response: a sinkholed or restored name keeps its old behaviour for hours because of RFC 2308 negative caching or serve-stale. Plan TTLs before changes and flush your own resolvers.
- Trusting names whose registration lapsed: allow-lists, CNAMEs and NS delegations that point at domains you no longer own let whoever re-registers them take over. The ForcedLeak Salesforce Agentforce incident turned on an expired domain still present in a content security policy allow-list, re-registered for about five dollars.
- Clients bypassing the resolver: applications with built-in DoH, or malware using public resolvers, skip your logging entirely unless egress is controlled.
Practitioner checklist
- Train analysts to read header flags, RCODEs, record types and TTLs from raw packets and from resolver logs.
- Route all client DNS through approved recursive resolvers and block other outbound DNS, including 853 and known DoH endpoints.
- Log queries and responses with RCODE, answers and TTL at the resolver, and DNS events with process context at the endpoint.
- Allow TCP 53 alongside UDP 53 for resolvers and authoritative servers, as RFC 7766 requires.
- Restrict recursion to internal ranges and zone transfers to named secondaries.
- Deploy DGA, tunnelling, fast flux and resolver-bypass analytics with domain popularity baselines.
- Monitor NS, MX, A and SOA SERIAL changes on every domain you own, and audit CNAMEs and delegations for lapsed targets.
- Document TTL lowering and cache flushing in DNS change and incident runbooks.
Standards and references
- IETF: RFC 1035 Domain Names, Implementation and Specification
- IETF: RFC 9499 DNS Terminology (BCP 219)
- IETF: RFC 2308 Negative Caching of DNS Queries
- IETF: RFC 7766 DNS Transport over TCP, Implementation Requirements
- IETF: RFC 6891 Extension Mechanisms for DNS (EDNS(0))
- IETF: RFC 8767 Serving Stale Data to Improve DNS Resiliency
- IETF: RFC 9156 DNS Query Name Minimisation to Improve Privacy
- MITRE ATT&CK: Application Layer Protocol: DNS
Related NHI Mgmt Group resources: DNS Security Guide · NIST SP 800-81 Rev. 3 Secure DNS Deployment Guide · Protective DNS Guide · Command and Control (C2) Guide · DNS Logging and Detection Guide