Detection coverage decays quietly: log sources stop sending, rules break after a schema change, and techniques that were covered last year no longer fire, while the SOC dashboard still looks green. This health check is a quarterly audit of detection coverage and rule health: twelve metrics with formulas, data source types and green, amber and red thresholds, plus a cadence, a findings log and a remediation map. It is run by the detection engineering lead, checked by the threat intelligence or red team function and internal audit, and signed off by the head of security operations. Day-to-day SOC performance reporting is covered in the Security Operations Metrics Guide; this page focuses on whether the detections you rely on still exist and still work.
What this health check covers
Scope is the detection programme: the log sources the SOC depends on, the detection rules and analytics built on them, their mapping to MITRE ATT&CK, and the testing that proves they fire. Two response measures, mean time to detect and mean time to respond, are included because a coverage review that ignores them can report progress while outcomes worsen. The health of the SIEM platform itself (ingestion pipeline, storage, licensing, platform access) is a product question and is not scored here; see Securing the SIEM itself.
If you searched for a SIEM rule health check, a MITRE ATT&CK coverage assessment or a set of SOC KPIs for detection, this is the run sheet. It assumes you already have a detection engineering practice as described in the Detection Engineering Guide.
Key takeaways
- Coverage is only as real as the last test: a technique counts as covered when a rule for it has fired on a test in the review window, not when a rule exists.
- Silent sources are the commonest gap: a log source that stops sending removes every detection built on it without a single error in the rule list.
- Do not chase 100% ATT&CK coverage: MITRE itself cautions against treating coverage as a checklist; weight coverage to the techniques in your threat profile.
- Rule hygiene is a metric: unowned and disabled rules are where coverage silently erodes, so measure them directly.
- Thresholds are starting points: the bands are NHI Mgmt Group recommendations; baseline against your own estate and record why you change them.
How to use this template
Copy the metrics table and the findings log into your tracker and run this SOC Detection Coverage Health Check at the cadence below. Collect each figure from the data source types listed, score it against the thresholds and record every amber or red result as a finding with an owner and a due date. Keep each completed run and compare quarter on quarter; coverage drift shows up as a trend long before it shows up in an incident. Replace bracketed placeholders such as [Organisation] and [Log source] with your own values.
Cadence and sign-off
| Review | Frequency | Owner | Approver | Evidence kept |
|---|---|---|---|---|
| Silent source check (D2) | Weekly | SOC shift lead | Detection engineering lead | Source silence periods joined to the rule-to-source mapping, tickets raised |
| Rule test run (D4) | Monthly, and after any parser or schema change | Detection engineering | Detection engineering lead | Test results per rule, failures with tickets |
| Full metric run (D1 to D12) | Quarterly | Detection engineering lead | Head of security operations | Metric workbook, ATT&CK coverage layer, findings log, sign-off record |
| Purple team validation (D12) | Quarterly or twice yearly | Red team or purple team lead | Head of security operations | Exercise plan, technique results, detection gaps raised |
| Independent re-performance | Yearly | Internal audit or second line | Audit committee | Re-performed samples, variance notes |
Metrics
NHI Mgmt Group recommended thresholds; adjust to your risk appetite. "Priority techniques" means the ATT&CK techniques in your threat profile, built from the threat actors and incidents relevant to your sector.
| ID | Metric | Formula (numerator / denominator) | Data source type | Green | Amber | Red |
|---|---|---|---|---|---|---|
| D1 | Log source coverage (by technique) | Priority techniques for which every log source needed by at least one of their detections is onboarded / priority techniques | Log source coverage matrix mapped to ATT&CK data sources; threat profile; rule repository | 95% or above | 85% to below 95% | Below 85% |
| D2 | Silent source rate (detection impact) | Enabled rules for priority techniques whose input source was silent beyond its expected interval at any time in the quarter / enabled rules for priority techniques | Per-source silence periods from source health reporting; rule-to-source mapping | 2% or below | Above 2% to 10% | Above 10% |
| D3 | ATT&CK technique coverage | Priority techniques with at least one enabled rule that passed a test in the last 6 months / priority techniques | Rule repository with ATT&CK tags; test results; threat profile | 80% or above | 60% to below 80% | Below 60% |
| D4 | Rule test pass rate | Enabled rules that fired as expected on their test case in the last test run / enabled rules with a test case | Rule test harness or emulation results | 95% or above | 85% to below 95% | Below 85% |
| D5 | Rules without owner | Enabled rules with no named owning person or team / enabled rules | Rule repository metadata | 0% | Above 0% to 5% | Above 5% |
| D6 | False positive rate | Alerts closed as false positive / alerts closed in the period; reported per rule for the top 20 rules by volume | Case management closure codes | Below 30% overall, no rule above 90% | 30% to below 60%, or any rule above 90% | 60% or above |
| D7 | MTTD | Median hours from earliest attacker activity to detection, for confirmed incidents and purple team cases in the period | Incident records with activity and detection timestamps | 24 hours or less | Above 24 to 72 hours | Above 72 hours |
| D8 | MTTR | Median hours from detection to containment, for confirmed incidents in the period | Incident records with detection and containment timestamps | 8 hours or less | Above 8 to 24 hours | Above 24 hours |
| D9 | Alert backlog age | 95th percentile age in hours of open, untriaged alerts at the time of the check | Case management or alert queue snapshot | 4 hours or less | Above 4 to 24 hours | Above 24 hours |
| D10 | Disabled rule share | Rules disabled in production / rules in the repository; with the share disabled over 30 days without a recorded reason (suppressions and allowlists are scored in the SIEM product check) | Rule repository and SIEM rule state | 5% or below, none without reason | Above 5% to 15% | Above 15%, or any priority-technique rule disabled without reason |
| D11 | Detection as code share | Production rules held in version control and deployed through a reviewed pipeline / production rules | Rule repository; SIEM rule list; deployment pipeline logs | 90% or above | 50% to below 90% | Below 50% |
| D12 | Purple team validation rate | Priority techniques exercised in a purple team or emulation exercise in the last 12 months that were detected / priority techniques exercised | Exercise reports; detection results per technique | 80% or above | 60% to below 80% | Below 60% |
Notes on the formulas
- D1 measures coverage per technique, not per system: whether every system is sending logs is a SIEM platform question; here the test is whether the sources each priority detection needs are present. Keep the mapping in the Log Source Coverage Matrix Template. NIST SP 800-92 Rev. 1, an initial public draft from October 2023, frames log management as a planning exercise; this matrix is the plan.
- D2 turns silence into lost coverage: source uptime belongs to the platform check; this metric counts the priority detections that were blind while a source was silent. Set the expected interval per source type, since a firewall that logs every second and a weekly batch job do not share a silence threshold.
- D3 is weighted to your threat profile: MITRE's guidance on using ATT&CK to assess defensive coverage cautions against aiming for 100% coverage, and against counting a technique as covered after finding one way to detect one procedure. Count only tested rules, and keep the coverage layer in a navigator-style view for the evidence pack.
- D4 and D12 are different tests: D4 replays a known test case against one rule; D12 runs an end-to-end exercise through real telemetry. A rule can pass D4 and still miss in D12 because the log source is silent or parsed differently.
- D6 is diagnostic: high false positive rates hide true positives and drive analysts to disable rules, which then shows up in D10. See the Alert Fatigue Reduction Guide.
- D7 and D8 are outcomes: report them with the incident count; with few incidents, include purple team cases and say so.
Where identity fits
Identity telemetry is where coverage gaps are hardest to see. Service accounts, OAuth apps and API keys rarely trigger user-behaviour rules, so include identity provider sign-in and audit logs, cloud control plane logs and secret store access logs in D1, and tag non-human identity techniques such as T1078.004 Cloud Accounts and T1528 Steal Application Access Token in the threat profile behind D3. The Detection Engineering for Non-Human Identities Guide lists the analytics.
How to collect the data
- Freeze the threat profile: agree the priority technique list with threat intelligence before the run, so D3 and D12 have a fixed denominator.
- Export the rule repository: every rule with state (enabled, disabled, suppressed), owner, ATT&CK tags, last modified date, test case reference and whether it is deployed from version control.
- Export source health: last event time and daily volume per log source from the SIEM or data lake, joined to the required source list.
- Run or collect tests: the latest rule test harness results (D4) and purple team or emulation reports for the last 12 months (D12). The Purple Team Exercise Plan Template gives a format.
- Export case data: alert closure codes, queue snapshot and incident timestamps for D6 to D9.
- Sample: pick five rules marked as covering priority techniques and confirm each fired on its test and maps to the stated technique.
Red flags that need action now
- Any rule for a priority technique blind for more than 24 hours because its identity provider, EDR or cloud control plane source was silent (D2).
- A rule covering a priority technique disabled or suppressed with no recorded reason (D10).
- Rules failing tests after a parser, schema or agent upgrade, with no ticket raised (D4).
- Coverage reported from rule tags alone, with no test evidence (D3).
- Untriaged alerts older than 24 hours in the queue (D9).
- A purple team exercise in which no priority technique was detected (D12).
- Production rules edited directly in the console with no review trail (D11).
Findings log
| Finding ID | Finding | Metric | Severity | Owner | Due date | Status |
|---|---|---|---|---|---|---|
| HC-DET-01 | [Log source] silent for [hours] | D2 | [High / Medium / Low] | [Role] | [Date] | [Open / In progress / Closed / Risk accepted] |
| HC-DET-02 | [Number] priority techniques with no tested rule | D3 | [Severity] | [Role] | [Date] | [Status] |
| HC-DET-03 | [Number] rules failing tests after [change] | D4 | [Severity] | [Role] | [Date] | [Status] |
| HC-DET-04 | [Number] rules with no owner | D5 | [Severity] | [Role] | [Date] | [Status] |
| HC-DET-05 | [Finding] | [Metric] | [Severity] | [Role] | [Date] | [Status] |
Group findings with one root cause, such as a parser change that broke several rules, into one finding. Risk accepted findings carry an expiry date and the approver's name and are re-opened at the next quarterly run.
Remediation map
| Finding type | Metrics | Where the fix is explained |
|---|---|---|
| Missing or silent log sources | D1, D2 | SIEM Log Source Onboarding Checklist; Log Source Coverage Matrix Template |
| Coverage gaps and untested rules | D3, D4 | Detection Engineering Guide; Sigma Rules Guide |
| Rule hygiene and change control | D5, D10, D11 | Detection as Code Guide |
| Noisy rules and backlog | D6, D9 | Alert Fatigue Reduction Guide |
| Slow detection and response | D7, D8 | Security Operations Metrics Guide |
| Detections missed in exercises | D12 | Purple Teaming Guide; Breach and Attack Simulation (BAS) Guide |
Related guides
- Detection Engineering Guide: the practice this health check audits.
- Detection Engineering Maturity Model: map the results to a level.
- Security Operations Metrics Guide: the wider SOC metric catalogue.
- If your SIEM is compromised: when a coverage gap is caused by tampering.
- Purple Teaming Guide: how to run the exercises behind D12.
References
Framework anchors for the metrics: NIST SP 800-53 AU-2 (Event Logging), AU-6 (Audit Record Review, Analysis, and Reporting), AU-12 (Audit Record Generation), SI-4 (System Monitoring) and CA-7 (Continuous Monitoring); NIST CSF 2.0 DE.CM-01, DE.CM-09, DE.AE-02 and ID.IM-02; CIS Controls v8.1 Control 8 (Audit Log Management).
- MITRE ATT&CK: Resources (assessment, engineering and Navigator)
- NIST: SP 800-92 Rev. 1 Cybersecurity Log Management Planning Guide (initial public draft)
- NIST: SP 800-53 Rev. 5
- NIST: Cybersecurity Framework 2.0
- CISA and partners: Best Practices for Event Logging and Threat Detection
- CIS: Control 8 Audit Log Management
Related NHI Mgmt Group resources: Detection Engineering Guide, Detection Engineering Maturity Model, Security Operations Metrics Guide, Log Source Coverage Matrix Template.