Detection Engineering Guide: Detection as Code, Sigma and Coverage

Most security teams own hundreds of detection rules but cannot say which of them work, which attack techniques they actually cover, or why a given rule fires fifty times a day. Detection engineering treats detections as software: each one has a documented purpose, a tested implementation, an owner, measurable quality and a planned retirement. This guide covers the detection lifecycle, documentation standards, writing and converting Sigma rules, detection as code pipelines, testing and validation, coverage measurement against MITRE ATT&CK and D3FEND, tuning and alert quality metrics, and detections for service accounts, API keys and workload identities. It is written for detection engineers, SOC leads and security architects; platform selection and log source onboarding are covered in the SIEM Guide.

Key takeaways

• A detection is a product, not a query. It needs a goal, data requirements, logic, known blind spots, false positive expectations, a response plan and an owner.
• Write once, convert many times. Sigma rules in YAML, converted with sigma-cli and pySigma pipelines, keep content portable across SIEM platforms and migrations.
• Every change goes through a pipeline. Version control, peer review, schema validation, conversion, automated tests and staged deployment make rollback a revert.
• Coverage means tested, not mapped. A technique counts as covered only when a rule exists, its data source is healthy and a test proves it fires.
• Measure alert quality per rule. Precision, alert volume, time to detect and time to fix noisy rules show whether the programme is improving.

The detection lifecycle

Detection work fails when it is a ticket queue of "write a rule for X". A lifecycle with explicit exit criteria for each stage keeps the detection backlog honest and stops untested rules reaching analysts.

Prioritise the backlog by asking which techniques are used against organisations like yours, which assets they would reach, and whether you already have data. Threat intelligence is the main input here: see the Threat Intelligence Guide.

Choosing the detection logic

• Atomic or signature rules: one event matches known-bad values (a log cleared, a specific process and argument). Cheap and precise, but easy to evade by changing one value.
• Behavioural rules: combinations of fields that describe a technique rather than a tool, such as an office application spawning a script interpreter. More robust, but need careful exclusions.
• Threshold and correlation rules: counts or sequences over a time window, such as many failed sign-ins followed by a success. Depend on consistent timestamps and fields across sources.
• Anomaly and baseline rules: deviation from an entity's normal behaviour. Strong for non-human identities with regular patterns, weak for users whose behaviour varies, and they need a stable learning period.

Prefer behavioural logic for the core of a rule set, and use atomic rules for high-confidence indicators and emerging threats with a planned expiry.

Documenting detections

Detection documentation is what lets an analyst at 3am understand an alert, and lets an engineer a year later decide whether a rule is still needed. Palantir's Alerting and Detection Strategy (ADS) framework, "a framework for developing alerting and detection strategies for incident response", defines nine sections that remain a good template:

• Goal: the behaviour the detection is meant to find, in one or two sentences.
• Categorisation: ATT&CK tactic and technique.
• Strategy abstract: a plain description of how the detection works.
• Technical context: what an analyst needs to know about the system and the technique.
• Blind spots and assumptions: what the rule will miss, and what must be true for it to work (for example, command-line auditing enabled).
• False positives: known benign triggers.
• Validation: how to generate a true positive safely.
• Priority: severity and the reasoning for it.
• Response: the triage and containment steps.

Keep this documentation in the same repository as the rule, so a pull request that changes logic also updates the blind spots and validation steps.

Writing rules in Sigma

Sigma rules are "YAML files that contain all the information required to detect odd, bad or malicious behaviour". The SigmaHQ repository holds "more than 3000 detection rules" under the Detection Rule License 1.1, split into rules (generic, "threat agnostic" behaviour), rules-threat-hunting, rules-emerging-threats (specific campaigns, zero-days and malware), rules-compliance and rules-placeholder.

An example written for this page, detecting creation of a long-term AWS IAM access key, a common persistence step after a cloud or CI credential is stolen:

title: AWS IAM access key created
id: b603a88b-efc4-4f73-be0a-e437200d44d9
status: experimental
description: Creation of a new long-term IAM access key
references:
    - https://attack.mitre.org/techniques/T1098/001/
author: NHI Mgmt Group
date: 2026-10-16
tags:
    - attack.persistence
    - attack.t1098.001
logsource:
    product: aws
    service: cloudtrail
detection:
    selection:
        eventSource: iam.amazonaws.com
        eventName: CreateAccessKey
    condition: selection
falsepositives:
    - Approved key issuance by identity automation
level: medium

Converting rules with sigma-cli and pySigma

• Converter: sigma-cli is "the official command-line converter" (installed with pip3 install sigma-cli); pySigma is the library for building conversion into your own toolchain. Backends are installed with sigma plugin install and include Splunk, Elasticsearch, Microsoft Sentinel and Defender, Grafana Loki and CrowdStrike LogScale.
• Processing pipelines: SigmaHQ puts it simply: "Pipelines handle the mapping; log sources describe what your rules match against." Maintain one pipeline per platform that maps Sigma field names to your schema, and test it like code: a wrong field mapping produces a rule that silently never matches.
• Filters: Sigma filters let you "build lists of these exclusions without having to modify the Sigma rule itself", applied with --filter at conversion. Local tuning then survives upstream rule updates.
• Correlation rules: Sigma correlations add event_count, value_count, temporal and temporal_ordered types with group-by, timespan and a condition. The hyphen matters: SigmaHQ warns that group_by is silently ignored. Not every backend supports every type, so test the converted output.

Sigma is not the only format. Elastic's detection-rules repository, for example, stores rules as TOML with a Python CLI (validate-rule, validate-all, test), and most platforms have a native query language. Choose Sigma when portability or community content matters; write native rules when you need platform features Sigma cannot express, and record the reason.

Detection as code

Detection as code applies software practice to detection content: rules, pipelines, filters, tests and documentation live in Git, every change is a pull request, and a CI/CD pipeline validates and deploys them.

• Repository layout: one folder per platform or log source, with rules, test data and ADS documentation side by side, plus folders for pipelines and filters.
• Detection pull request review: a second engineer checks logic, field names, exclusions and documentation; a SOC analyst reviews severity and the response steps.
• Drift control: block or alert on rule changes made directly in the console, which bypass review and are lost on the next deploy.
• Rollback: revert the commit and redeploy. Keep the deploy step idempotent so rollback is routine.
• Pipeline credentials: the deploy job holds an API key or service principal that can disable every rule in the SIEM. Scope it to detection content, store it in the CI secret store and alert on its use outside the pipeline. See the CI/CD Pipeline Identity Security Guide.

Testing and validating detections

Untested detections are assumptions. Use three layers of testing, each catching different faults.

1. Unit tests on recorded events: keep at least one true positive and one near-miss true negative per rule. They run in seconds in CI and catch logic and field mapping errors.
2. Adversary emulation in a lab: Atomic Red Team provides "small and highly portable detection tests based on MITRE's ATT&CK", each mapped to a technique and runnable through the Invoke-Atomic framework. Run them only on dedicated test hosts or approved pilot systems, with change approval, recording host and time so the expected alert can be matched. Clean up after each test.
3. Production validation: scheduled, approved tests on a small ring of real hosts and cloud accounts prove the full chain: logging enabled, events forwarded, parsed, rule converted and alert routed. Pair with red team and purple team exercises for techniques that tests cannot reproduce.

Stage every new or changed rule: run it silently for one to two weeks, review its results with the SOC, then promote it. If a promoted rule floods the queue, demote it to staging rather than disabling it, and fix it through the normal pull request flow.

Measuring detection coverage

Current ATT&CK (v20) models detection directly: techniques link to Detection Strategies, which contain platform-specific analytics, the data components they need and "mutable elements" you tune per environment. For example, DET0289, the strategy for T1685.002 Disable or Modify Cloud Log, has four analytics (AN0801 to AN0804) for IaaS, identity provider, office suite and SaaS, using data components such as DC0090 Cloud Service Disable and DC0069 Cloud Service Modification, with tunable elements such as AdminRoles and RegionScope. Use these as a checklist for what your rule and data must include.

• Count procedures, not just techniques: one rule for T1059.001 PowerShell rarely covers encoded commands, download cradles and script block content together. Record which variations each rule covers.
• Weight by threat: a heatmap with every cell green is a vanity metric. Report coverage of the techniques your threat model prioritises.
• Map defences with D3FEND: MITRE D3FEND, "a knowledge graph of cybersecurity countermeasures", organises defensive techniques under Model, Harden, Detect, Isolate, Deceive, Evict and Restore. Tagging rules with Detect techniques such as D3-PA Process Analysis, D3-UBA User Behavior Analysis and D3-NTA Network Traffic Analysis shows which analytic approaches you depend on.
• Track data source health: a tested rule on a silent source is a gap. Link coverage reports to log source monitoring, and see the Windows Event Log Guide for the audit settings many endpoint rules rely on.

Tuning and alert quality metrics

Detection tuning should make rules more precise without making them blind. Exclusions belong in Sigma filters or a dedicated exclusion file with an owner, reason and expiry date, scoped to the narrowest combination of host, account, process and command line. Broad exclusions for a whole service account or subnet are where attackers hide.

Require analysts to close every alert with a disposition (true positive, benign true positive, false positive). Without it, none of these metrics can be calculated.

Where identity fits

Stolen credentials and tokens let attackers act as legitimate identities, so many high-value detections look for unusual behaviour by a known identity rather than malware. Non-human identities are attractive because their behaviour is regular, which also makes deviations easier to detect.

Build a baseline per non-human identity (source addresses, APIs called, schedule) and alert on deviation. Enrich every alert with identity context: owner, human or non-human, privilege level. See the ITDR Guide and the Service Account Security Guide.

Choosing an operating model

• Small teams: rely on vendor-maintained and SigmaHQ content, write custom rules only for your crown jewels and identity estate, and still keep exclusions and custom rules in version control.
• Large teams: a dedicated detection engineering function with a backlog, detection as code across several platforms, a test lab, and quarterly coverage reporting against the threat model.
• Managed or AI-assisted SOC: outsourcing triage or adding AI-driven alert handling does not remove the need to own detection logic, exclusions and coverage. Contract for access to rule logic and the ability to export it, and measure the provider with the same alert quality metrics.
• Vendor content versus custom: vendor rules update automatically but encode someone else's assumptions about your environment. Review them, record which you have enabled, and tune through filters so updates do not erase local changes.

Common failure patterns

• Rules on data you do not collect: a rule imported from a community repository silently never fires because the log source, audit setting or field mapping is missing.
• No detection for your own crown jewels: generic content rarely covers the specific systems that matter most. The Microsoft Storm-0558 Key Breach was first spotted by a government customer's own alerting on mailbox access audit events, not by vendor content.
• Exclusions that never expire: a temporary suppression for a migration becomes a permanent blind spot covering a privileged service account.
• Coverage claims without tests: a heatmap built from rule tags, not from passing tests, overstates what the SOC would see.
• No detection for defence impairment: ATT&CK records that Salt Typhoon "has disabled logging and disabled the distribution of logs to log aggregators"; rules that depend on those logs then go quiet. Alert on source silence and logging changes. See the Salt Typhoon Telecom Intrusions 2025.
• Console edits bypassing review: hot fixes made in the SIEM drift from the repository and are overwritten or lost.

Framework mapping

Practitioner checklist

• Keep every rule, pipeline, filter, test and ADS document in version control.
• Require peer review and automated lint, conversion and unit tests on every detection pull request.
• Store at least one true positive and one true negative sample per rule.
• Stage new rules silently before they page anyone, and record the promotion decision.
• Put exclusions in Sigma filters or an exclusion file with owner, reason and expiry.
• Report coverage by tested level against your prioritised technique list, not by rule count.
• Run approved emulation tests on dedicated hosts at least quarterly for priority techniques.
• Track precision, volume and silence per rule, and retire rules that fail review.
• Build baselines and detections for service principals, access keys and other non-human identities.
• Scope, vault and monitor the credentials your detection pipeline uses to deploy content.

Standards and references

• SigmaHQ: Rules
• SigmaHQ: Sigma rule repository
• SigmaHQ: Correlation rules
• MITRE ATT&CK: Detection Strategy for Disable or Modify Cloud Log (DET0289)
• MITRE D3FEND: A knowledge graph of cybersecurity countermeasures
• Palantir: Alerting and Detection Strategy Framework
• Red Canary: Atomic Red Team
• Elastic: Detection rules repository

Related NHI Mgmt Group resources: SIEM Guide · Threat Intelligence Guide · Incident Response Guide · ITDR Guide · Digital Forensics (DFIR) Guide