Detections edited by hand in a SIEM console break silently, cannot be reviewed and cannot be rolled back when a change floods the queue. Detection as code (DaC) applies software engineering practice to detection content: rules live in version control, every change goes through review and automated tests, and a pipeline deploys them to the SIEM, EDR or data lake. This guide covers repository design and rule formats, the CI pipeline stages and test types, deployment and rollback, securing the pipeline itself, decision guidance for different team sizes, and how to measure whether it is working. It is written for detection engineers, SOC leads and platform teams who already write detections and want them to be reliable at scale. For the wider discipline see the Detection Engineering Guide.
Key takeaways
- The repository is the source of truth. Microsoft's Sentinel documentation warns that edits made in the console rather than the repository may be overwritten; the same principle applies to every platform.
- Test at several levels. Schema and syntax checks, query validation against the target platform, unit tests with sample events, and true positive replay each catch different failures.
- Document with the code. Pair each rule with an ADS style record (goal, blind spots, false positives, validation, response) so review covers intent as well as syntax.
- The pipeline is a privileged identity. Whoever controls it can disable your detections; protect it with branch protection, review rules and short-lived credentials.
- Deletion and drift need handling. Removing a file from the repository does not always remove the rule from the platform, so pipelines must reconcile state.
What detection as code is, and is not
Elastic's detections as code reference describes four core components: managing rules in version control, syncing from version control to the security platform, managing rules in the platform, and syncing changes from the platform back to version control. Its testing vocabulary includes unit testing, rule schema validation, detection logic validation and pre-production testing. Elastic is explicit that DaC is not a single tool but an approach applying software development practice to the creation, management and deployment of rules.
- It is: version control, peer review through pull requests, automated tests, repeatable deployment, change history, and the ability to revert.
- It is not: a library of community rules you import wholesale, a replacement for threat-informed prioritisation, or a reason to skip tuning in your own environment.
- Relationship to Sigma: Sigma is a vendor-neutral rule format. DaC can use Sigma as the authoring format and convert it at build time, or use the platform's native format directly. See the Sigma Rules Guide.
Repository design and rule formats
| Format or tooling | Authoring format | Build and test support (from project docs) | Deploys to |
|---|---|---|---|
| Sigma with pySigma and sigma-cli | YAML per the Sigma rules specification (v2.1.0, August 2025) | pySigma parses and converts rules through backends and processing pipelines; sigma convert -t <backend> -p <pipeline> can write one output file per rule | Any platform with a backend (SIEM query languages) |
| Elastic detection-rules | TOML files per rule | Python module for parsing, validating and packaging; unit tests; KQL parser; import and export commands to sync with Kibana | Elastic Security |
| Splunk security_content with contentctl | YAML detections and analytic stories | contentctl builds, tests and packages content; Attack Range and attack_data supply test telemetry; pre-commit hooks for formatting | Splunk Enterprise Security |
| Microsoft Sentinel repositories | ARM templates or Bicep (Bicep needs a connection created after 1 November 2024) | Generated GitHub Actions workflow or Azure DevOps pipeline; smart deployments push only content changed since the last deployment | Sentinel analytics rules, automation rules, hunting queries, parsers, playbooks, workbooks |
| Native API or Terraform | Platform JSON or HCL | Your own CI tests | Most SIEM and EDR platforms |
A workable layout separates rule logic, documentation, tests and pipeline code:
rules/<platform-or-logsource>/: one file per detection. Follow Sigma's file naming conventions even for native formats: lowercase, underscores, a log source prefix, 10 to 70 characters.docs/: one record per rule using the Detection Rule Documentation Template, linked by rule ID.tests/: sample positive and negative events per rule, plus expected results.exceptions/: allowlists and suppressions as reviewed code, with an owner and expiry date on each entry.pipelines/and.github/or equivalent: build, test and deploy definitions, protected by code owners.
Metadata should be machine readable so the pipeline can enforce it: unique ID, status (Sigma's experimental, test, stable, deprecated, unsupported), level, ATT&CK tags such as attack.t1003.001, owner, data source dependencies, and links between rules through Sigma's related types (derived, obsolete, merged, renamed, similar).
Building blocks and correlations: keep enrichment and intermediate rules as first-class content. Elastic's repository holds building block rules in their own directory, and Sigma defines an informational level for rules intended to tag events rather than raise cases, plus a separate correlation format for rules that combine others. Test dependencies explicitly: when a building block changes, the pipeline should rerun the tests of every rule that consumes it, because a renamed tag or field in one rule can silently break several detections downstream.
Parsers and log source mappings: version parsers, field mappings and processing pipelines in the same repository, since a parser change can break more detections than any single rule edit.
The CI pipeline: stages and tests
| Stage | What it checks | Fails the build when |
|---|---|---|
| 1. Lint and schema | File format, required fields, naming, YAML style (Sigma: UTF-8, LF, 4-space indentation, lowercase keys) | Missing ID, owner, ATT&CK tag, status or documentation link |
| 2. Conversion and query validation | Rule converts for each target backend and the resulting query parses on the platform | Conversion error, unsupported modifier, query syntax error |
| 3. Field and data model check | Fields exist in the target schema or processing pipeline mapping | Rule references a field the log source does not produce |
| 4. Unit tests | Rule logic against stored positive and negative sample events | Positive sample not matched, negative sample matched |
| 5. Backtest | Run against a recent window of production data in a non-alerting mode | Projected alert volume above threshold for the rule's level |
| 6. True positive replay | Emulated attack or recorded telemetry (for example Atomic Red Team tests or attack data sets) produces the alert end to end | No alert within the expected time |
| 7. Peer review | Logic, documentation, blind spots, response guidance | Reviewer not from code owners, or record incomplete |
| 8. Staged deployment | Deploy to a test workspace or as non-alerting, then promote | Errors on deploy, or health check fails after promotion |
Stages 1 to 4 should run on every pull request and finish in minutes. Stages 5 and 6 can run nightly or before promotion from test to stable. Palantir's ADS framework makes true positive validation mandatory and states that a rule which cannot be made to fire, or cannot reach a low false positive rate, may need to be broken down, refactored or discarded. For test design in depth see the Detection Validation and Testing Guide.
Deployment, drift and rollback
- Environments: keep at least a test target (separate workspace, index or tenant, or the production platform in a non-alerting mode) and production. Promote by merging to a protected branch.
- Incremental deployment: deploy only changed content where the platform supports it. Sentinel's smart deployments do this by default based on commits since the last deployment.
- Deletion and retirement: handle removals explicitly. Microsoft documents that deleting content from a connected repository does not delete it from the Sentinel workspace, and that removing a connection leaves deployed content in place. Mark rules
deprecated, have the pipeline disable them, then delete after a set period. - Drift detection: export the platform's rule state on a schedule and compare with the repository. Console edits during an incident are sometimes necessary; the pipeline should flag them and require a follow-up pull request. Elastic's tooling provides export commands for this direction.
- Rollback: revert the commit and redeploy. Keep the deploy job idempotent so a revert restores the exact previous state. Practise it: a rollback that has never been tested is a hope.
- Version compatibility: pin tool and backend versions in the pipeline. Elastic warns that it does not guarantee backward compatibility of rule schemas across versions, so upgrades need their own test run.
Securing the pipeline: where identity fits
The detection pipeline holds write access to your security monitoring. An attacker who can merge to it, or steal its credentials, can disable or weaken detections, which ATT&CK v20 places under the Defense Impairment tactic (TA0112), for example T1685 Disable or Modify Tools. Treat the pipeline as a privileged non-human identity.
- Short-lived credentials: use workload identity federation instead of stored secrets. GitHub documents that with OpenID Connect a workflow receives a short-lived token valid for a single job, and that the cloud role can require claims such as repository, environment and branch to match.
- Least privilege: the deploy identity needs rights to manage detection content, not tenant administration. Sentinel's repositories feature requires the Owner role on the resource group to create the connection; scope that resource group tightly and keep the human Owner role separate from day-to-day work.
- Branch protection and code owners: require review by a detection owner for
rules/andexceptions/, and by a platform owner for pipeline files. Block force pushes and self-approval. - Exception review: suppressions are the easiest way to blind a rule. Require an owner, a reason and an expiry for each, and alert on bulk changes.
- Audit: forward repository and pipeline audit logs to the SIEM and detect disabled rules, deleted rules and exceptions added outside the pipeline.
See the CI/CD Pipeline Identity Security Guide and the GitHub and GitHub Actions Security Guide.
Implementing detection as code
- Export what you have. Pull every existing rule from the platform into a repository as-is. Record owner, status and last true positive for each. This alone gives you history and backup.
- Add lint and schema checks. Enforce required metadata on new and changed rules only, so the team is not blocked by legacy debt.
- Pilot with one log source. Pick a high-value area such as identity provider sign-ins or endpoint process creation, add unit tests and documentation for 10 to 20 rules, and deploy them through the pipeline to a test target.
- Switch production deployment to the pipeline. Lock console editing for pipeline-managed rules where the platform allows, or at least detect drift.
- Add backtesting and true positive replay. Start with critical and high rules.
- Expand and retire. Migrate remaining rules in batches; deprecate those with no owner, no threat justification and no true positives.
- Rollback plan throughout: until the pipeline is trusted, keep the platform's own export of the previous rule set so you can restore it independently of the pipeline.
Decision guidance
| Decision | Option A | Option B | Choose A when |
|---|---|---|---|
| Authoring format | Sigma, converted at build | Platform native | You run more than one platform, share rules with partners or expect to migrate SIEM; choose native when you depend on features Sigma cannot express or a backend does not support |
| Pipeline tooling | Vendor DaC tooling (Elastic detection-rules, Splunk contentctl, Sentinel repositories) | Your own scripts against the platform API | You are on one platform and want tested building blocks; choose your own when spanning several platforms |
| Test data | Recorded attack telemetry | Live emulation in a lab | You need fast, repeatable unit tests; add live emulation for end-to-end validation including the log pipeline |
| Deployment trigger | Merge to main | Scheduled release | Small team with good tests; scheduled releases suit large teams with change boards |
- Small teams: a repository, a lint step, peer review and scripted deploy are enough to start. The biggest gain is history and rollback.
- Large teams and MSSPs: multi-tenant deployment with per-tenant exceptions as code, release trains, coverage dashboards and formal separation between authors and approvers.
Measuring detection as code
- Pipeline coverage: percentage of production detections deployed from the repository (target 100 per cent).
- Test coverage: percentage of rules with unit tests, and percentage with a true positive replay in the last 90 days.
- Change failure rate: deployments that were reverted or caused an alert flood or a broken rule.
- Lead time for a detection: from pull request opened to production, by priority.
- Drift events: console changes detected per month, and time to reconcile.
- Silent failures caught: rules found broken by scheduled tests before an incident exposed them.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.PS-01, DE.AE-02, ID.IM-02 | Configuration management practice applied to detection content; analysis of adverse events; improvements from tests |
| NIST SP 800-53 Rev. 5 | CM-3, CM-5, SA-11, SI-4, CA-7 | Change control, access restrictions for change, developer testing, system monitoring, continuous monitoring |
| ISO/IEC 27001:2022 | Annex A 8.16, 8.32 | Monitoring activities, change management |
| MITRE ATT&CK v20 | Technique tags; TA0112 Defense Impairment, T1685 | Coverage measurement; threats to the pipeline itself |
| NCSC CAF | C1.c Generating alerts | Detection rules tested and tuned to reduce false positives |
Common failure patterns
- Repository and platform diverge: analysts tune in the console during an incident and the next deployment silently overwrites the fix, or the repository is never updated.
- Tests that only prove syntax: rules that convert and parse but never match real telemetry because a field name differs in production.
- Deleted rules that keep running: removal from the repository not reconciled on the platform, leaving noisy or obsolete rules alive.
- Unreviewed exceptions: suppressions added as quick fixes with no owner or expiry, gradually blinding critical rules.
- Over-privileged pipeline credentials: a long-lived secret with tenant admin rights stored in CI, giving anyone who compromises the pipeline control of monitoring.
- Bulk import without ownership: thousands of community rules imported at once, none tuned or owned, producing alert fatigue that hides real incidents.
Practitioner checklist
- Export every production detection into version control and assign an owner.
- Enforce required metadata (ID, status, level, ATT&CK tags, owner, data source, documentation link) in CI.
- Add positive and negative unit test events for every critical and high rule.
- Validate conversion and query syntax against each target platform on every pull request.
- Backtest new and changed rules against recent production data before promotion.
- Run true positive replay on a schedule and after log source changes.
- Manage exceptions as code with owner, reason and expiry.
- Deploy with a short-lived, least-privilege pipeline identity under branch protection and code owners.
- Detect and reconcile drift and deletions between repository and platform.
- Practise rollback by reverting a commit in the test environment each quarter.
Standards and references
- SigmaHQ: Sigma Rules Specification
- SigmaHQ: pySigma
- Elastic: Detections as Code Reference
- Elastic: detection-rules repository
- Splunk: Security Content and contentctl
- Microsoft Learn: Manage custom content with Microsoft Sentinel repositories
- Palantir: Alerting and Detection Strategy Framework
- GitHub Docs: OpenID Connect
Related NHI Mgmt Group resources: Detection Engineering Guide · Sigma Rules Guide · Detection Engineering Maturity Model · SIEM Guide