Join our Newsletter — 33% off our NHI Course
Home› Guides› Detection as Code Guide: version control, CI testing…
Guide SOC & Detection Engineering

Detection as Code Guide: version control, CI testing and safe deployment of detections

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 27 October 2026 13 min read
On this page

Detections edited by hand in a SIEM console break silently, cannot be reviewed and cannot be rolled back when a change floods the queue. Detection as code (DaC) applies software engineering practice to detection content: rules live in version control, every change goes through review and automated tests, and a pipeline deploys them to the SIEM, EDR or data lake. This guide covers repository design and rule formats, the CI pipeline stages and test types, deployment and rollback, securing the pipeline itself, decision guidance for different team sizes, and how to measure whether it is working. It is written for detection engineers, SOC leads and platform teams who already write detections and want them to be reliable at scale. For the wider discipline see the Detection Engineering Guide.

Key takeaways

  • The repository is the source of truth. Microsoft's Sentinel documentation warns that edits made in the console rather than the repository may be overwritten; the same principle applies to every platform.
  • Test at several levels. Schema and syntax checks, query validation against the target platform, unit tests with sample events, and true positive replay each catch different failures.
  • Document with the code. Pair each rule with an ADS style record (goal, blind spots, false positives, validation, response) so review covers intent as well as syntax.
  • The pipeline is a privileged identity. Whoever controls it can disable your detections; protect it with branch protection, review rules and short-lived credentials.
  • Deletion and drift need handling. Removing a file from the repository does not always remove the rule from the platform, so pipelines must reconcile state.

What detection as code is, and is not

Elastic's detections as code reference describes four core components: managing rules in version control, syncing from version control to the security platform, managing rules in the platform, and syncing changes from the platform back to version control. Its testing vocabulary includes unit testing, rule schema validation, detection logic validation and pre-production testing. Elastic is explicit that DaC is not a single tool but an approach applying software development practice to the creation, management and deployment of rules.

  • It is: version control, peer review through pull requests, automated tests, repeatable deployment, change history, and the ability to revert.
  • It is not: a library of community rules you import wholesale, a replacement for threat-informed prioritisation, or a reason to skip tuning in your own environment.
  • Relationship to Sigma: Sigma is a vendor-neutral rule format. DaC can use Sigma as the authoring format and convert it at build time, or use the platform's native format directly. See the Sigma Rules Guide.

Repository design and rule formats

Format or toolingAuthoring formatBuild and test support (from project docs)Deploys to
Sigma with pySigma and sigma-cliYAML per the Sigma rules specification (v2.1.0, August 2025)pySigma parses and converts rules through backends and processing pipelines; sigma convert -t <backend> -p <pipeline> can write one output file per ruleAny platform with a backend (SIEM query languages)
Elastic detection-rulesTOML files per rulePython module for parsing, validating and packaging; unit tests; KQL parser; import and export commands to sync with KibanaElastic Security
Splunk security_content with contentctlYAML detections and analytic storiescontentctl builds, tests and packages content; Attack Range and attack_data supply test telemetry; pre-commit hooks for formattingSplunk Enterprise Security
Microsoft Sentinel repositoriesARM templates or Bicep (Bicep needs a connection created after 1 November 2024)Generated GitHub Actions workflow or Azure DevOps pipeline; smart deployments push only content changed since the last deploymentSentinel analytics rules, automation rules, hunting queries, parsers, playbooks, workbooks
Native API or TerraformPlatform JSON or HCLYour own CI testsMost SIEM and EDR platforms

A workable layout separates rule logic, documentation, tests and pipeline code:

  • rules/<platform-or-logsource>/: one file per detection. Follow Sigma's file naming conventions even for native formats: lowercase, underscores, a log source prefix, 10 to 70 characters.
  • docs/: one record per rule using the Detection Rule Documentation Template, linked by rule ID.
  • tests/: sample positive and negative events per rule, plus expected results.
  • exceptions/: allowlists and suppressions as reviewed code, with an owner and expiry date on each entry.
  • pipelines/ and .github/ or equivalent: build, test and deploy definitions, protected by code owners.

Metadata should be machine readable so the pipeline can enforce it: unique ID, status (Sigma's experimental, test, stable, deprecated, unsupported), level, ATT&CK tags such as attack.t1003.001, owner, data source dependencies, and links between rules through Sigma's related types (derived, obsolete, merged, renamed, similar).

Building blocks and correlations: keep enrichment and intermediate rules as first-class content. Elastic's repository holds building block rules in their own directory, and Sigma defines an informational level for rules intended to tag events rather than raise cases, plus a separate correlation format for rules that combine others. Test dependencies explicitly: when a building block changes, the pipeline should rerun the tests of every rule that consumes it, because a renamed tag or field in one rule can silently break several detections downstream.

Parsers and log source mappings: version parsers, field mappings and processing pipelines in the same repository, since a parser change can break more detections than any single rule edit.

The CI pipeline: stages and tests

StageWhat it checksFails the build when
1. Lint and schemaFile format, required fields, naming, YAML style (Sigma: UTF-8, LF, 4-space indentation, lowercase keys)Missing ID, owner, ATT&CK tag, status or documentation link
2. Conversion and query validationRule converts for each target backend and the resulting query parses on the platformConversion error, unsupported modifier, query syntax error
3. Field and data model checkFields exist in the target schema or processing pipeline mappingRule references a field the log source does not produce
4. Unit testsRule logic against stored positive and negative sample eventsPositive sample not matched, negative sample matched
5. BacktestRun against a recent window of production data in a non-alerting modeProjected alert volume above threshold for the rule's level
6. True positive replayEmulated attack or recorded telemetry (for example Atomic Red Team tests or attack data sets) produces the alert end to endNo alert within the expected time
7. Peer reviewLogic, documentation, blind spots, response guidanceReviewer not from code owners, or record incomplete
8. Staged deploymentDeploy to a test workspace or as non-alerting, then promoteErrors on deploy, or health check fails after promotion

Stages 1 to 4 should run on every pull request and finish in minutes. Stages 5 and 6 can run nightly or before promotion from test to stable. Palantir's ADS framework makes true positive validation mandatory and states that a rule which cannot be made to fire, or cannot reach a low false positive rate, may need to be broken down, refactored or discarded. For test design in depth see the Detection Validation and Testing Guide.

Deployment, drift and rollback

  1. Environments: keep at least a test target (separate workspace, index or tenant, or the production platform in a non-alerting mode) and production. Promote by merging to a protected branch.
  2. Incremental deployment: deploy only changed content where the platform supports it. Sentinel's smart deployments do this by default based on commits since the last deployment.
  3. Deletion and retirement: handle removals explicitly. Microsoft documents that deleting content from a connected repository does not delete it from the Sentinel workspace, and that removing a connection leaves deployed content in place. Mark rules deprecated, have the pipeline disable them, then delete after a set period.
  4. Drift detection: export the platform's rule state on a schedule and compare with the repository. Console edits during an incident are sometimes necessary; the pipeline should flag them and require a follow-up pull request. Elastic's tooling provides export commands for this direction.
  5. Rollback: revert the commit and redeploy. Keep the deploy job idempotent so a revert restores the exact previous state. Practise it: a rollback that has never been tested is a hope.
  6. Version compatibility: pin tool and backend versions in the pipeline. Elastic warns that it does not guarantee backward compatibility of rule schemas across versions, so upgrades need their own test run.

Securing the pipeline: where identity fits

The detection pipeline holds write access to your security monitoring. An attacker who can merge to it, or steal its credentials, can disable or weaken detections, which ATT&CK v20 places under the Defense Impairment tactic (TA0112), for example T1685 Disable or Modify Tools. Treat the pipeline as a privileged non-human identity.

  • Short-lived credentials: use workload identity federation instead of stored secrets. GitHub documents that with OpenID Connect a workflow receives a short-lived token valid for a single job, and that the cloud role can require claims such as repository, environment and branch to match.
  • Least privilege: the deploy identity needs rights to manage detection content, not tenant administration. Sentinel's repositories feature requires the Owner role on the resource group to create the connection; scope that resource group tightly and keep the human Owner role separate from day-to-day work.
  • Branch protection and code owners: require review by a detection owner for rules/ and exceptions/, and by a platform owner for pipeline files. Block force pushes and self-approval.
  • Exception review: suppressions are the easiest way to blind a rule. Require an owner, a reason and an expiry for each, and alert on bulk changes.
  • Audit: forward repository and pipeline audit logs to the SIEM and detect disabled rules, deleted rules and exceptions added outside the pipeline.

See the CI/CD Pipeline Identity Security Guide and the GitHub and GitHub Actions Security Guide.

Implementing detection as code

  1. Export what you have. Pull every existing rule from the platform into a repository as-is. Record owner, status and last true positive for each. This alone gives you history and backup.
  2. Add lint and schema checks. Enforce required metadata on new and changed rules only, so the team is not blocked by legacy debt.
  3. Pilot with one log source. Pick a high-value area such as identity provider sign-ins or endpoint process creation, add unit tests and documentation for 10 to 20 rules, and deploy them through the pipeline to a test target.
  4. Switch production deployment to the pipeline. Lock console editing for pipeline-managed rules where the platform allows, or at least detect drift.
  5. Add backtesting and true positive replay. Start with critical and high rules.
  6. Expand and retire. Migrate remaining rules in batches; deprecate those with no owner, no threat justification and no true positives.
  7. Rollback plan throughout: until the pipeline is trusted, keep the platform's own export of the previous rule set so you can restore it independently of the pipeline.

Decision guidance

DecisionOption AOption BChoose A when
Authoring formatSigma, converted at buildPlatform nativeYou run more than one platform, share rules with partners or expect to migrate SIEM; choose native when you depend on features Sigma cannot express or a backend does not support
Pipeline toolingVendor DaC tooling (Elastic detection-rules, Splunk contentctl, Sentinel repositories)Your own scripts against the platform APIYou are on one platform and want tested building blocks; choose your own when spanning several platforms
Test dataRecorded attack telemetryLive emulation in a labYou need fast, repeatable unit tests; add live emulation for end-to-end validation including the log pipeline
Deployment triggerMerge to mainScheduled releaseSmall team with good tests; scheduled releases suit large teams with change boards
  • Small teams: a repository, a lint step, peer review and scripted deploy are enough to start. The biggest gain is history and rollback.
  • Large teams and MSSPs: multi-tenant deployment with per-tenant exceptions as code, release trains, coverage dashboards and formal separation between authors and approvers.

Measuring detection as code

  • Pipeline coverage: percentage of production detections deployed from the repository (target 100 per cent).
  • Test coverage: percentage of rules with unit tests, and percentage with a true positive replay in the last 90 days.
  • Change failure rate: deployments that were reverted or caused an alert flood or a broken rule.
  • Lead time for a detection: from pull request opened to production, by priority.
  • Drift events: console changes detected per month, and time to reconcile.
  • Silent failures caught: rules found broken by scheduled tests before an incident exposed them.

Framework mapping

FrameworkReferenceRelevance
NIST CSF 2.0PR.PS-01, DE.AE-02, ID.IM-02Configuration management practice applied to detection content; analysis of adverse events; improvements from tests
NIST SP 800-53 Rev. 5CM-3, CM-5, SA-11, SI-4, CA-7Change control, access restrictions for change, developer testing, system monitoring, continuous monitoring
ISO/IEC 27001:2022Annex A 8.16, 8.32Monitoring activities, change management
MITRE ATT&CK v20Technique tags; TA0112 Defense Impairment, T1685Coverage measurement; threats to the pipeline itself
NCSC CAFC1.c Generating alertsDetection rules tested and tuned to reduce false positives

Common failure patterns

  • Repository and platform diverge: analysts tune in the console during an incident and the next deployment silently overwrites the fix, or the repository is never updated.
  • Tests that only prove syntax: rules that convert and parse but never match real telemetry because a field name differs in production.
  • Deleted rules that keep running: removal from the repository not reconciled on the platform, leaving noisy or obsolete rules alive.
  • Unreviewed exceptions: suppressions added as quick fixes with no owner or expiry, gradually blinding critical rules.
  • Over-privileged pipeline credentials: a long-lived secret with tenant admin rights stored in CI, giving anyone who compromises the pipeline control of monitoring.
  • Bulk import without ownership: thousands of community rules imported at once, none tuned or owned, producing alert fatigue that hides real incidents.

Practitioner checklist

  • Export every production detection into version control and assign an owner.
  • Enforce required metadata (ID, status, level, ATT&CK tags, owner, data source, documentation link) in CI.
  • Add positive and negative unit test events for every critical and high rule.
  • Validate conversion and query syntax against each target platform on every pull request.
  • Backtest new and changed rules against recent production data before promotion.
  • Run true positive replay on a schedule and after log source changes.
  • Manage exceptions as code with owner, reason and expiry.
  • Deploy with a short-lived, least-privilege pipeline identity under branch protection and code owners.
  • Detect and reconcile drift and deletions between repository and platform.
  • Practise rollback by reverting a commit in the test environment each quarter.

Standards and references

Related NHI Mgmt Group resources: Detection Engineering Guide · Sigma Rules Guide · Detection Engineering Maturity Model · SIEM Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 27 October 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.