What happens when SOC teams automate detections before they fix data quality?

Why Detection Automation Breaks First When Telemetry Is Unreliable

Automation only improves SOC output when the data underneath it is already trustworthy. If alert logic, correlation rules, or detection-as-code pipelines are built on incomplete fields, inconsistent labels, duplicate events, or weak enrichment, the SOC does not become more accurate, it becomes more efficient at repeating the same errors. That creates operational drag, alert fatigue, and misplaced confidence in coverage. The NIST Cybersecurity Framework 2.0 is useful here because it frames detection as part of a broader security outcome, not as a substitute for the quality of the underlying control environment.

Teams often assume that better orchestration will compensate for poor telemetry, but in practice the first failure is usually not the rule logic itself. It is the data pipeline that feeds it: collection gaps, broken parsers, schema drift, and inconsistent asset or identity context make the automation look functional while silently degrading the decisions it produces.

How Detection-as-Code Inherits Data Problems

Detection engineering depends on stable, normalised inputs. When ingestion varies by source, fields are renamed without notice, timestamps are inconsistent, or enrichment fails to resolve the right asset, user, or process context, the rule may still fire but for the wrong reason. That means automation can scale three common problems at once: false positives, missed detections, and triage paths that waste analyst time on low-value noise.

Good detection automation usually assumes a chain that is already working:

• events are collected consistently from the right sources;
• records are parsed into a common schema;
• enrichment adds reliable context such as host, user, application, or environment;
• normalisation preserves meaning across tools and log types;
• rule logic is validated against known-good and known-bad examples.

When any of those steps are weak, automation can still appear successful because the pipeline runs and alerts are produced. The practical issue is that the SOC now trusts output that was never validated against clean data. NIST SP 800-53 Rev. 5 Security and Privacy Controls is relevant because it reinforces the need for logging, monitoring, and system integrity controls that make telemetry dependable before higher-order automation depends on it.

In operational terms, the sequence matters. SOC teams should first confirm that the most important data sources are complete, normalised, and observable enough to support decision-making, then automate the detections that depend on those sources. If they reverse that order, they usually end up tuning around noise instead of fixing the cause. Where telemetry comes from multiple platforms, the hardest failures are often silent schema changes and enrichment gaps that do not stop the pipeline but do break confidence in the result.

When the Shortcut Becomes a Control Gap

Stabilising data quality before automation creates overhead, but that tradeoff is real and necessary. Faster detection engineering is valuable only when the underlying signal is already dependable, otherwise the organisation simply industrialises poor judgement. The common industry disagreement is not whether automation is useful, but whether teams can safely automate around known data defects while they continue to improve the pipeline. Our view is that limited exceptions can be acceptable for low-risk use cases, but not for detections that drive escalation, containment, or reporting decisions.

The edge case is mature environments with tightly governed telemetry and a narrow set of high-confidence detections. In those cases, teams may automate some workflows while still hardening data quality in parallel. The risk is scale: once the same logic is copied across additional sources, products, or business units, small data inconsistencies become systemic misclassification. The ENISA Threat Landscape is a useful external reference when teams need a broader view of how visibility gaps and operational weaknesses can compound into larger defensive blind spots.

For practitioner judgement, the key distinction is whether the automation is merely speeding up a known-good process or amplifying an untrusted one. The first is efficiency; the second is technical debt that now makes decisions for the SOC.

Risk and Threat Considerations

Poor telemetry quality creates a material operational and security risk because automated detections tend to inherit the same blind spots, bias, and context loss at machine speed. The result is not just noisy alerts, but weakened visibility into real attacker activity, especially when the environment depends on enrichment to distinguish normal from malicious behaviour.

Failure mechanism: Incomplete collection, parser errors, schema drift, duplicate records, and unreliable enrichment cause detection logic to fire on the wrong events or miss the right ones. Attackers can benefit from those gaps by blending into noisy sources, exploiting blind spots in logging coverage, or triggering alert overload that distracts analysts from higher-value signals.

Impact: The SOC may waste time on false positives, delay escalation on real incidents, and develop false confidence in coverage. Over time, that degrades detection quality, slows response, and leaves the organisation more exposed to persistence and lateral movement that should have been visible.

Practitioner Guidance

What to prioritise: Validate the data sources that drive your highest-consequence detections first. If the fields, timestamps, and enrichment needed for triage are not dependable, do not promote the rule into broad automation.

What to verify: Check that each automated detection can be traced back to a known-good input path, a documented schema, and an explicit test case. If analysts cannot explain why the alert fired from the raw telemetry upward, the automation is not mature enough for high-trust use.

Decision rule: Automate only when the detection logic remains stable under normal data variation. If small source changes materially alter alert volume or classification, treat data quality as the blocking issue rather than tuning the rule endlessly.

Practitioner takeaway: The most dangerous automation is the kind that looks successful while silently encoding bad telemetry decisions across the SOC.

Related resources from NHI Mgmt Group

• What do teams get wrong when they automate before normalizing data?
• Should teams automate SOX access certifications before standardising entitlement data?
• How should security teams model SOC capacity before adding more detections?
• What breaks when SOC teams automate triage on poor-quality telemetry?