Detection engineering now hinges on telemetry pipelines and control quality

At a glance

What this is: This is an analysis of modern SOC detection engineering, showing that telemetry pipelines and log governance now shape both detection quality and cost.

Why it matters: It matters to IAM and security teams because identity logs, endpoint telemetry, and access events only become useful when source ownership, normalization, and retention are controlled end to end.

By the numbers:

• CrowdStrike said its 2025 acquisition of Onum was aimed at redefining the SOC data layer by streaming enriched telemetry and cutting SIEM storage needs.
• CrowdStrike described Onum as capable of up to 50% cost reduction in storage and 5x event throughput.
• 2026, article says security budgets are not necessarily growing in 2026, which increases pressure to optimize hot ingest and preserve full-fidelity history.

👉 Read Abstract Security's analysis of modern SOC detection engineering and telemetry pipelines

Context

Detection engineering fails when teams treat telemetry as a technical afterthought instead of a governed input to security operations. In practice, the problem starts before detection logic is written: if identity logs, endpoint telemetry, and network events are inconsistent, incomplete, or poorly owned, the SOC inherits noise instead of evidence.

For IAM practitioners, this intersects directly with access visibility and authentication telemetry. Identity events only support detection when they are normalized, mapped to a common model, and routed with clear accountability, which is why log onboarding, field mapping, and retention controls now sit adjacent to broader identity governance rather than outside it.

Key questions

Q: How should security teams implement telemetry pipelines before the SIEM?

A: Teams should treat the telemetry pipeline as a governed control point. Define source ownership, intake gates, schema mapping, and quality checks before data reaches the SIEM. That approach reduces noise, preserves useful context, and prevents the SIEM from becoming an expensive raw-log repository with inconsistent inputs.

Q: Why do identity logs matter so much in detection engineering?

A: Identity logs anchor correlation across authentication, endpoint, and network activity. Without complete and normalized auth events, the SOC cannot reliably connect a login, a device action, and a network sequence to the same user or service account. That makes identity telemetry a foundational detection input, not a side feed.

Q: What do security teams get wrong about log normalization?

A: They often treat normalization as formatting work instead of a detection requirement. In reality, field mapping determines whether rules can compare events accurately across sources. If user, time, and source fields are not aligned, false positives rise and real attack paths can be missed.

Q: How do teams know if detection engineering is actually improving?

A: They should measure whether new detections survive variant testing, whether telemetry is complete enough to support triage, and whether alerts feed back into updated logic without long delays. If the programme only changes during quarterly reviews, it is probably drifting faster than it is improving.

Technical breakdown

Why telemetry pipelines sit in front of the SIEM

A telemetry pipeline ingests logs, enriches them, filters out low-value noise, and routes events to the right destination before they reach the SIEM. That matters because SIEMs are expensive to use as raw ingestion bins, especially when source formats differ and high-volume logs add little detection value. In modern SOCs, the pipeline becomes the control point for normalization, retention, and delivery quality. It also reduces the chance that detections fail simply because the upstream data was never shaped correctly for correlation.

Practical implication: define pipeline ownership and schema standards before expanding SIEM coverage.

How normalization improves identity and cross-source correlation

Normalization is the process of mapping different log field names and value formats into a common model such as CIM or OCSF. One source may call an address src_ip, another source.ip, and a third sourceAddress. Without consistent mapping, correlation rules miss relationships or generate false positives because they cannot reliably compare the same entity across systems. For identity-centric detections, this is especially important because user identifiers, timestamps, and result values must align across authentication, VPN, and endpoint logs before analysis becomes trustworthy.

Practical implication: map identity and endpoint fields to a shared schema before writing multi-source detections.

How CTI-driven detection engineering changes SOC priorities

CTI-driven detection engineering starts with threat techniques, not with whichever logs happen to be available. Teams convert priority intelligence requirements into detections, attach ATT&CK technique IDs, document telemetry prerequisites, and test the logic with positive and negative cases before deployment. That approach treats detections as code, which makes quality measurable through precision, recall, and replay testing. It also keeps the backlog tied to current adversary behavior instead of stale rule sets that no longer reflect the threat environment.

Practical implication: tie each new detection to a threat technique, a data prerequisite, and a test case set.

NHI Mgmt Group analysis

Detection engineering is becoming a governance problem, not just an analytics problem. The article shows that SOC performance now depends on whether source systems, security engineering, and operations teams agree on intake gates, ownership, and success criteria. When log onboarding lacks a formal process, the organization creates detection debt that no rule tuning exercise can fully remove. For practitioners, the lesson is that telemetry governance is part of security governance.

Normalization is the real control surface for modern correlation. Common schemas such as CIM and OCSF do more than clean up data. They determine whether identity events, endpoint events, and network events can be joined reliably enough to support meaningful detections. The named concept here is telemetry normalization debt: the cumulative risk that inconsistent log formats create when teams postpone schema alignment. For practitioners, reducing that debt improves both signal quality and analyst efficiency.

Identity telemetry sits at the center of SOC value creation. The article explicitly highlights auth events as a foundational source, which means IAM teams influence detection quality far beyond access reviews. Authentication logs only become operationally useful when they are complete, normalized, and retained under policy. That makes identity visibility a shared dependency between IAM, SOC, and engineering teams. For practitioners, access telemetry should be treated as a monitored security asset.

Pipeline economics are now part of security architecture decisions. The shift toward filtering and enrichment before the SIEM reflects a broader market change in how organisations pay for visibility. That changes evaluation criteria for detection platforms, log routing, and storage strategy. Teams should recheck whether their current data path preserves full-fidelity history where needed while avoiding unnecessary hot ingest. For practitioners, architecture decisions now need to balance cost, fidelity, and detection utility together.

Detection quality must be measured like an engineering outcome. Precision, recall, MTTD, MTTR, and analyst time saved are all operational indicators that matter more than rule count. A detection program that cannot test content, replay history, or retire obsolete logic will accumulate noise and miss changing attacker behavior. For practitioners, the right governance model is continuous validation rather than static rule ownership.

What this signals

Detection engineering programmes should expect more of the data layer, not less. The practical shift is toward treating telemetry pipelines as governance infrastructure, with schema control, ownership, and routing decisions shaping whether the SOC can see meaningful behaviour or only filtered noise.

Telemetry normalization debt: when teams postpone schema alignment, every downstream rule inherits avoidable ambiguity. That is especially true where identity telemetry is involved, because authentication, endpoint, and network events only become operationally useful when they can be joined reliably across systems. For practitioners, reducing that debt should be a standing programme objective, not an ad hoc cleanup.

Identity-heavy environments should also watch the boundary between access control and observability. If the SOC cannot reliably ingest and normalize auth events, then IAM decisions, investigations, and response actions all slow down. That makes identity telemetry a shared dependency across NIST Cybersecurity Framework 2.0 functions for identify, protect, detect, and respond.

For practitioners

• Formalise log onboarding as a gated process Require source ownership, sample logs, field lists, and explicit success criteria before any new log source enters production. Use intake gates for schema review, privacy checks, and retention approval so detection content is built on known inputs rather than guesswork. This is especially important for identity logs and other high-value telemetry.
• Map identity and endpoint data to a common model Standardise fields such as user identifiers, timestamps, IP addresses, and action results across authentication, VPN, EDR, and firewall sources. Use a shared schema such as CIM or OCSF so correlation rules can compare like with like and avoid source-specific parsing logic.
• Treat detection content as code with tests Write each rule with documented telemetry prerequisites, ATT&CK mapping, and positive and negative test cases. Replay historical data before deployment and track precision, recall, and false positive rate after rollout so tuning is based on evidence, not intuition.
• Separate hot ingest from full-fidelity retention Route low-value or high-volume telemetry through filtering and enrichment before it reaches the SIEM, then preserve the relevant raw history in a cheaper tier for later investigation. This keeps analytics responsive without sacrificing forensic depth.

Key takeaways

• Modern detection engineering depends on governed telemetry pipelines, not raw SIEM ingestion.
• Identity logs become operationally valuable only when ownership, normalization, and retention are managed as part of the SOC control plane.
• Teams should measure detection quality and data fitness together, because noisy inputs undermine even well-written rules.

Key terms

• Telemetry Pipeline: A telemetry pipeline is the processing layer that ingests, filters, enriches, and routes security data before it reaches analytics tools. In SOC operations, it determines whether raw events are usable, affordable, and correlated consistently across sources.
• Normalization: Normalization is the process of converting different log formats into a shared structure and value set. It lets detections compare identity, endpoint, and network events reliably, which reduces parsing complexity and lowers false positives.
• Detection Engineering: The discipline of designing, testing, and maintaining detection logic so it remains useful against real attacker behaviour. It covers telemetry selection, rule quality, false-positive management, and the operational workflow needed to keep alerts actionable.
• Priority Intelligence Requirements: Priority intelligence requirements are the specific threat questions a security team wants answered by intelligence and detection work. They help teams decide which tactics, techniques, and behaviors deserve coverage first, instead of building rules around whatever data is easiest to collect.

What's in the full article

Abstract Security's full article covers the operational detail this post intentionally leaves for the source:

• Step-by-step log onboarding process design, including intake gates and success criteria
• Field mapping examples across endpoint, identity, and network sources for CIM-style normalization
• Detection engineering QA practices, including replay testing, tuning, and change management
• SOC pipeline and cost optimisation examples for routing telemetry before the SIEM

👉 The full Abstract Security post covers the data pipeline architecture, detection QA process, and cost-routing examples

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course, the industry's only accredited NHI security programme. It is designed for practitioners who need a stronger identity lens across access, lifecycle, and governance decisions.