TL;DR: Telemetry pipelines are increasingly being asked to normalise, enrich, and route data before it reaches SIEM, because distributed systems, schema drift, and rising ingestion costs have made downstream-only observability too slow and too expensive, according to DataBahn. The governance shift is upstream: context now determines both security value and storage cost, so observability architecture becomes a data integrity and decision-making problem, not a dashboard problem.
At a glance
What this is: This analysis says observability is breaking at the pipeline layer, where raw telemetry is too inconsistent and expensive to defer enrichment until after SIEM ingestion.
Why it matters: It matters to security and IAM practitioners because upstream context, identity resolution, and routing decisions now shape both detection quality and the economics of retaining security data.
Context
Observability has become a data-governance problem as much as a monitoring problem. In distributed cloud environments, telemetry arrives from many services, regions, containers, and tooling layers, and the old assumption that downstream platforms can sort everything out no longer holds.
The article's core claim is that enrichment, normalization, and routing need to happen before SIEM ingestion because raw telemetry is too noisy, too expensive, and too inconsistent to be treated as a finished security signal. Where identity, asset ownership, and context are attached in the pipeline changes what the organisation can trust downstream.
Key questions
Q: Where does pre-SIEM enrichment fail in practice?
A: It fails when teams treat enrichment as a post-ingestion convenience instead of a routing control. By the time context is added after SIEM ingestion, the organisation has already paid full price for the event and analysts still have to reconstruct ownership, relevance, and risk across separate tools.
Q: Why does enrichment timing change SIEM cost and detection quality?
A: Because the same context that improves triage also determines whether an event deserves expensive retention. If enrichment happens before ingestion, teams can separate high-value telemetry from routine noise while still preserving the evidence needed for investigations.
Q: What are the signs that an observability pipeline is failing in practice?
A: Common warning signs include duplicated data, missing data, runaway storage costs, and hard-to-debug routing logic across multiple agents. High-cardinality telemetry can also quietly push costs up while obscuring useful signals. When teams cannot explain what is being collected, where it goes, or why it is kept, the pipeline has lost control of the telemetry flow.
Q: Should teams centralise enrichment or keep it close to the source?
A: They should keep it as close to the source as throughput and governance allow. Centralising everything in the SIEM increases cost and delays context, while source-adjacent enrichment gives the pipeline enough information to route, tier, and retain data intelligently.
Technical breakdown
Why downstream observability breaks under schema drift
Modern telemetry is heterogeneous by design. Logs, metrics, traces, and events arrive with different schemas, different update cycles, and different levels of completeness, so a downstream platform has to infer relationships after the fact. That works poorly when fields change, cardinality spikes, or sources multiply faster than the pipeline can be reworked. The real issue is not volume alone. It is that the meaning of the data is undecidable until context is attached. When enrichment is deferred, the observability stack becomes a reconstruction engine rather than a decision engine.
Practical implication: Treat schema drift and inconsistent telemetry as pipeline design problems, not dashboard problems.
How pre-SIEM enrichment changes the security economics
Enrichment is only cheap when it happens before the SIEM bills the event. Once raw data is ingested, the cost is already committed, even if later analysis decides the signal was low value. Pre-ingestion enrichment lets the pipeline attach asset ownership, geolocation, threat intelligence, and identity context before routing, so retention can be based on enriched value rather than raw volume. That is why upstream filtering and enrichment can reduce storage and licensing pressure without removing the original evidence needed for investigations.
Practical implication: Push enrichment upstream so routing decisions are made before ingestion-tier pricing applies.
Why identity context is now part of observability design
The article's most important identity angle is that telemetry without ownership and source context is operationally incomplete. If a log line cannot be tied to an asset, user, workload, or environment, analysts spend time rebuilding identity context across tools that should already agree on it. For IAM and NHI programmes, this means observability data should preserve machine identity, workload identity, and asset provenance as first-class fields. Otherwise, the SOC inherits signals that are technically present but not governable.
Practical implication: Preserve identity, asset, and workload context in telemetry before it reaches downstream detection tooling.
Threat narrative
Attacker objective: The practical objective is not theft but operational overload, where noisy telemetry and poor context degrade detection quality and inflate SIEM cost.
- Entry occurs through high-volume telemetry streams, where raw events from distributed systems reach the pipeline before they are normalised or enriched.
- Escalation happens when downstream tools inherit inconsistent schemas, forcing analysts and data engineers to manually reconstruct context across multiple platforms.
- Impact is operational and economic: SIEM storage becomes more expensive, investigations slow down, and the organisation pays full ingestion cost for low-value data.
Breaches seen in the wild
- CI/CD pipeline exploitation case study: Credentials in an exposed .git/config let a researcher edit a Bitbucket pipeline so it planted their SSH key on the server. No victim was named.
- Shai Hulud npm malware campaign: Shai Hulud campaign: npm malware exposed secrets on GitHub.
Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.
NHI Mgmt Group analysis
Pre-SIEM enrichment is a governance control, not just a performance optimisation. Once telemetry volume, schema drift, and ingestion pricing are coupled, the question is no longer whether teams can see enough data. The question is whether they can decide fast enough which data deserves SIEM-tier treatment. For IAM and NHI programmes, that means identity context must be preserved upstream, not reconstructed later. Practitioners should treat enrichment timing as part of control design.
Observability stacks are now data integrity systems. The article shows that the architectural problem is not missing dashboards but inconsistent source data that arrives too late to be made useful at scale. This is where identity security intersects directly with cyber operations: workload identity, asset ownership, and source provenance become part of the observability record. If those fields are absent or delayed, the SOC gets volume without trust. Practitioners should design telemetry as governed evidence.
Cost control and detection quality are converging into the same design decision. Pre-ingestion routing based on enriched context means security teams are no longer choosing between visibility and budget in separate steps. That changes the operating model for security engineering, because enrichment now influences retention, investigation speed, and alert fidelity at the same time. The practical conclusion is that teams need pipeline-level policy, not only SIEM tuning.
Identity resolution belongs in the telemetry pipeline. The article implicitly describes a shift toward what can be called telemetry identity context: the ability to know which asset, workload, or environment produced a signal before the event reaches downstream tools. This matters because detection logic is only as good as the trustworthiness of the source metadata. Practitioners should fold identity-aware enrichment into observability architecture reviews.
From our research library:
- Kubernetes production adoption has surged from 66% in CNCF's 2023 survey to 82% in 2025.
- Read next: AI Agent Observability, Audit and Incident Response Guide
What this signals
Telemetry governance is becoming a control plane decision. Once teams accept that enrichment timing determines both operational clarity and storage cost, observability architecture has to be reviewed alongside security architecture. The programme question is no longer whether data exists, but whether it arrives with enough context to be trusted and routed correctly.
Telemetry identity context: the value of observability now depends on preserving workload, asset, and environment context before downstream tools receive the event. That makes upstream enrichment a dependency for IAM, NHI governance, and SOC effectiveness rather than a later optimisation. Practitioners should expect identity-aware telemetry to become part of platform standards, not an optional enhancement.
For practitioners
- Define enrichment points before SIEM ingestion Map where asset ownership, identity context, threat intelligence, and geolocation are added so routing decisions happen before ingestion costs are locked in.
- Classify telemetry by retention value Create clear rules for which enriched events merit full-fidelity SIEM retention and which can move to lower-cost storage without losing investigative usefulness.
- Preserve identity metadata in the pipeline Carry workload identity, asset provenance, and environment labels through collection and enrichment stages so analysts do not have to rebuild context later.
- Measure pipeline health as a security signal Track schema drift, parser failures, noisy sources, and burst handling as operational indicators that the observability layer is degrading before downstream tools fail.
Key takeaways
- Raw telemetry that reaches SIEM without context forces security teams to pay for storage before they know whether the event is worth retaining.
- Upstream enrichment changes observability from a dashboard problem into a governed data pipeline where identity, asset, and threat context are attached early.
- Teams that move enrichment earlier can improve triage quality, reduce manual correlation, and lower ingestion costs at the same time.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while NIST CSF 2.0, CIS Controls v8 and CSA Cloud Controls Matrix set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | PR.DS-01 — Data-at-rest is protected | Telemetry enrichment and retention decisions affect how security data is handled and stored. |
| DE.CM-01 — The network is monitored to detect potential cybersecurity events | The article is about monitoring data quality and visibility across distributed systems. | |
| Recommendation — Treat telemetry as governed data and apply PR.DS-01 to protect stored logs and enriched events. Use DE.CM-01 to ensure telemetry collection is continuous enough to support detection. | ||
| CIS Controls v8 | CIS-8 — Audit Log Management | The article centers on how logs are collected, enriched, routed, and retained for security use. |
| Recommendation — Apply CIS-8 to define which logs are captured, enriched, and retained at security value. | ||
| MITRE ATT&CK | TA0007;TA0010 — Discovery; Exfiltration | Pipeline visibility and telemetry integrity are essential to spotting attacker discovery and data movement. |
| Recommendation — Map telemetry gaps to TA0007 and TA0010 so missing context does not hide discovery or exfiltration. | ||
| CSA Cloud Controls Matrix | SEF — Security Event and Logging | Cloud telemetry routing and enrichment sit directly inside logging and security event management. |
| Recommendation — Use SEF to govern how cloud events are collected, enriched, and routed for retention. | ||
Key terms
- Pre-SIEM Enrichment: Pre-SIEM enrichment is the process of attaching security context to telemetry before it reaches the SIEM. That context can include identity data, asset ownership, geolocation, or threat intelligence, allowing teams to make a routing decision before they pay indexed-storage costs.
- Telemetry Normalization: Telemetry normalization is the process of turning data from different security tools into a consistent format that can support one policy decision. It is essential when identity, endpoint, and asset systems all feed the same control plane, because conflicting data can otherwise create gaps or overblocking.
- Ingest-Based Pricing: A billing model where security cost scales with the amount of data sent into a platform. It changes the economics of SIEM operations because every unnecessary byte increases cost, making upstream filtering and enrichment a governance decision as much as a technical one.
- Identity context: The entitlement, ownership, and purpose information that explains why an action occurred and whether it was expected. For security operations, identity context turns raw alerts into decisions by showing which human or non-human identity acted and what it was allowed to do.
Deepen your knowledge
The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management for practitioners who need to connect identity controls to broader security operations.It helps security, IAM, and platform teams build stronger governance for identities that operate outside traditional human login flows.
Published by the NHIMG editorial team on August 2, 2026.
Updated on October 12, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org