Notifications
Clear all
Topic starter
19/08/2026 3:34 pm
TL;DR: Telemetry pipelines decide what logs, metrics, and traces exist, how they are shaped, and where they go, and Sawmills argues that this upstream control layer is where observability spend is actually won or lost. Grafana's 2025 Observability Survey found that 74% of respondents now treat cost as a top priority, which makes pipeline governance a budget and reliability issue, not just a tooling choice.
NHIMG editorial — based on content published by Sawmills: Telemetry pipelines: the control layer between your services and your observability bill
By the numbers:
- 74% of respondents now treat cost as a top priority when selecting observability tools.
- 41% increased their observability spend in the past year.
Questions worth separating out
Q: How should teams reduce observability costs without losing useful telemetry?
A: Start at the pipeline, not the backend.
Q: Why do telemetry pipelines matter for governance rather than just engineering efficiency?
A: Because they decide what data is allowed to exist in the observability system at all.
Q: What breaks when telemetry pipelines are not treated as sensitive data flows?
A: Secrets, tokens, and identity context can replicate across brokers, caches, and SIEM systems before redaction occurs, which means attackers may harvest usable credentials long before defenders notice.
Practitioner guidance
- Move filtering upstream of backend ingestion Drop health checks, readiness probes, and other low-value signals before they reach billable storage or indexing.
- Strip high-cardinality attributes before export Remove volatile keys such as pod_name, request_id, and user_id from metrics unless they are explicitly required for troubleshooting.
- Reserve tail sampling for gateway tier decisions Keep errors and slow traces with tail sampling, but only after all spans for a trace have converged in one collector instance.
What's in the full article
Sawmills' full article covers the operational detail this post intentionally leaves for the source:
- Receiver, processor, and exporter configuration examples for OpenTelemetry pipelines in production
- Practical guidance on filter, transform, and tail sampling ordering inside agent and gateway tiers
- Reliability tradeoffs for backpressure, memory limits, and buffering under backend outages
- Where the agentic operator model changes day-to-day telemetry maintenance and suppression-rule drift
👉 Read Sawmills' analysis of telemetry pipelines as the observability control layer →
Telemetry pipelines and observability costs: where does control belong?
Explore further
19/08/2026 4:16 pm
Telemetry pipeline governance is becoming a control problem, not a tooling problem. The article correctly treats the pipeline as the place where value and waste are separated before cost is incurred. That is a governance pattern security and identity teams should recognise, because once noisy or over-permissive telemetry lands in a backend, the organisation has already paid for its collection and often its retention. Practitioners should think about upstream decision rights, not just downstream storage efficiency.
A question worth separating out:
Q: How should security teams think about telemetry pipeline reliability during incidents?
A: Treat the pipeline as critical infrastructure. Backpressure, memory limits, and buffering need to be tuned so severe logs and audit evidence survive backend slowness, while low-value streams can degrade gracefully. Otherwise, the organisation either loses evidence or destabilises the systems producing it.
👉 Read our full editorial: Telemetry pipelines are the missing control layer for observability costs
