Notifications
Clear all
Topic starter
19/08/2026 3:55 pm
TL;DR: Exploding observability costs, inconsistent data quality, and brittle OpenTelemetry pipelines that break under change define a telemetry problem in KubeCon North America conversations, according to Sawmills. The practical implication is that teams need upstream control over telemetry volume, schema quality, and pipeline reliability before cost and signal loss become structural.
NHIMG editorial — based on content published by Sawmills: KubeCon NA recap on observability cost, telemetry quality, and pipeline innovation
Questions worth separating out
Q: How should teams reduce observability costs without losing useful telemetry?
A: Start at the pipeline, not the backend.
Q: Why do brittle telemetry pipelines create risk for IAM and SOC programmes?
A: Brittle pipelines create risk because identity evidence and detection signals lose value when they are delayed, incomplete, or trapped in a proprietary format.
Q: What do organisations get wrong about telemetry cost optimisation?
A: They often optimise for storage and CPU without measuring the impact on correlation, auditability, and incident reconstruction.
Practitioner guidance
- Implement upstream telemetry filtering Reduce ingestion cost by filtering noisy logs, low-value metrics, and redundant traces before they reach premium storage and analytics layers.
- Validate pipeline change safety Test OpenTelemetry Collector updates for hot reload behaviour, rollback safety, and continuity before you allow production pipeline changes.
- Standardise telemetry schema quality Require consistent attributes, field naming, and enrichment rules so automation and incident response can depend on the data.
What's in the full article
Sawmills' full recap covers the operational detail this post intentionally leaves for the source:
- The conference talk details how hot reloading works across OpenTelemetry Collector processors, filters, samplers, and transformers.
- It includes the practical pipeline changes that can be made without restarts or data gaps, which matters for production rollout planning.
- It shows how the platform handles waste detection, schema fixes, and remediation in-stream for teams that need implementation detail.
- The source also frames the observability cost problem in the vendor's own terms, including how teams are thinking about Datadog, New Relic, Grafana, and Loki.
👉 Read Sawmills' recap of KubeCon telemetry cost and pipeline innovation →
Telemetry pipeline fragility and cost pressure: what should teams change?
Explore further
19/08/2026 4:32 pm
Telemetry governance is becoming a control-plane problem, not a tooling problem. The article shows that costs, noise, and pipeline fragility are converging into one operational failure mode: teams cannot govern what they cannot change safely. That matters because telemetry underpins detection, troubleshooting, and audit evidence. When change itself creates gaps, the issue is governance maturity, not just observability spend. Practitioners should treat telemetry pipelines as part of the control plane.
A question worth separating out:
Q: How should organisations govern telemetry changes in production?
A: Treat telemetry changes like any other production control change. Require validation for schema consistency, hot reload behaviour, rollback paths, and continuity of collection before deployment. If a change can interrupt data flow, it should pass the same operational scrutiny as a release that could affect service availability.
👉 Read our full editorial: Telemetry cost and pipeline fragility are outpacing observability teams
