Notifications
Clear all
Topic starter
19/08/2026 3:26 pm
TL;DR: Kubernetes observability becomes expensive when telemetry policy is delegated to dozens of service teams, because cluster-level collection, churn-driven cardinality, and retention drift turn normal platform behaviour into runaway cost, according to Sawmills. The practical fix is central policy enforcement at the collector layer, where runtime controls can bound volume, attribution, and routing.
NHIMG editorial — based on content published by Sawmills: Kubernetes observability for platform teams, from kubelet metrics to cost-controlled telemetry
By the numbers:
- 69% of organisations now have more machine identities than human ones.
- 61% of organisations rely on spreadsheets or manual, tracking for machine identity management.
Questions worth separating out
Q: How should platform teams control Kubernetes observability costs without blocking developers?
A: Centralise telemetry policy at the collector layer, not in individual application teams.
Q: Why do Kubernetes observability budgets grow faster than workload usage?
A: Because Kubernetes multiplies telemetry through churn.
Q: What breaks when telemetry attribution metadata is missing in a shared cluster?
A: Chargeback, ownership, and remediation prioritisation all become unreliable.
Practitioner guidance
- Enforce high-cardinality label controls at ingestion Block pod UID, container ID, and other churn-heavy fields from becoming permanent metric dimensions unless a documented use case exists.
- Make collector RBAC a monitored dependency Validate that the k8sattributes processor and similar enrichment components still have get, list, and watch permissions on pods, namespaces, deployments, and replicasets.
- Separate node-local collection from central policy enforcement Use DaemonSets for local log and kubelet ingestion, then route telemetry through a central gateway for filtering, redaction, and tail-based sampling.
What's in the full article
Sawmills' full article covers the operational detail this post intentionally leaves for the source:
- Collector deployment patterns and when DaemonSet, sidecar, or gateway architectures fit best
- Example k8sattributes enrichment configuration and the RBAC needed for reliable attribution
- Cost-control mechanics for logs, metrics, and traces across Kubernetes workloads
- Policy-enforcement patterns for sampling, redaction, routing, and backend selection
👉 Read Sawmills' analysis of Kubernetes observability for platform teams →
Kubernetes observability costs: what platform teams need to control?
Explore further
19/08/2026 4:10 pm
Telemetry governance is now an access-control problem as much as an observability problem. In shared Kubernetes environments, the collector's ServiceAccount, RBAC, and namespace attribution rules decide who can emit, enrich, and route data. That makes telemetry policy part of the broader identity governance surface, especially where platform teams control the substrate but not every workload. Practitioners should treat collector permissions and metadata trust as governance controls, not plumbing details.
A question worth separating out:
Q: Which controls matter most when enforcing observability policy in Kubernetes?
A: Start with RBAC on collector service accounts, metadata enrichment integrity, and label governance. Then add routing rules, retention tiers, and sampling policy at the gateway. Those controls together determine whether telemetry is merely collected or actually governed across the cluster.
👉 Read our full editorial: Kubernetes observability cost control is a platform policy problem
