Notifications
Clear all
Topic starter
19/08/2026 4:00 pm
TL;DR: Metric cardinality turns ordinary labels into a cost, performance, and availability problem across Prometheus-like stacks and SaaS monitoring platforms, according to Sawmills. The governance answer is not more dashboards, but label discipline, pipeline enforcement, and blast-radius controls that stop unbounded dimensions before they reach storage or billing.
NHIMG editorial — based on content published by Sawmills: Why metric cardinality keeps exploding and how to tame it
Questions worth separating out
Q: How should security teams control metric cardinality in shared observability platforms?
A: Security teams should treat metric labels like governed metadata, not free-form text.
Q: Why does metric cardinality become an availability problem during incidents?
A: High cardinality forces query engines to scan too many series, which slows dashboards, raises memory use, and can delay alert evaluation.
Q: What do teams get wrong about high-cardinality labels?
A: Teams often focus on whether a label is useful in one dashboard and ignore whether it is stable across the whole fleet.
Practitioner guidance
- Publish a stable label vocabulary Allow only low-entropy labels such as env, region, service, and status code, and explicitly deny user_id, session, request_id, raw URL paths, and container-specific identifiers.
- Enforce filtering at the collector edge Use Prometheus relabeling or OpenTelemetry Collector processors to drop or rewrite unstable labels before data reaches storage or indexing layers.
- Set per-service series budgets Track active series counts by metric and by service, then cap or aggregate the offending metric when a service exceeds its expected baseline.
What's in the full article
Sawmills' full article covers the operational detail this post intentionally leaves for the source:
- Concrete Prometheus relabeling examples for dropping risky labels before storage
- Datadog indexing guidance for keeping high-volume metrics queryable without exploding cost
- Step-by-step policy patterns for allowlists, denylists, and threshold-based blocking
- Tool-by-tool comparison of telemetry pipeline controls across self-hosted and SaaS stacks
👉 Read Sawmills' full analysis of why metric cardinality keeps exploding and how to tame it →
Metric cardinality is the governance gap observability teams are missing?
Explore further
19/08/2026 4:37 pm
Metric cardinality is an observability governance failure, not a telemetry tuning issue. Once teams allow arbitrary labels into hot metrics, they create a control problem that is structurally similar to credential sprawl in identity systems. The issue is not that observability is inherently expensive, but that unbounded dimensions create unmanaged state. Practitioners should treat label policy as a security-adjacent control with auditability, ownership, and enforcement.
A question worth separating out:
Q: How do teams know if cardinality controls are actually working?
A: Look for falling active series counts, fewer sudden series spikes, faster query response times, and lower variance between services. If an observability platform still shows repeated fanout, expensive indexing, or noisy metrics after policy changes, the controls are not reaching the source. Effective governance shows up in stable baselines, not just cleaner dashboards.
👉 Read our full editorial: Metric cardinality is breaking observability governance at scale
