Start by treating cardinality as a policy issue, not a cleanup issue. Enforce limits on tags such as user_id, request_id, and pod_name at the point telemetry is created, then aggregate or drop dimensions that add little diagnostic value. The goal is to prevent billable timeseries from expanding silently as services scale.
Why high-cardinality telemetry becomes a cost problem
High-cardinality telemetry is expensive because each unique tag combination can generate a new timeseries, and those series are what observability platforms meter, store, and query. A small amount of uncontrolled variation, especially from per-request, per-user, or per-pod labels, can multiply into a billing problem long before it becomes an obvious signal quality problem.
The practical issue is not just volume, but multiplicative growth. If cardinality rises in a fast-moving service, teams can see usage spike even when overall event rates stay flat. That is why cost control has to start at instrumentation design, not after the data has already landed.
In mature telemetry programs, the key decision is which dimensions are truly diagnostic and which are merely descriptive. A tag should exist because it helps answer a recurring operational question, not because it is easy to attach at emit time. NIST AI Risk Management Framework is a useful analogue for treating measurement design as a governance choice, even outside AI, because the control objective is disciplined signal creation.
How to reduce cardinality without losing useful observability
Teams should define a small, approved set of dimensions for metrics and traces, then enforce those limits where telemetry is created. That usually means rejecting or normalising fields such as user_id, request_id, session_id, and pod_name when they create too many unique combinations to support routine analysis.
Aggregation is the safest default when a field is useful for debugging but too granular for billing or dashboards. For example, you can roll up by service, endpoint, tenant tier, region, or error class while dropping ephemeral identifiers that only help with one-off investigations. This preserves operational visibility while preventing silent timeseries explosion.
The best control point is the source library, sidecar, or agent that emits telemetry, because downstream filtering arrives too late to stop cost from accruing. NIST Cybersecurity Framework 2.0 fits here because the underlying discipline is governance of what gets produced, protected, and monitored, not just what gets stored.
At scale, teams should also separate low-cost diagnostic samples from high-value business metrics. A metric used for capacity planning deserves different treatment from an exploratory label used during incident response. When those two uses are blended, cost and noise tend to rise together.
What teams should govern to keep Datadog spend predictable
Cardinality control works best when it is treated as an engineering policy with ownership, thresholds, and review, not as a one-time cleanup task. Teams should set allowed-tags standards, define a process for adding new dimensions, and require explicit approval for any field that can expand series counts unpredictably.
It also helps to monitor cardinality as a first-class operational metric. If a service starts emitting new series faster than expected, that is often a sign of a code change, a new deployment pattern, or an instrumentation regression rather than a genuine increase in business complexity. The goal is to catch that shift before it becomes a recurring monthly surprise.
Where telemetry is tied to APIs or request pipelines, high-cardinality dimensions are often a symptom of poor request-shape discipline. OWASP API Security Top 10 is relevant as a reminder that unbounded resource use and weak object-level discipline are architectural issues, not just cost issues.
Risk and Threat Considerations
Unchecked cardinality creates a quiet failure mode: costs rise gradually, dashboards become harder to query, and teams start turning off useful telemetry to contain spend. The risk is not only financial. Excessive series counts can also degrade alert quality and make operational investigations slower when systems are under stress.
Failure mechanism: A single high-entropy tag such as request_id or user_id multiplies series across services, environments, and labels, causing storage, query, and alerting load to grow faster than traffic.
Impact: Datadog spend can become volatile and difficult to forecast, and teams may disable telemetry that was actually helping them detect or triage incidents.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP API Security Top 10 addresses the attack and risk surface, while NIST CSF 2.0 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.PO-01 — Policy Establishment | Telemetry tag limits need an explicit governance policy. |
| GV.RM-01 — Risk Management Strategy | Runaway observability spend is a measurable operational risk. | |
| Recommendation — Define telemetry cardinality rules and make service teams accountable for enforcing them. Treat cardinality growth as a managed risk with thresholds and review triggers. | ||
| OWASP API Security Top 10 | API4 — Unrestricted Resource Consumption | Unbounded tag growth can drive storage, query, and alerting cost. |
| Recommendation — Limit high-entropy request dimensions that create unbounded observability resource use. | ||
Practitioner Guidance
What to prioritise: Start with the tags that can explode combinations fastest, especially per-request and per-entity identifiers. The first pass should target metrics and traces where the same business event is being recorded with many unique label values.
What to verify: Confirm that every emitted dimension has a documented diagnostic use case and an owner who can explain why it belongs on a billable timeseries. If a tag is only useful during rare deep dives, consider making it available in logs or sampled traces instead of always-on metrics.
What good looks like: Stable series growth, predictable spend, and a small set of tags that are consistent across services. The best outcome is not zero cardinality, it is cardinality that is intentionally bounded.
Practitioner takeaway: The most durable fix is to govern telemetry shape at creation time, because once high-cardinality data is flowing, cost control becomes reactive and much harder to sustain.
Related resources from NHI Mgmt Group
- Why do unused SaaS licences keep creating cost even when teams stop using the app?
- How should teams control Datadog cost without cutting useful telemetry?
- How should security teams stop SMS toll fraud before cost accumulates?
- How should security teams stop AI coding tools from creating secrets sprawl?