The most common mistake is using attributes that create too many unique combinations, which inflates storage, memory use, and cost. Teams should keep attributes limited to stable, low-cardinality values that support useful filtering. Avoid user IDs, session IDs, and timestamps as labels. Well-designed attributes improve analysis, while overly detailed labels can overwhelm the monitoring pipeline.
Why This Matters for Security Teams
OpenTelemetry attributes are powerful because they let teams slice telemetry by service, route, region, tenant, and deployment context. The failure mode is treating every useful dimension as a label without thinking about cardinality. Once attributes vary too widely, the monitoring system spends more effort storing and indexing unique combinations than helping teams investigate incidents. That pushes up memory use, query cost, and noise, and it can hide the very signals teams expect to surface.
This is a design issue, not just a dashboard issue. Instrumentation decisions affect ingestion pipelines, retention, and alert quality upstream. Stable attributes make it possible to aggregate meaningfully, while unstable values turn metrics into a high-volume approximation of raw events. In practice, many teams discover the problem only after a rollout increases telemetry cost or after a dashboard becomes too slow to use during an incident.
How It Works in Practice
Good metric design starts with deciding whether a field is meant for grouping or for tracing detail. Metric labels should usually describe a bounded category, such as environment, service name, region, endpoint family, or outcome class. Those values remain useful because they support filtering and roll-up without exploding the number of series. By contrast, request-specific values belong in logs or traces, not in metric labels.
A practical test is whether the value could be written on a finite set of buckets that stays stable over time. If the answer is no, it is usually the wrong attribute for a metric. Timestamps, UUIDs, user IDs, session IDs, and full URLs with unbounded path parameters are common examples of values that create unnecessary series churn. Teams should also review labels added by libraries and middleware, since the worst cardinality problems often come from convenience defaults rather than explicit design.
- Keep labels coarse enough to support aggregation and alerting.
- Prefer dimensions that change slowly and predictably.
- Move request-level uniqueness to traces or logs.
- Review new instrumentation for series growth before release.
- Use dashboards to answer trends, not to replay every event.
Where this guidance breaks down is in systems that need very high-fidelity per-entity measurement, because the operational cost of that granularity can be justified only when the metric pipeline and storage model are built for it.
Common Variations and Edge Cases
Tighter attribute discipline often reduces analytical flexibility, so teams have to balance observability depth against operational cost. That tradeoff becomes sharper in multi-tenant systems, highly dynamic infrastructure, and APIs with many customer-generated paths, where a seemingly harmless label can multiply series counts quickly.
Some environments can safely tolerate more dimensions when the value set is genuinely bounded. For example, a small fixed list of deployment regions or a handful of product tiers is usually acceptable. The risk rises when teams confuse “useful for debugging” with “safe for metric aggregation.” Best practice is evolving around the same core principle: metrics should expose patterns, not every unique occurrence. If a value is only useful once, it probably does not belong in a metric label.
Another edge case is vendor or framework auto-instrumentation. It can be convenient, but it may also introduce labels that look innocuous until they encounter tenant IDs, user agents, or path fragments. Teams should validate exported series in staging, then watch for sudden increases in cardinality after deploys, schema changes, or feature launches.
Practitioner Guidance
What to prioritise: Review the highest-volume and highest-churn metric families first. If a label can vary per request, per user, or per session, treat it as suspect until proven bounded.
Decision rule: If the value helps you group related events over time, it belongs in a metric; if it mainly identifies one occurrence, it belongs elsewhere.
What to verify: Confirm that new labels do not create unbounded series in staging, and check whether auto-generated attributes from libraries are silently increasing cardinality.
Common mistake: Teams often preserve “debug-friendly” labels long after they stop being operationally useful, then pay for them in storage, query latency, and alert fatigue.
Practitioner takeaway: The best metric labels are boring on purpose, because stability and bounded variation matter more than perfect contextual detail.
