Billable timeseries expand multiplicatively, so a single metric can become thousands or millions of unique series. That drives cost far faster than most teams expect and makes manual cleanup ineffective because the issue is embedded in the data shape, not the number of metrics alone.
Why This Matters for Security Teams
Uncontrolled high-cardinality telemetry tags turn observability from a control asset into a cost and reliability risk. When tag values such as user IDs, request IDs, container hashes, or full URLs are allowed to multiply without restraint, monitoring systems can be flooded with near-unique series that are difficult to query, expensive to store, and noisy to operate. This is not just a finance issue. It affects detection latency, alert quality, and incident review because analysts spend more time sorting signal from metadata churn.
For security teams, the risk is that telemetry governance becomes accidental rather than designed. A mature program should define which dimensions are useful for troubleshooting, which are prohibited, and how retention and sampling interact with volume growth. That fits the intent of the NIST Cybersecurity Framework 2.0, which treats visibility, governance, and resilience as operational controls rather than afterthoughts. The same logic applies to cloud and application security telemetry, where poorly bounded labels can create hidden exposure to both overspend and blind spots.
In practice, many security teams encounter the tag problem only after the monitoring bill spikes or an outage forces them to discover that the most useful dashboards are the least affordable to keep.
How It Works in Practice
Cardinality is the number of distinct values a field can take. A metric with a low-cardinality tag like region may produce a small, stable set of series. A metric tagged with customer ID, session token, or full endpoint path can explode into an unbounded number of series. The platform then stores, indexes, and queries each series separately, which increases ingestion load, query cost, memory use, and dashboard latency.
Operationally, the key question is whether a tag helps answer an expected security or engineering question. If it does not support a repeatable use case, it should usually be removed, normalized, hashed carefully, or moved to logs with stricter retention. Security teams often apply a tiered approach:
- Allow only approved dimensions on high-volume metrics.
- Normalize labels before ingestion, such as collapsing dynamic paths into templates.
- Separate investigative detail into logs or traces rather than metric tags.
- Set budgets and alerts for series growth, not just total ingest volume.
- Review whether tags contain secrets, identifiers, or regulated personal data.
For detection engineering, this matters because uncontrolled labels reduce the usefulness of aggregations and can hide anomalies inside millions of near-duplicate time series. Guidance from CISA secure development guidance and logging best practice generally supports collecting only the data needed for a defined control objective. In distributed systems, especially Kubernetes, serverless, and microservice environments, the problem becomes harder because auto-generated identifiers and dynamic resource names are constant sources of cardinality growth. These controls tend to break down when developers can add new labels at deployment time without a central allowlist because the data shape changes faster than telemetry governance can review it.
Common Variations and Edge Cases
Tighter tag control often increases engineering overhead, requiring organisations to balance troubleshooting detail against cost, performance, and privacy constraints. That tradeoff is especially visible in incident response, where investigators want rich context but production observability must remain bounded. Current guidance suggests that the safest approach is to reserve high-cardinality fields for short-lived, purpose-built investigations rather than always-on metrics.
There is no universal standard for exactly which tags are acceptable across all environments. A tag that is harmless in a small internal service may become unmanageable in a multi-tenant SaaS platform or a high-throughput API gateway. Edge cases also include security tooling that auto-attaches identifiers, APM agents that capture raw request paths, and platform defaults that expose pod names, image digests, or instance IDs. Those values may be useful, but they should be tested for series growth before broad rollout.
Teams should also consider whether telemetry tags expose identity data. In environments with customer accounts, workforce identities, or non-human identities, tags can unintentionally create an audit trail that is broader than intended. For that reason, observability governance should sit alongside access control, data minimisation, and retention policy. Where identity data is present, the same discipline that protects secrets and tokens should be applied to telemetry labels, because uncontrolled metadata can become a privacy and security issue even when the underlying metric is benign.
Related operational patterns are discussed in OpenMetrics and Prometheus metric naming and labeling guidance, both of which reinforce the principle that labels should be stable, intentional, and limited.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK address the attack surface, NIST CSF 2.0 set the technical controls, and PCI DSS v4.0 define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.PO | Telemetry tag limits are a governance policy decision, not just an ops tweak. |
| MITRE ATT&CK | T1036 | Telemetry manipulation can conceal malicious activity by blending into noisy data. |
| PCI DSS v4.0 | 10 | Logging and monitoring data must stay controlled when it can carry sensitive identifiers. |
Restrict telemetry fields that could broaden audit data scope or expose payment-related identifiers.
