Join our Newsletter — 33% off our NHI Course
Home› FAQ› Architecture & Implementation› How should teams secure Prometheus queries when metrics…
Architecture & Implementation

How should teams secure Prometheus queries when metrics must be filtered by tenant or permission system?

← Back to all FAQ
By NHI Mgmt Group Editorial Team Updated November 4, 2026 Domain: Architecture & Implementation

Teams should place an authorization layer in front of Prometheus so every query is checked before results are returned. A request proxy can validate the caller’s bearer token, map request parameters to the target object, and enforce a required label in the query. That pattern keeps shared observability data scoped to the right permissions system while still allowing normal Prometheus access patterns.

Why Prometheus needs a query authorization layer, not just a frontend filter

Prometheus itself is usually not the right place to enforce tenant or permission boundaries at query time. The safer pattern is to put an authorization layer in front of it so every request is checked before results are returned, and so the access decision is made with the caller context, the query parameters, and the target metric object all in view.

That matters because a shared metrics system often contains data from multiple teams, clusters, environments, or customers. If you only filter in the UI or dashboard layer, a user may still be able to issue a raw query that reaches data outside their scope. An enforcement point in front of the query path closes that gap and keeps the access model consistent.

In practice, the proxy or gateway should treat the bearer token as the caller’s proof of access, not as a decorative header. It can validate the token, resolve which tenant or permission set the caller belongs to, and then evaluate whether the requested label selector or query target is permitted before Prometheus answers.

How label-based scoping protects shared observability data

Prometheus queries become safer when the authorization rule is tied to a stable attribute in the data model, such as an enforced tenant label. The proxy can require that label to be present, constrain it to the caller’s allowed scope, or reject queries that try to omit or override it. That keeps the data plane aligned with the permission model instead of relying on user discipline.

This approach is strongest when the tenant or permission boundary is already represented in metric labels or in a query parameter that maps cleanly to an object. If the mapping is ambiguous, the gateway has to make the decision explicit, otherwise a seemingly harmless query can expand into a broader result set than the caller should see.

For teams building authorization around shared telemetry, the important design choice is to make the boundary deterministic. A query should either match the caller’s permitted scope or fail closed. Anything that depends on dashboard conventions, naming discipline, or downstream report trimming is too easy to bypass once users can speak PromQL directly.

What to verify before you trust tenant-aware Prometheus access

Before relying on this model, confirm that the authorization layer sees the same query path that users actually use, including API access, saved dashboards, and automation. If one path reaches Prometheus without passing through the enforcement point, the whole control becomes inconsistent and the bypass path becomes the real access model.

Also verify that the proxy can distinguish between the caller’s identity, the requested tenant scope, and the metric target being queried. If those elements are collapsed too early, you may end up allowing a valid user to read an invalid tenant, or denying legitimate access because the query object was not mapped accurately.

When the enforcement layer is correct, the practical outcome is simple: the user experience still feels like Prometheus, but every response is scoped by policy before the data leaves the backend.

Risk and Threat Considerations

Shared metrics stores are attractive targets because they can reveal operational detail, incident patterns, service dependencies, and sometimes sensitive business signals. If query authorization is weak, a user with ordinary access can exfiltrate data across tenant boundaries by changing selectors, altering labels, or using a raw query path that bypasses the intended permission system.

Failure mechanism: The control fails when the authorization decision is made after query execution, or in a layer that users can bypass. In that case, Prometheus returns data first and policy is applied too late, which creates an exposure window for unauthorized reads.

Impact: Cross-tenant leakage, policy bypass, and broader observability exposure become possible, especially in environments where metrics reveal infrastructure names, workload identifiers, error rates, or incident timing. That can weaken both confidentiality and operational isolation.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

OWASP API Security Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST SP 800-53 Rev 5 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.

FrameworkControl / ReferenceRelevance
OWASP API Security Top 10API1 — Broken Object Level AuthorizationQuery objects and tenant scope must be authorized before results are returned.
Recommendation — Enforce object-level authorization on every Prometheus query and tenant selector.
NIST SP 800-53 Rev 5AC-3 — Access EnforcementThe proxy must enforce access rules before Prometheus returns shared metric data.
IA-5 — Authenticator ManagementBearer token validation is central to identifying the caller before query execution.
Recommendation — Place an enforcement point in front of Prometheus and deny out-of-scope queries. Validate bearer tokens at the gateway before any Prometheus query is processed.
NIST CSF 2.0PR.AA-05 — Identity and Access ManagementTenant-scoped metric access depends on consistent identity and access decisions.
Recommendation — Tie Prometheus query access to identity-based authorization rules.
OWASP Non-Human Identity Top 10NHI-05 — Overprivileged NHIService-to-service query paths and tokens can expose excess metric access if scoped too broadly.
Recommendation — Restrict machine and service access to only the metric scopes they need.

Practitioner Guidance

What to verify: Confirm that every Prometheus access path, including direct API use, goes through the same authorization decision. If any path can reach the backend without token validation and scope enforcement, treat it as a bypass rather than an edge case.

Decision rule: If the tenant or permission boundary is not expressed in a way the proxy can validate deterministically, do not rely on query-time filtering alone. Tighten the data model or the enforcement point first, then allow access.

Practitioner takeaway: The safest design is not “Prometheus plus a filter”, it is “Prometheus behind an enforcing gate” so the access decision is made before the query can expose shared telemetry.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Reviewed and updated by the NHIMG editorial team on November 4, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org