Building metrics in a backend means the platform handles the transformation after ingestion, which can be convenient but often increases cost and creates vendor-specific logic. Building them in the telemetry pipeline processes data earlier, closer to the source, so the metric design is easier to carry forward. That approach usually improves portability and reduces reliance on one backend.
Why backend-built and pipeline-built log metrics behave differently
Building log-based metrics in the observability backend usually means the backend ingests raw events first and then derives metrics later. That makes setup convenient, but the metric logic lives inside one platform’s rules, query language, and billing model. Building them in the telemetry pipeline moves the transformation earlier, so the metric is shaped before it reaches the backend and can travel more cleanly across tools.
The practical difference is where you pay for computation and where you lock in semantics. Backend-built metrics are often easier to prototype because the observability platform already has the logs and the transformation engine, but each saved rule becomes part of that backend’s operating model. Pipeline-built metrics require more upfront design, yet they usually produce a more portable signal because the transformation is handled upstream of any single destination.
That distinction matters most when the same log source must serve multiple consumers. A metric created in the backend can be efficient for a single team or a single dashboard set, but it can also create duplication if another platform needs the same measurement. A pipeline-created metric is better when you want one normalized definition that feeds several systems, or when you want to preserve the logic even if you switch observability vendors.
Where the trade-offs show up in practice
Backend creation tends to shift cost toward the observability platform and can encourage convenience-first metric design. You can iterate quickly, but you are also depending on that backend to keep the transformation logic available, consistent, and affordable as volume grows. Pipeline creation shifts some of that effort into your collection or processing layer, which can improve control but demands clearer ownership and more disciplined schema design.
Portability is the other major trade-off. A backend metric may be easy to query and visualize, yet hard to reproduce exactly elsewhere because the definition is embedded in a platform-specific feature. A pipeline metric is more likely to remain useful across destinations because the event reduction or aggregation happens before storage, which makes the definition easier to standardize across environments, teams, or vendor changes.
If your logs already contain stable fields and a small number of high-value counters, backend metrics can be the fastest path to useful visibility. If you are trying to establish a reusable signal for multiple tools, long-lived reporting, or migration resilience, the pipeline is usually the stronger design choice. That is especially true when the metric definition itself is part of the operational contract rather than just a dashboard convenience.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| CIS Controls v8 | 6 — Access Control Management | Standardize reusable metric logic to reduce platform-specific dependence. |
| 8 — Audit Log Management | Log-derived metrics are a reporting layer built from audit data and should remain consistent. | |
| Recommendation — Define metric transformations upstream so observability consumers receive consistent, reusable signals. Preserve auditable metric definitions and ensure derived signals remain traceable to source logs. | ||
| NIST CSF 2.0 | GV.1 — Govern | Metric location affects ownership, consistency, and technology dependence. |
| ID.AM — Asset Management | Telemetry transformations are part of the observable data pipeline that must be inventoried. | |
| Recommendation — Assign ownership for derived telemetry logic and review portability risk across platforms. Inventory where log-to-metric transformations occur so teams know which system owns each definition. | ||
Practitioner Guidance
What to verify: Check whether the metric definition must be reused outside one observability backend. If yes, treat pipeline construction as the default and use the backend only for narrow, team-specific views or rapid experimentation.
Decision rule: If the metric is likely to drive alerting, reporting, or cross-platform analysis, define it upstream and keep the backend as a consumer. If it is only for a single dashboard and the volume is modest, backend creation may be acceptable.
Common mistake: Teams often prototype in the backend and never formalize the metric elsewhere, which makes the measure expensive to port and hard to govern later. The hidden cost is not just compute, it is semantic drift when another system tries to recreate the same signal.
Practitioner takeaway: Choose the backend when speed and convenience matter most, but choose the pipeline when consistency, portability, and long-term ownership matter more than ease of setup.
Related resources from NHI Mgmt Group
- How should security and observability teams decide where to build log-based metrics in a telemetry pipeline?
- What is the difference between an observability pipeline and a telemetry pipeline?
- What is the difference between traces and metrics in OpenTelemetry-based observability?
- What is the difference between log processing and log analytics in a modern observability pipeline?
