Teams should usually build log-based metrics closer to the edge, inside an OpenTelemetry-backed telemetry pipeline, when they want portability and lower long-term lock-in. Doing the transformation in a backend can work, but it often ties custom logic to one platform and makes migration harder. Build near ingestion when you need reusable metrics, lower log volume, and easier platform changes.
Build log-based metrics at ingestion when the goal is portability
For most telemetry pipelines, the best place to build log-based metrics is as close to ingestion as possible, ideally in an OpenTelemetry-backed telemetry pipeline rather than inside a proprietary backend. That placement keeps the transformation closer to the raw event stream, so the metric definition can travel with the pipeline instead of being trapped in one observability product. It also makes the metric easier to reuse across teams and environments.
That matters most when the same log signal needs to support multiple consumers, dashboards, or alert rules. If you build the metric in the backend, the logic often becomes platform-specific, which means a later migration can silently change meaning, labels, or aggregation behavior. Near-edge transformation reduces that coupling and makes the metric model part of the pipeline design, not an afterthought.
When backend metrics still make sense
Backend-side log-to-metric conversion is still reasonable when the metric is tightly bound to one platform feature, one retention model, or one team-owned dashboard layer. If the transformation is simple, disposable, or only needed for a narrow use case, the convenience of keeping it in the backend can outweigh the portability benefit. The trade-off is that you are accepting more lock-in and less control over how the metric evolves.
A practical rule is to ask whether the metric is a pipeline concern or a product concern. If it expresses a business or operational signal that should remain stable across tooling changes, build it upstream. If it is merely a presentation convenience for one observability stack, backend generation is easier to live with, but the team should treat it as platform-dependent logic and document that dependency explicitly.
Examples of upstream-friendly metrics include counters and ratios derived from repeated log patterns, especially when the log volume is high and the raw stream is too noisy to keep at full fidelity. In those cases, conversion near ingestion can lower storage and query cost while preserving the signal you actually need. A compact metric also tends to be easier to alert on than a search-heavy log query that re-parses the same events over and over.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
CIS Controls v8 and NIST CSF 2.0 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| CIS Controls v8 | 8 — Audit Log Management | Log-based metrics depend on log collection and reduction from audit data. |
| Recommendation — Define log-derived metrics upstream so audit data remains usable across tools and retention changes. | ||
| NIST CSF 2.0 | DE.CM — Continuous Monitoring | Telemetry pipelines support ongoing detection and measurement from log signals. |
| GV.1 — Cybersecurity Risk Management Strategy | Choosing pipeline placement is a governance decision about lock-in and control durability. | |
| Recommendation — Place log-to-metric logic where it best preserves continuous monitoring across platforms. Treat metric placement as a governance choice that balances portability, reuse, and vendor dependency. | ||
Practitioner Guidance
What to prioritise: Start with the metric’s intended lifespan and portability requirement. If you expect the pipeline, vendor, or storage backend to change, place the transformation upstream so the definition survives that change.
What to verify: Confirm whether the metric depends on backend-specific parsing, enrichment, or aggregation semantics. If it does, document that dependency and decide whether losing that behavior during migration would be acceptable.
Common mistake: Teams often build log-based metrics where they are easiest to see today, then discover later that every alert, dashboard, and threshold is coupled to one observability platform.
Practitioner takeaway: Build near ingestion when the metric is meant to be durable infrastructure; keep it in the backend only when you are comfortable trading portability for short-term convenience.
Related resources from NHI Mgmt Group
- How should security teams decide where to use OCSF in a telemetry pipeline?
- How should security teams size log pipeline windows for high-volume NHI telemetry?
- How should security and observability teams standardize telemetry pipelines across multiple log sources and destinations?
- How should security teams decide what telemetry to send into a cloud SIEM when ingest-based pricing is in place?
