What is the difference between received, filtered, parsed, and output log metrics in Kubernetes observability?

Received metrics count how many events entered the collector, filtered metrics show what was removed, parsed metrics track how much content was successfully interpreted, and output metrics show what was delivered onward. Together, they describe the full log pipeline. Operators use the separation to distinguish collection issues from parsing problems and from downstream delivery failures.

What each log metric tells you about the pipeline

These four metrics describe different stages of the same log flow, so they answer different operational questions. Received metrics tell you whether events are entering the collector at all, filtered metrics show what was deliberately dropped or excluded, parsed metrics show whether the collector could understand the log content, and output metrics show whether records were successfully handed off to the next destination.

The practical value is that the metrics separate volume, selection, interpretation, and delivery. That makes them useful for finding where a failure is happening without assuming every missing log means the source stopped sending data.

In kubernetes observability, that distinction matters because collection pipelines often sit between many moving parts, including node agents, sidecars, log processors, and downstream stores. A healthy received count with weak parsed or output counts usually points to processing or transport issues, not source silence.

For a closer look at the container and collector side of that pipeline, NIST’s NIST SP 800-190 Container Security is a useful reference point for how containerized environments change the logging and monitoring problem.

How to read the gaps between the metrics

A high received count with a high filtered count usually means the pipeline is seeing data but is intentionally discarding it because of routing rules, noise suppression, or scope filters. That is not necessarily a fault, but it becomes a problem when the filtering rules are broader than intended.

If received stays high while parsed drops, the collector is getting logs that it cannot interpret reliably. Common causes are format drift, multiline records that are not assembled correctly, or mismatched parsers after application changes. If parsed looks healthy but output falls, the issue is usually downstream delivery, such as backpressure, retry failures, destination limits, or transport disruption.

Seen together, the metrics let operators isolate whether the break is at ingestion, transformation, or egress. That is especially important in Kubernetes because logs can move through multiple layers before they ever reach a central platform.

For the broader control and hardening context around those pipeline stages, CIS Benchmarks provide a practical baseline for reducing configuration drift in the systems that host log collectors and log-forwarding components.

Risk and Threat Considerations

Log metrics are operational signals, but they also expose whether your observability pipeline is blind, noisy, or partially broken. A sustained gap between received, parsed, and output counts can hide security events, delay investigations, or make it look as though evidence is available when it never reached the destination.

Failure mechanism: malformed records, bad parsing rules, over-aggressive filtering, or downstream delivery failures can silently remove telemetry at the exact point where investigators expect completeness.

Impact: teams may miss container abuse, lateral movement, or application faults, and incident response can be slowed because the pipeline no longer provides a trustworthy view of what actually happened.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

CIS Controls v8, NIST CSF 2.0 and NIST IR 8596 set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
CIS Controls v8	8 — Audit Log Management	Log pipeline metrics support reliable audit logging and collection integrity.
	12 — Network Infrastructure Management	Collector delivery depends on stable network and endpoint paths in Kubernetes.
Recommendation — Monitor log collection and delivery gaps to preserve audit log completeness and integrity. Validate network paths and service endpoints when output metrics fall behind received metrics.
NIST CSF 2.0	DE.CM — Continuous Monitoring	The metrics are monitoring signals that show where telemetry is failing in the pipeline.
	RC.RP — Response Plan Execution	Metric gaps guide the first response step during observability degradation.
	PR.PT — Protective Technology	Collector configuration and parsing rules are protective controls around log flow.
Recommendation — Use continuous monitoring to detect breaks between ingestion, parsing, filtering, and delivery. Use the metric split to prioritize whether the incident is source, parser, or downstream related. Harden log-processing components so parsing and forwarding failures are less likely.
NIST IR 8596	GOVERN — Cyber AI Governance	No

Practitioner Guidance

What to verify: Compare the metric deltas over the same time window, not as isolated counters. If received is steady but parsed or output declines, treat the collector configuration, parser definition, and downstream endpoint health as the first places to inspect.

Decision rule: If the source application changed its log format, prioritise parser validation before assuming transport failure; if parsed is stable but output drops, focus on destination availability, throttling, or retry behavior.

What good looks like: the relationship between received, filtered, parsed, and output remains stable enough that a sudden divergence is easy to explain, and each drop can be tied to a known rule, format change, or delivery issue.

Practitioner takeaway: The useful insight is not the raw count itself, but where the counts diverge, because that tells you whether you are dealing with intentional filtering, parsing failure, or a broken handoff.