How should security teams reduce the cost of agent telemetry without losing detection coverage?

Why pipeline-native detection cuts telemetry cost without blinding the SOC

The cost problem is usually not the telemetry itself, it is the habit of treating every event as SIEM-worthy. Agent pipelines produce high-volume, low-signal logs, traces, and tool calls. If teams push all of it into the SIEM, they pay to ingest noise, pay again to store it, and still struggle to distinguish routine agent chatter from meaningful security findings.

The more efficient pattern is to separate evidence retention from detection routing. Keep the full event stream in cheaper storage for forensics and replay, then run rules, correlation, and anomaly checks earlier in the pipeline so only validated findings and high-value alerts continue downstream. That reduces ingest pressure without giving up coverage.

Pipeline-native detection works because most coverage value comes from a smaller set of security-relevant states, not from every raw event. For agent telemetry, those states often include failed tool authorization, unexpected privilege changes, repeated retries, unusual destination systems, secret access attempts, and actions that exceed the agent’s normal operating pattern. When detections are tied to those states, the SOC receives less volume but more context.

What to keep in the data lake and what to forward

A useful split is to treat the data lake as the system of record and the SIEM as the system of action. Raw OpenTelemetry, execution traces, prompts, tool invocations, and intermediate decisions stay available for investigation, but the SIEM only receives normalized findings, risk-scored events, and incidents that need analyst attention. That preserves chain-of-custody style evidence without paying SIEM rates for every record.

The split only works if the upstream pipeline keeps enough fidelity to reconstruct what happened. Teams should preserve timestamps, correlation identifiers, actor context, tool targets, and enough payload metadata to explain a detection later. If the pipeline strips away those fields, you may reduce cost but also weaken incident triage and post-incident analysis.

Filtering should be based on security relevance, not just on volume. A noisy but ordinary health check can stay out of the SIEM, while a single denied access to a privileged tool, a rare cross-environment call, or a burst of failed lookups may deserve forwarding immediately. The design goal is not fewer events, it is fewer irrelevant events.

How to preserve detection coverage while lowering ingest cost

Coverage is maintained by moving some analytic logic closer to collection. Lightweight parsing, enrichment, suppression, and thresholding can happen before events ever reach the SIEM. More expensive correlation can run in the pipeline or in a separate analytics tier, where storage and compute are cheaper and tuning is easier.

Teams should map each detection to its required inputs before deciding where it should run. If a rule only needs a single event and a few context fields, it can usually run early. If it depends on long-range correlation, historical baselines, or cross-system joins, it may still belong in the SIEM or a lakehouse analytics layer. This avoids forcing every use case into one expensive destination.

It also helps to distinguish alert fidelity from event fidelity. You do not need every raw event in the SIEM to preserve fidelity if the pipeline can emit a stable finding that includes the original evidence pointer. The analyst can still pivot into the lake when deeper inspection is needed.

Risk and Threat Considerations

Cost optimisation can become a control failure if teams over-filter before they understand which telemetry signals actually reveal abuse. The main risk is blind spots created by suppression rules that look harmless at low volume but remove the exact event sequences needed to spot credential abuse, privilege escalation, or destructive agent actions.

Failure mechanism: Detection logic is pushed too far downstream, or suppression is applied without measuring what patterns disappear from the remaining event set. Attackers then benefit from cheaper telemetry because their activity is recorded, but not elevated into actionable alerts.

Impact: Teams lower storage and ingest spend while increasing mean time to detect, reducing confidence in investigations, and risking missed compromise when the agent takes a high-impact action only once.

Practitioner Guidance

What to prioritise: Protect the few telemetry fields and event transitions that actually drive detection, then move everything else toward lower-cost retention. If a field is needed only for later investigation, it belongs in the lake first, not the SIEM by default.

What to verify: Every suppression rule should be tested against a representative incident set and against the detections it could affect. The question is whether an analyst could still explain and validate the finding, not whether the raw event still exists somewhere.

What practitioners underestimate: The hidden cost is often not ingestion alone, but duplicated parsing, index expansion, and analyst fatigue from low-value alerts. A good design reduces all three by making the pipeline responsible for triage quality, while the SIEM remains the place for actionable security output.

Practitioner takeaway: The right optimisation is selective forwarding, not blind reduction. Preserve the raw record for evidence, but let the pipeline decide what deserves analyst attention and SIEM spend.

Related resources from NHI Mgmt Group

• How should security teams reduce the cost of ingesting noisy AWS GuardDuty logs into a SIEM without losing useful detection coverage?
• What should teams do first when they need to reduce security pipeline cost without losing coverage?
• How should security teams reduce the cost of inline AI guardrails without losing detection quality?
• How should security teams monitor AI agent activity without disrupting developers?