Notifications
Clear all
Topic starter
02/08/2026 12:16 pm
TL;DR: Security teams are collecting far more telemetry than they can use, with medium environments now processing around 50,000 events per second and large ones reaching 1 million, according to Abstract Security. The real governance issue is not just detection quality but whether expensive real-time storage is being used for data that could sit elsewhere without reducing visibility.
NHIMG editorial — based on content published by Abstract Security: Security Shrinking the Stack of Needles
By the numbers:
- Back in the early 2000s, a large SIEM deployment was around 50,000 events per second, maybe five terabytes a day.
Questions worth separating out
Q: How should security teams decide which telemetry belongs in the SIEM?
A: Teams should keep only the telemetry that supports active detection, response, or short-horizon investigation in the SIEM.
Q: Why do IAM and NHI events need special treatment in log pipelines?
A: IAM and NHI events often provide the earliest evidence of abuse because they show who or what accessed a system, when privileges changed, and whether credentials were used unexpectedly.
Q: What breaks when every log is stored in the same high-performance tier?
A: Searches slow down, storage costs rise, and analysts spend more time filtering noise than finding threats.
Practitioner guidance
- Separate detection data from retention data Classify each log source by whether it supports an active detection scenario, then route non-detection data to low-cost storage instead of the SIEM.
- Prioritise identity and privilege telemetry Keep authentication, privilege escalation, token usage, and account activity in the fast analytics path because those events are the most likely to reveal abuse.
- Define ingestion rules by use case Require every new data source to justify the detection or investigation use case it serves before it is added to the real-time stack.
What's in the full article
Abstract Security's full article covers the operational detail this post intentionally leaves for the source:
- How the vendor structures security data pipelines to keep detection data separate from cheaper retention storage
- Practical examples of which log sources are worth keeping in the high-performance analytics tier and which are not
- The logic behind reducing SIEM volume without sacrificing visibility for authentication and privilege monitoring
- The vendor's explanation of the "log lemming effect" and how it changes collection decisions
👉 Read Abstract Security's analysis of shrinking the security telemetry stack →
Telemetry overload in security operations: what should teams change?
Explore further
02/08/2026 1:18 pm
Telemetry sprawl is a governance failure when collection is not tied to a detection outcome. Security teams often justify ingestion with compliance or visibility arguments, but those are not the same as operational need. When the stack expands faster than detection value, analysts pay the cost in search latency, storage expense, and missed signal. The right governance question is not how much can be collected, but which events actually justify real-time treatment. Practitioners should treat collection scope as a control decision, not a default.
A question worth separating out:
Q: How do security teams know whether agent telemetry is actually working?
A: Telemetry is working when it captures tool calls, lifecycle events, and message traces in a way that supports investigation and correlation. If logs omit tool execution or are easy to alter, the control has failed. Durable output with redaction and forwarding to monitoring systems is the practical test.
👉 Read our full editorial: Shrinking the telemetry stack of needles in security operations
