Government SIEM migration shows the cost of fragile log pipelines

At a glance

What this is: This is a government SIEM migration case study showing that unstable log collection, data loss, and hidden coverage gaps can dominate the cost and complexity of a platform move.

Why it matters: It matters because SIEM programmes fail or succeed on pipeline reliability, not just on the target platform, and identity or NHI telemetry can be lost if collection layers are not observable and governed.

By the numbers:

• The organisation cut infrastructure by 85% during the migration.
• The organisation cut log volume by 40% by filtering redundant data.
• AxoRouter handled peaks over 90k EPS on a single node.

Context

A SIEM migration is not just a tooling swap. It depends on whether log collection, normalisation, and forwarding can preserve signal while reducing cost and operational complexity, and that problem becomes harder in hybrid environments with many producers and inconsistent telemetry quality.

This case is also an identity-adjacent governance story because log pipelines increasingly carry evidence from Windows hosts, infrastructure services, and workload or service identities. When those flows drop messages or mask gaps, teams lose confidence in detection, investigation, and compliance reporting before they even reach the new SIEM.

The government organisation described here started from a fragile and underperforming collection layer, which is typical of large environments trying to modernise ingest without first fixing the pipeline itself.

Key questions

Q: What fails when log pipelines drop messages during a SIEM migration?

A: Missing events break confidence in detection, investigation, and reporting because teams can no longer tell whether a gap reflects true inactivity or telemetry loss. In a migration, that means the new SIEM may look healthy while still missing security-relevant data from key sources.

Q: Why does data reduction create risk if the discard policy is not explicit?

A: Because filtering can remove useful evidence as easily as it removes noise. If teams cannot explain what is being dropped and why, they lose the ability to defend detection coverage, replay investigations, or prove that compliance-relevant telemetry still exists in the pipeline.

Q: What are the signs that a log collection layer is undermining SIEM quality?

A: Frequent message drops, inconsistent source visibility, unexplained coverage gaps, and delayed troubleshooting are the main warning signs. When pipeline health is opaque, the SIEM inherits uncertainty and downstream rules cannot be trusted to reflect the real environment.

Q: How do teams know whether a SIEM migration is actually working?

A: They should compare the old and new systems on coverage, latency, alert fidelity, retention, and downstream report accuracy. If the new pipeline cannot reproduce the same investigative and compliance outcomes as the old one, it is not yet ready for full production traffic.

Technical breakdown

Why fragile ingestion breaks SIEM migrations

A SIEM only works as well as the data it receives. If collection tools drop messages, misclassify events, or fail to expose source health, the migration team cannot tell whether missing detections reflect real absence or broken ingestion. In practice, the hardest part is not forwarding logs but preserving their structure, volume discipline, and traceability across heterogeneous sources. Normalisation also matters because downstream analytics depend on consistent event shape, not just raw throughput.

Practical implication: validate ingestion reliability and source observability before moving detection workloads to the new SIEM.

How log normalisation and reduction change detection economics

Log reduction is not the same as data loss. Properly governed filtering removes redundant or low-value events while preserving security-relevant telemetry for detection and investigation. Normalisation then maps different source formats into a common schema so correlation rules can run consistently across Windows, syslog, and other inputs. Without that control layer, cost reduction efforts often become blind pruning, and the organisation pays less but sees less. The governance task is to decide what can be discarded safely and what must stay visible.

Practical implication: define filtering and normalisation rules as security controls, not merely as storage optimisations.

Why observability has become part of security operations governance

Modern pipeline observability means more than uptime monitoring. It includes endpoint metrics, source health, throughput, error rates, and delivery confirmation across every stage of the path from producer to SIEM. That visibility creates operational accountability because teams can prove where data is delayed, malformed, or missing. For hybrid environments, this is becoming a core control plane for security operations, especially where large parts of the estate are managed through services, agents, and machine identities whose telemetry must remain trustworthy.

Practical implication: treat pipeline observability as a prerequisite for dependable detection, compliance evidence, and SIEM migration sign-off.

Threat narrative

Attacker objective: The practical objective of the failure mode is to create blind spots in security monitoring by weakening the integrity and completeness of telemetry.

1. Entry occurs through unreliable collection tooling and fragmented log pipelines that fail to move events consistently from producers to the SIEM.
2. Credential or identity evidence is obscured when pipeline instability hides which sources are healthy, which are misconfigured, and which telemetry streams are incomplete.
3. Impact is operational and governance related: detections miss context, investigations lose confidence, and migration teams inherit blind spots while trying to cut cost.

Breaches seen in the wild

• reviewdog Action compromise 2025: A stolen maintainer token poisoned reviewdog/action-setup, leaking CI secrets including the tj-actions bot token used in the next attack.
• CI/CD pipeline exploitation case study: Credentials in an exposed .git/config let a researcher edit a Bitbucket pipeline so it planted their SSH key on the server. No victim was named.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Pipeline reliability is now a security control, not an infrastructure detail. A SIEM migration that cannot prove complete and timely ingestion is not a governance-ready migration, because the monitoring outcome depends on pipeline integrity before analytics begin. This is especially true in large estates where Windows, syslog, and cloud-adjacent telemetry all travel through different collectors. Practitioners should treat data path assurance as part of security control validation.

Log reduction only helps when the discard policy is explicit. Cutting 40% of volume can be a benefit or a liability depending on whether the removed events were redundant noise or useful evidence. That is why normalisation, classification, and filtering need reviewable rules, not ad hoc tuning. The named concept here is ingestion trust gap: the point at which teams no longer know whether missing data means lower risk or broken collection.

Identity-rich telemetry depends on the same observability discipline as infrastructure logs. Service accounts, workloads, and administrative sessions often surface first in the very logs that pipeline teams are tempted to compress or simplify. When those flows become opaque, identity investigations lose the trail needed for detection and forensics. Practitioners should preserve traceability for identity-bearing events even while reducing total data volume.

Migration success should be measured by control continuity, not by platform cutover speed. A fast move to a new SIEM is only meaningful if source coverage, data quality, and troubleshooting visibility improve at the same time. Government environments in particular need demonstrable evidence that the new stack can scale without masking loss. Practitioners should ask whether the migration reduced risk or merely moved the blind spots.

Cloud and SIEM programmes are converging around telemetry governance. As log pipelines become more automated, the boundary between security engineering and security operations gets thinner. That means teams must design for provenance, classification, and health checks across the full path, or the organisation will optimise cost at the expense of assurance. Practitioners should build pipeline governance into the broader detection architecture.

What this signals

Ingestion trust gap: once teams cannot prove that telemetry arrived intact, every downstream detection decision becomes weaker. That is why SIEM migration planning needs source health, delivery assurance, and coverage checks before cutover, not after the first alert misses.

Pipeline governance is becoming a first-class part of security operations because the organisation now depends on data quality to validate access, detect abuse, and defend compliance evidence. In practice, that means the log path must be observable with the same discipline as the systems it monitors.

For practitioners

• Validate collection reliability before migration Test message loss, source health, and delivery guarantees across Windows, syslog, and other producers before any cutover to the target SIEM.
• Separate cost reduction from security signal loss Define which events can be filtered or normalised away and which telemetry must remain intact for detection, investigation, and compliance evidence.
• Instrument the entire log path Track endpoint metrics, throughput, processing errors, and delivery status from source to SIEM so teams can identify where data quality degrades.
• Preserve identity-bearing telemetry Keep high-value authentication, administrative, and service activity visible even when reducing volume, because those records support investigations and access reviews.

Key takeaways

• The case shows that SIEM migrations fail when the collection layer is fragile, even if the destination platform is technically sound.
• A large reduction in infrastructure and log volume can be useful, but only if teams can prove that security-relevant telemetry still arrives intact.
• Practitioners should treat ingestion reliability and observability as control requirements for any migration, especially in hybrid environments.

Key terms

• Observability Pipeline: An observability pipeline is the layer that collects, shapes, and routes telemetry before it reaches storage or analytics tools. In security operations, it determines which records are preserved, transformed, enriched, anonymized, or dropped, and therefore strongly influences both detection quality and auditability.
• Telemetry Normalization: Telemetry normalization is the process of turning data from different security tools into a consistent format that can support one policy decision. It is essential when identity, endpoint, and asset systems all feed the same control plane, because conflicting data can otherwise create gaps or overblocking.
• Telemetry Optimisation: Telemetry optimisation is the practice of reducing low value data while preserving the signals needed for operations, troubleshooting, and reliability. In modern pipelines, it focuses on filtering noise, controlling ingestion growth, and improving signal to noise ratio so observability remains useful rather than expensive.
• Ingestion Trust Gap: The point at which teams can no longer prove that telemetry arrived intact, on time, and from the expected sources. Once that gap appears, every downstream security decision becomes less certain because the monitoring foundation is no longer verifiable.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. Explore the NHI Foundation Level course if your programme needs sharper control over machine identities, secrets, and access governance.