Why do identity and directory logs become risky when full fidelity ingestion is too expensive?

Why This Matters for Security Teams

When identity and directory logs are too expensive to ingest at full fidelity, teams often reduce volume by filtering, sampling, or dropping fields. That may keep the SIEM bill under control, but it also weakens one of the few telemetry sources that shows who requested access, which group changed, and whether privilege shifted unexpectedly. NHI Mgmt Group notes that only 5.7% of organisations have full visibility into their service accounts in its Ultimate Guide to NHIs, which makes selective ingestion especially dangerous.

The operational risk is not just missing alerts. It is false confidence: analysts may believe coverage exists across identity events, while the highest-risk actions are never retained. That matters for directory changes, token issuance, group membership updates, and service account activity, all of which can precede lateral movement or privilege escalation. Current guidance suggests identity telemetry should be treated as control evidence, not optional observability. In practice, many security teams discover coverage gaps only after an account has already been abused, rather than through intentional validation.

How It Works in Practice

The practical challenge is that identity logs are high value, high volume, and often highly repetitive. Directory platforms can emit successful and failed sign-ins, audit changes, administrative actions, replication events, and application-driven lookups. When ingestion budgets are tight, teams commonly preserve only failures, only admin actions, or only one source system. That reduces spend, but it also breaks event chaining, because investigations often depend on seeing the sequence across identity provider, directory, endpoint, and downstream application logs.

Good practice is to decide what must be kept at full fidelity based on detection use cases, not raw storage appetite. For example, changes to privileged groups, conditional access policy updates, token minting, and service account lifecycle events usually deserve higher retention and faster routing than routine lookups. NIST’s NIST Cybersecurity Framework 2.0 supports this kind of risk-based coverage planning, and 52 NHI Breaches Analysis shows how identity abuse often becomes visible only when telemetry is rich enough to reconstruct the path.

• Keep high-signal identity events unfiltered, especially privilege changes and token issuance.
• Use tiered retention, with longer windows for directory audits than for low-value access noise.
• Validate that sampled streams still support detection logic and forensic reconstruction.
• Test coverage assumptions by replaying known attack paths through the logging pipeline.

This guidance tends to break down in hybrid environments with multiple identity providers and inconsistent audit schemas, because the same event may be normalized away before analysts ever see the context needed to detect abuse.

Common Variations and Edge Cases

Tighter logging controls often lower cost, but they also increase the chance that important identity evidence is lost, so organisations have to balance retention against budget and query performance. Best practice is evolving on where to place the boundary between searchable telemetry and archived evidence, and there is no universal standard for this yet. The right answer depends on whether the identity system is used for humans, service accounts, or both.

One common edge case is directory synchronization. Events may appear duplicated, delayed, or partially transformed as they move from cloud identity providers into SIEM pipelines. Another is delegated administration, where a small number of administrator actions can trigger broad downstream effects but are easy to miss if only the final change event is kept. Organisations that rely on NIST SP 800-53 Rev 5 Security and Privacy Controls should map logging retention to auditability requirements, not to the cheapest storage tier. Where identity data is tied to NHIs, the Top 10 NHI Issues also makes clear that visibility gaps compound rotation and offboarding failures. The hardest environments are multi-cloud estates with separate retention rules, because no single log source captures the whole identity story.

Related resources from NHI Mgmt Group

• When does kernel-level workload identity enforcement become too risky?
• When does near-certainty in identity verification become too risky?
• When does scripting become too risky for identity customization?
• When does manual identity governance become too risky for growing organisations?