Ship the logic, not the data for security telemetry workflows

At a glance

What this is: This is an analysis of why security telemetry architecture should follow workload needs, with the key finding that detection, hunting, and investigation should not all run in the same place.

Why it matters: It matters because IAM, NHI, and security operations teams increasingly depend on telemetry-driven decisions, and the wrong execution model can break latency-sensitive controls, investigation depth, and evidence integrity.

By the numbers:

• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read AiStrike's analysis of why security architecture should follow the workload

Context

Security telemetry architecture is no longer just a storage decision. Detection rules, hunt queries, and investigation workflows have different time horizons, state requirements, and evidence needs, so placing all of them in the same execution model creates avoidable control gaps.

The article argues that the old assumption, that telemetry should be centralized first and analysed later, is too narrow for modern environments. That matters for IAM and NHI programmes because identity signals, workload events, and agent activity often need different handling depending on whether the task is continuous detection, retrospective search, or automated investigation.

As security operations become more agent-driven, the distinction becomes sharper rather than weaker. Centralized history still matters, but the workload now determines where logic belongs, which is the more durable design principle.

Key questions

Q: How should security teams decide where detection logic should run?

A: Place detection logic where its timing, state, and dependency requirements can be met. Streaming rules belong close to the event source when milliseconds or seconds matter, while retrospective hunts can run against centralized history. The deciding factors are latency tolerance, statefulness, and whether the workload needs continuous evaluation or historical breadth.

Q: Why does federation create risk for security investigations?

A: Federation makes investigations depend on source-system retention, availability, schema consistency, and API behavior. That is acceptable when the data is constrained or rarely queried, but it becomes risky when teams assume federated sources provide complete evidence. The failure mode is not just slowness. It is incomplete or misleading answers that still look valid.

Q: What breaks when streaming detections are treated like search queries?

A: Polling creates gaps, overlapping windows, duplicate evaluations, and delayed responses that undermine continuous detection. A streaming detector needs stateful evaluation as events arrive, not repeated snapshots of a changing dataset. When teams misuse search for detection, they often lose the very timing guarantees the control was supposed to provide.

Q: How do security teams balance centralized history with federated access?

A: Use centralized indexed storage for datasets that are practical to collect, normalize, and retain, then reserve federation for sources that cannot be exported well or should stay in place for legal, technical, or operational reasons. The right model is usually mixed, with each workload matched to its own execution point.

Technical breakdown

Why detection logic is not the same as search

Detection and search use the same data, but they do not use it in the same way. A detection rule is usually pre-defined, stateful, and latency-sensitive, which means it needs to evaluate a stream continuously and often produce a verdict in seconds or milliseconds. Search is usually retrospective, hypothesis-driven, and tolerant of slower execution. That difference is architectural, not just operational. If teams force both into the same execution path, they inherit the worst properties of each model: polling gaps for detection and unnecessary coordination overhead for search.

Practical implication: Place streaming detection near the event source when verdict timing matters more than broad historical reach.

How federation changes the control plane for telemetry

Federation keeps telemetry in source systems and queries it where it lives, which is attractive when data cannot be moved or exported cheaply. The trade-off is that the query must now depend on each source's retention, schema, rate limits, availability, and API behavior. That means the security system inherits other platforms' operational constraints as part of its own control plane. For retrospective hunting, that can be acceptable. For high-frequency detection or investigation automation, it can create fragile dependency chains and inconsistent results.

Practical implication: Use federation selectively for constrained data sources, not as the default architecture for all security analytics.

Why agents make telemetry architecture more sensitive

Autonomous or semi-autonomous investigation agents can generate far more queries than human analysts, and that changes the economics of telemetry placement. In a federated model, every branch in an investigation can trigger repeated remote calls, quota pressure, and cost spikes. In a centralized model, the organization controls the compute path and can reason about capacity internally. The important design point is that agents amplify whatever architectural weakness already exists, especially where query volume and external dependencies are part of the workflow.

Practical implication: Account for agent-driven query amplification when deciding whether telemetry should be centralized, federated, or split by workload.

Threat narrative

Attacker objective: The practical objective is not a breach in the classic sense but reduced detection fidelity, slower response, and less trustworthy investigative evidence.

1. Entry occurs when a security workload is forced into the wrong execution path, such as treating a streaming detector like a retrospective query.
2. Escalation follows as latency, polling gaps, API limits, or schema drift reduce the system's ability to preserve state and evaluate events correctly.
3. Impact is delayed or incomplete detection, weaker evidence reconstruction, and investigation workloads that become expensive or operationally fragile.

NHI Mgmt Group analysis

Security telemetry architecture is becoming a workload-placement problem, not a storage problem. The old model assumed that once data was centralized, analysis should follow automatically. This article shows that detection, hunting, and investigation have different execution requirements, so governance now has to distinguish between where data lives and where logic should run. For identity programmes, that means workload identity, access review, and security monitoring cannot all be treated as the same analytical class.

Agentic automation will expose weak telemetry architectures faster than human-led operations. When software can branch into hundreds of queries, every API limit, retention gap, and schema inconsistency becomes part of the operating model. That creates a governance burden across SOC, IAM, and cloud teams because the control plane now includes query throughput as well as data quality. Practitioners should expect more pressure on systems that were designed for human-scale investigation.

Indexed historical storage is a control enabler, but not a universal answer. Cheap object storage and high compression reduce the economic argument for leaving everything federated, yet they do not remove the need for selective federation. The useful concept here is execution-plane separation: retain history where it is economical, but move detection logic toward the event source when latency and state matter. That is a stronger operating model for both security operations and identity-driven monitoring.

Identity and non-human identity signals benefit from the same architectural split as other telemetry. Authentication events, service-account activity, and agent actions often need continuous evaluation, while older records support forensic review. If those workloads share a single execution path, teams either overpay for real-time response or underperform on historical analysis. The better discipline is to map each identity control to its natural tempo and place the logic accordingly.

Security teams should treat data locality as a dependency, not a design principle. The article correctly reframes centralization versus federation as an implementation choice, not a strategy. That aligns with modern governance thinking in NIST CSF and zero trust architecture, where the control objective is reliable verification and response rather than simply collecting everything in one place. Practitioners should design for answer quality, not storage convenience.

What this signals

Execution-plane separation will matter more as security teams operationalise autonomous investigation and continuous detection. The more queries software can generate, the more expensive it becomes to rely on federated dependencies as the default path for every analytic decision. Practitioners should expect programme designs to favour near-source evaluation for latency-sensitive work and centralized history for retrospective analysis.

Identity and non-human identity telemetry will increasingly follow the same design rule. Authentication events, service-account activity, and workload signals need different handling depending on whether the control objective is immediate containment or later forensic review. The stronger programme model is to align telemetry placement with the tempo of the control, not with the convenience of the storage tier.

This is a good place to revisit identity lifecycle governance alongside detection architecture, because telemetry and lifecycle controls reinforce each other. See the NHI Lifecycle Management Guide for how provisioning, rotation, and offboarding reduce the evidence gaps that analytics must later compensate for. For architecture context, the NIST Cybersecurity Framework 2.0 remains a useful way to anchor detection, response, and recovery as distinct operational outcomes.

For practitioners

• Separate streaming detections from retrospective hunts Classify each security use case by latency tolerance, statefulness, and evidence depth, then place streaming detections near the event source and reserve centralized history for retrospective search.
• Map federated dependencies into the security control plane Inventory source-system retention, schema behavior, API quotas, and export limits before allowing federation to support investigations or detections.
• Adopt execution-plane separation for identity telemetry Keep authentication, service-account, and agent activity signals available for local or near-source evaluation when response timing matters, while retaining searchable history centrally where practical.
• Design for agent-driven query amplification Assume investigation agents will generate more remote queries than human analysts and test capacity, quota handling, and cost impact under branch-heavy workflows.

Key takeaways

• Security analytics should be placed by workload, not by storage convenience.
• Streaming detection, retrospective hunting, and autonomous investigation have different latency and state requirements.
• Agentic query growth makes execution-plane design a control issue, not just a performance issue.

Key terms

• Streaming Detection: Streaming detection evaluates events while they are still in motion through the pipeline instead of waiting for them to be indexed in a SIEM. That reduces latency, shortens mean time to detect, and can make alerting less dependent on a single downstream analytics platform.
• Federated Query Execution: The process of running a single investigative hypothesis across multiple data sources at the same time. In security operations, this matters because attacker behaviour is often only visible when SIEM, EDR, identity, and threat intelligence data are correlated together.
• Control-Plane Separation: The architectural separation between the system that runs a workload and the system that authorises, observes, and can stop it. For AI agents, this prevents the workload from granting itself more access, altering policy, or claiming the environment is healthy when it is not.
• Query Amplification: Query amplification is the increase in remote searches, API calls, and computational load caused by automated or branching investigations. In security operations, it can turn incident response into a capacity and quota problem if the analytics architecture was designed for human-scale use.

What's in the full article

AiStrike's full blog covers the operational detail this post intentionally leaves for the source:

• The article's deeper comparison of centralization, federation, and hyper-indexed object storage across security workloads.
• Practical examples of where detection should execute at the event source versus where historical hunts should run.
• The discussion of how agent-driven investigation changes query volume, cost, and external dependency management.
• The operational failure modes of each architecture, including completeness gaps, parser issues, and API limitations.

👉 AiStrike's full post covers the storage models, execution trade-offs, and failure modes in more detail.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps security and identity teams align lifecycle controls with broader operational resilience.