TL;DR: Its query engine and compute model split SOC work across ingest, worker and aggregation nodes, with each search planned at query time and executed in one of four shapes, according to LogSeam. The architectural point is that bursty investigation workloads need elastic, local execution, not a monolithic lake engine tuned for batch joins.
At a glance
What this is: LogSeam describes a search architecture that separates ingest, worker and aggregation compute, then plans each query dynamically so SOC workloads can scale without a fixed cluster shape.
Why it matters: This matters because identity, detection and investigation teams need compute patterns that match bursty search demand, especially when privileged access, AI-driven queries or high-volume telemetry force rapid, short-lived processing.
Context
The core problem is not raw storage capacity but how security search behaves under load. SOC queries are bursty, uneven and often short-range, which makes batch-oriented lake engines a poor fit when the goal is fast investigation rather than large-scale data processing. Where identity, NHI and agentic AI workflows are involved, that pressure increases because access review, hunt and response tasks can arrive in rapid bursts and need immediate compute.
LogSeam frames the answer as a managed, per-customer environment with separate ingest and worker layers plus a controller that schedules jobs and shapes each query. That design is relevant beyond one platform because it highlights a broader governance issue: search performance is now a control-plane problem as much as a storage problem. For teams running identity-heavy or telemetry-heavy environments, the architectural question is whether the system can adapt to the workload or forces the workload to adapt to the system.
Key questions
Q: How should security teams design search infrastructure for bursty SOC workloads?
A: They should optimise for many small, time-sensitive queries rather than for one large batch pipeline. That means separating ingest, search and aggregation responsibilities, using elastic compute for spikes, and planning execution so the platform can keep recent data close to the reader without forcing every search through a shared cluster bottleneck.
Q: Why do lake engines often struggle with SOC investigations?
A: Because they are usually tuned for large, preplanned data transformations with heavy shuffling between stages. SOC work is more fragmented, more urgent and more varied in time range, so an engine that depends on cluster-sized planning can create queueing, latency and unnecessary cost when demand spikes.
Q: What are the signs that search planning is not matching workload shape?
A: Long queue times, expensive cross-node movement, poor performance on small investigative queries and overprovisioning for rare deep searches all suggest a mismatch. If the platform cannot distinguish between everyday searches and heavy outlier jobs, it is probably using the wrong execution model.
Q: Should teams prefer isolated per-customer environments or shared services for security analytics?
A: When the workload is sensitive, high-volume and operationally time-bound, isolated environments usually simplify control boundaries and limit blast radius. Shared services can improve central efficiency, but they also make access scoping, troubleshooting and failure containment harder when the platform is responsible for both data and response workflows.
Technical breakdown
Why bursty SOC search does not behave like batch analytics
Security search is dominated by many small investigations, not a few giant transformations. In a SOC, most queries target recent data, run under time pressure and vary sharply in cost, which makes generic data-lake engines inefficient if they expect long-lived clusters, all-to-all shuffles and heavy upfront tuning. LogSeam's model reflects a different assumption: the query planner must decide where work runs, how much of it is distributed and which parts stay local to the node that read the data.
Practical implication: size search compute for burst handling and query planning, not for static cluster utilisation.
What ingest, worker and aggregation nodes each do
The architecture separates responsibilities across node types. Ingest nodes process logs as they arrive, worker nodes run searches, aggregations, AI tasks and scheduled jobs, and aggregation nodes handle heavier merge operations and memory-intensive SQL patterns. That separation matters because it keeps the hot path narrow and lets the system give each phase a different compute profile. When compute and storage are disentangled, the platform can grow one layer without forcing every layer to scale together.
Practical implication: map each workload class to a distinct compute tier so search, ingestion and merge operations do not compete for the same resources.
How query planning and local execution reduce cross-node overhead
LogSeam's planner chooses one of four query shapes and keeps work on the node that read the data whenever possible. Each node runs a vectorized, columnar SQL engine against Parquet in object storage, then spills to local NVMe if memory is exceeded. The key architectural trade-off is avoiding node-to-node shuffle except for merged partial results. That reduces network churn and supports short-lived dynamic nodes that can join, contribute and disappear without preserving state.
Practical implication: prefer execution models that minimise shuffles and tolerate ephemeral compute, especially for elastic security workloads.
NHI Mgmt Group analysis
Elastic query execution is now a governance issue, not just a performance feature. The article shows that security search increasingly depends on whether the platform can shape compute around workload, rather than whether it can simply store more data. That matters because investigation latency becomes part of operational risk when teams need answers during active incidents or access reviews. For identity-heavy programmes, the lesson is that the search layer must be designed for bursty, task-scoped work, not static capacity planning.
Distributed lake engines are often optimised for the wrong security problem. Spark, Trino and similar systems are strong when the task is large, preplanned and shufflable across a cluster. SOC telemetry and AI-assisted investigation are different: they produce many small queries, varying time windows and highly uneven demand. The named concept here is query-shape governance, meaning the planner has to decide the execution form before the workload starts to fan out. Practitioners should treat that as a control point, not an implementation detail.
Identity and AI workloads amplify the need for local, short-lived compute. When AI tools issue many small questions and human analysts chase privilege, access or anomaly trails, the platform cannot assume one query pattern. The article's model aligns with broader identity security reality: the right question often depends on state at the moment of execution. Practitioners should evaluate whether their telemetry and identity systems can support ephemeral, context-aware search without overcommitting permanent infrastructure.
Per-customer isolation and state minimisation are doing more than protecting performance. A single environment per customer with no shared services reduces blast radius and simplifies operational boundaries. That same pattern is relevant to identity governance because shared control planes can blur responsibility, make debugging harder and widen the impact of mis-scoped access. Practitioners should push for workload boundaries that are explicit, isolated and easy to reason about.
What this signals
Query-shape governance: security teams should start evaluating whether their platforms can choose execution form dynamically, because fixed cluster assumptions break down when investigations arrive in bursts. In practice, the question is not whether a system can store telemetry, but whether it can adapt the work plan to the query at the moment it is asked.
When identity, AI and SOC workflows converge, execution locality matters as much as raw scale. Systems that keep work close to the reader and minimise node-to-node movement are better aligned to short-lived investigations, access reviews and AI-driven question bursts than architectures built around all-to-all shuffle.
The broader signal is that telemetry platforms are moving toward control-plane intelligence, where planning, placement and elasticity are part of the security outcome. Teams should expect architecture reviews to focus more on workload shape, node isolation and ephemeral compute than on storage capacity alone.
For practitioners
- Define workload classes before sizing search infrastructure Separate recent investigative searches, heavy aggregations and scheduled jobs so each can be mapped to the right compute profile and failure domain.
- Design for ephemeral compute Treat dynamic nodes as disposable and keep no state on them that must survive rebalancing, reclamation or node replacement.
- Reduce cross-node shuffle dependence Prefer execution paths that keep work local to the reader and only move merged partials across the network.
- Align AI querying with query planning Make sure AI-assisted search tools send narrow, context-aware questions that the planner can shape without forcing full-cluster fan-out.
Key takeaways
- SOC search works best when compute is shaped around bursty investigation demand rather than around static cluster assumptions.
- The article's core design choice is to keep ingest, search and aggregation responsibilities separate so the platform can scale the right layer at the right time.
- For practitioners, the main implication is that query planning, local execution and node isolation are becoming governance decisions, not just infrastructure preferences.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack surface, NIST CSF 2.0 and CIS Controls v8 set the technical controls, and ISO/IEC 27001:2022 defines the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | GV.PO-01 — Policy Establishment | The article is about operating model choices for security analytics platforms. |
| PR.AA-05 — Access Permissions, Entitlements and Authorizations | Per-customer isolation and managed compute boundaries depend on scoped access. | |
| Recommendation — Define operating policies that align search architecture with security workload patterns. Restrict platform and workload access to the minimum permissions required for each environment. | ||
| CIS Controls v8 | CIS-5 — Account Management | Managed compute and isolated environments require disciplined account and workload governance. |
| Recommendation — Review platform accounts and workload identities to ensure they map cleanly to each customer environment. | ||
| MITRE ATT&CK | TA0007; TA0008 — Discovery; Lateral Movement | The article's isolation model is relevant to limiting adversary movement and discovery across workloads. |
| Recommendation — Use isolation boundaries to reduce discovery opportunities and constrain lateral movement between workloads. | ||
| ISO/IEC 27001:2022 | A.8.24 — Use of Cryptography | The article discusses secure storage and processing patterns, though cryptography is not the main topic. |
| Recommendation — Apply the relevant secure-processing controls only where telemetry, storage or search output requires them. | ||
Key terms
- Query Shape: The execution pattern a search follows once the planner decides how to run it. In this architecture, shape determines whether work stays local, fans out across nodes or concentrates on aggregation, which directly affects latency, cost and resilience under bursty security workloads.
- Dynamic Compute Node: A short-lived node that can join a workload, process a slice of work and then leave without keeping durable state. This model suits security analytics because it reduces idle capacity and lets the platform absorb spikes without forcing every job onto a permanently provisioned cluster.
- Query planner: A query planner is the component that decides which order and direction to evaluate parts of an authorization request. In graph-based access control, it turns reachability into a cost-based routing problem, choosing paths that can reduce fan-out, short-circuit sooner, and limit unnecessary work.
- Per-Customer Isolation: A deployment pattern in which each customer runs in a separate environment with no shared services. It reduces blast radius, simplifies boundaries for access control and makes operational responsibility clearer, especially when the platform handles telemetry, search and AI-driven workflows.
Deepen your knowledge
NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security and secrets management. It helps practitioners connect identity control decisions to the operational systems that depend on them.
Published by the NHIMG editorial team on October 14, 2026.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org