Topic starter
18/08/2026 2:19 pm
TL;DR: OCSF standardises security event JSON, but the storage layout still depends on the engine: Iceberg can tolerate a wide, sparse table because width is mostly metadata cost, while ClickHouse usually needs promoted columns and payload spillover to stay performant, according to TENZIR. The practical takeaway is that schema design for OCSF is an engine decision, not a one-size-fits-all pattern.
Editorial analysis by NHI Mgmt Group, based on content published by TENZIR: “OCSF storage strategies for your security lakehouse”.
Key questions
Q: When should teams use a wide OCSF table instead of splitting by source or class?
A: Use a wide OCSF table when the engine handles sparse columns efficiently, the query model benefits from cross-class analysis, and schema evolution needs to stay metadata-driven.
Q: Why do wide OCSF tables work better in Iceberg than in ClickHouse?
A: Iceberg stores sparse columns cheaply because absent values cost nothing in a file and schema changes are mostly metadata updates.
Q: What are the signs that an OCSF table layout is too wide for the chosen engine?
A: The warning signs are slower planning, bloated metadata, expensive merges, rising insert latency, and a growing number of columns that almost never carry values.
Practitioner guidance
- Select the storage strategy by engine first Decide whether the target is Iceberg or ClickHouse before fixing the OCSF table layout, because the same schema carries different physical costs in each engine.
- Use wide nested tables in Iceberg when schema breadth is expected Keep OCSF objects nested and accept sparsity when the engine is Iceberg, then focus governance on metadata hygiene, stats coverage, and partition design.
- Promote only hot OCSF fields in ClickHouse Model frequently queried attributes as real columns and route the long tail into a semi-structured payload so insert and merge overhead stays bounded.
Bottom line: OCSF standardises the event contract, but it does not decide how events should be physically stored for analysis.
Explore further
View Full Forum → | NHI Foundation Course → | Our Services → | Read the full analysis →
18/08/2026 3:11 pm
Engine choice is the real governance boundary for OCSF. OCSF gives teams a common event contract, but it does not erase physical storage trade-offs. The same wide schema that is tolerable in Iceberg can become operationally expensive in ClickHouse, which means schema governance has to include engine behaviour, not just data model consistency. Practitioners should treat engine selection as part of the telemetry architecture decision, not as an implementation detail.
A question worth separating out:
Q: How should security data teams balance query performance and schema completeness in OCSF pipelines?
A: Prioritise the fields analysts query most often, then preserve completeness in a spillover structure or nested payload. That approach keeps the active query path fast without throwing away low-frequency data needed for investigations. The right balance is not full flattening versus full nesting, but selecting the engine-friendly layout that keeps both performance and fidelity acceptable.
👉 Read our full editorial: OCSF lakehouse storage turns on engine choice, not schema width