How do OCSF and a data lake change investigation workflows?

Why This Matters for Security Teams

OCSF and a data lake change the investigation model from tool-by-tool hunting to evidence-led analysis. Instead of forcing analysts to normalize every product’s output by hand, a shared schema lets events from endpoint, cloud, identity, and network sources land in a form that is easier to correlate and retain. That matters when the question is not just what happened, but how far the activity spread and whether the same actor touched multiple systems.

This also shifts the operational burden. Teams can keep high-value, low-latency data in the SIEM while pushing longer-retained context into the lake, which reduces pressure to overstore everything in expensive hot storage. The practical benefit is better reconstruction of timelines, especially when investigations cross control boundaries such as IAM, endpoint response, and cloud activity. For governance, the architecture also supports clearer mapping to NIST Cybersecurity Framework 2.0 functions by making detection, analysis, and recovery evidence more accessible.

In practice, many security teams discover their investigation gaps only after an incident has already fragmented evidence across disconnected platforms, rather than through intentional evidence design.

How It Works in Practice

OCSF works best when it is treated as an ingestion and analysis contract, not just a reporting format. Sources such as cloud control planes, EDR, IAM, SaaS, and network sensors are mapped into a common event structure so that fields like actor, target, time, and action can be queried consistently. The data lake then becomes the long-term store for normalized telemetry, while the SIEM remains the operational layer for alerting, triage, and active response.

In a mature workflow, investigators pivot from a SIEM alert into lake queries that pull related events from earlier or later time windows. That is useful when one alert only shows a single authentication event, but the lake can reveal preceding token abuse, privilege escalation, or lateral movement signals. The lake also helps when a team needs to compare similar activity across multiple environments without rebuilding logic for each vendor log format.

• Use OCSF field mapping to standardize source-specific records before storage.
• Keep hot detections in the SIEM, but retain investigative depth in the lake.
• Preserve original source fields where possible for forensic validation.
• Build query patterns around actors, assets, sessions, and event sequences.

For teams defining detection architecture, the NIST Cybersecurity Framework 2.0 is a useful way to align this workflow with monitor, analyze, and recover outcomes, while preserving enough context to support incident handling and lessons learned. These controls tend to break down when source telemetry is incomplete or inconsistent across tenants because normalization then hides gaps instead of clarifying them.

Common Variations and Edge Cases

Tighter normalization often increases engineering overhead, requiring organisations to balance query consistency against the cost of maintaining mappings as sources change. That tradeoff becomes visible in environments with frequent SaaS onboarding, heterogeneous cloud accounts, or multiple EDR products, where schema drift can outpace the team’s ability to validate every field.

There is also no universal standard for how much raw data should remain searchable versus archived. Some teams keep only OCSF-normalized records in the lake, while others retain both normalized and source-native logs for evidentiary reasons. The second approach improves defensibility, but it raises storage and governance requirements. Current guidance suggests keeping source fidelity for high-risk log types, especially identity and admin activity, where small field differences can materially affect an investigation.

Another edge case appears when the SIEM and lake disagree on event timing because of ingestion delays or timestamp parsing. In those environments, investigators need an agreed rule for source of truth and time ordering, otherwise the same incident can appear to unfold in the wrong sequence. For identity-heavy investigations, that problem is especially important when correlating access changes, session creation, and privileged actions.

Related resources from NHI Mgmt Group

• How should teams govern AI systems that can change production data and workflows?
• Why do SIEM, ISOC, and data lake models still need the same investigation workflow?
• Why do AI assistants and MCP-connected workflows change data loss prevention requirements?
• How should security teams reduce the time lost between security data and an actionable investigation plan in AI-assisted workflows?