Splunk’s search-time extraction approach, where fields are interpreted when a query runs rather than when data is ingested. It offers flexibility, but it can make common searches slower because the system must examine more raw events at query time.
Expanded Definition
Schema on the fly describes a search-time parsing model in which fields are derived when a query is executed, not fixed in advance at ingestion. In security operations, this makes it easier to adapt to changing log formats, inconsistent telemetry, or ad hoc investigations without redesigning pipelines. The tradeoff is that the query engine must inspect more raw events and apply parsing logic at runtime, which can increase latency and make results depend on the quality of the search expression. In practice, the term is most often associated with Splunk-style search-time extraction, though usage in the industry is still evolving and not every platform uses the phrase identically. For governance and control mapping, the concept aligns most closely with NIST Cybersecurity Framework 2.0 because it affects how teams detect, analyze, and respond to events. The most common misapplication is treating schema on the fly as a substitute for sound log engineering, which occurs when teams rely on query-time parsing for critical detections despite unstable source formats.
Examples and Use Cases
Implementing schema on the fly rigorously often introduces query-time overhead and greater dependence on search precision, requiring organisations to weigh flexibility against performance and repeatability.
- A SOC analyst searches authentication logs where usernames, hostnames, and outcome codes are extracted dynamically from raw text during the investigation.
- A threat hunter pivots across a newly onboarded cloud source whose event structure is inconsistent, using field extractions created at search time instead of waiting for a new ingestion pipeline.
- An incident responder validates whether a suspicious IP address appears across several log types, using runtime parsing to normalise fields that were not mapped consistently at ingest.
- A security engineering team prototypes detection logic against a changing application log format before formalising a stable parser or index-time schema.
- A compliance team reviews whether search-time extraction introduces blind spots in monitoring, then compares query coverage against operational expectations and NIST Cybersecurity Framework 2.0 detection and analysis goals.
Why It Matters for Security Teams
Schema on the fly matters because it changes where cost, accuracy, and control live: at search time rather than ingest time. That can be useful for resilience when telemetry is messy, but it also means detection quality may vary by query author, and weak searches can miss events that a fixed schema would have surfaced more predictably. Security teams should understand that this is not just a convenience feature; it affects triage speed, investigation consistency, and the reliability of reporting. In regulated environments, the concern is not only performance but also whether a control depends on fragile parsing logic that analysts must remember to apply every time. Guidance from the NIST Cybersecurity Framework 2.0 reinforces the need for dependable detection and analysis processes, even when the underlying platform allows flexible extraction. Organisational risk often becomes visible only after an incident review shows that a key field was not extracted consistently, at which point schema on the fly becomes operationally unavoidable to fix.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST CSF 2.0 provides the primary governance reference for this term.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | DE.AE | Event analysis depends on reliable field extraction and interpretation. |
Ensure search-time parsing supports consistent anomaly detection and event analysis.
