How should security teams build telemetry for detection and hunting?

Why This Matters for Security Teams

Telemetry design is not just a logging exercise. It determines whether detection engineering, threat hunting, and incident response can answer a specific question quickly enough to matter. Security teams often collect whatever each platform exposes, then discover that the resulting data is fragmented, noisy, or missing the identity and context needed to explain what happened. A more effective approach starts with use cases, then maps required signals to sources, retention, and correlation rules.

This matters because modern attacks rarely stay inside one layer. A credential abuse event may begin with identity misuse, move through a cloud control plane, and end with application-level actions that look routine unless the logs are connected. The NIST Cybersecurity Framework 2.0 treats detection and response as outcomes of a broader security program, which is the right lens here: telemetry should support decisions, not just storage. In practice, many security teams encounter the weakness only after an alert cannot be confirmed because the supporting logs were never enabled or were kept for too short a period.

How It Works in Practice

Effective telemetry starts with a detection catalog. Each detection or hunting hypothesis should state what event would prove it, what systems generate that event, and what fields are required to connect it to identity, device, workload, or transaction context. That usually means collecting from four layers at minimum: identity providers, endpoints, cloud control planes, and application services. The goal is not maximum volume, but sufficient fidelity for correlation and timeline reconstruction.

Normalisation is the next step. Events should be mapped into a common schema so that fields like actor, target, action, source IP, device ID, tenant, and timestamp mean the same thing across sources. Without that consistency, analysts spend time translating data instead of investigating behaviour. Current guidance suggests preserving both raw events and normalised records, because raw logs are often needed for validation, while normalised records are better for analytics and hunting.

• Define priority use cases first, then identify the minimum signal set needed to support them.
• Ensure identity telemetry covers authentication, privilege changes, session creation, token issuance, and anomalous consent events.
• Capture host telemetry for process creation, command lines, persistence, and suspicious child process chains.
• Retain cloud audit logs for configuration changes, API calls, and resource creation or deletion.
• Include application logs for authorisation decisions, sensitive actions, and failed or unusual workflows.

Quality controls matter as much as collection. Teams should test for clock drift, field consistency, ingestion loss, and parsing errors, then validate detections against expected and unexpected event paths. The MITRE ATT&CK knowledge base is useful here because it ties telemetry to adversary techniques, helping teams verify whether their data actually supports the behaviours they claim to detect. These controls tend to break down when log ownership is split across infrastructure, platform, and application teams because no one enforces a shared schema or retention baseline.

Common Variations and Edge Cases

Tighter telemetry standards often increase storage, engineering, and privacy overhead, requiring organisations to balance detection value against operational cost and data minimisation obligations. That tradeoff is especially visible in multi-tenant SaaS, highly distributed cloud estates, and regulated environments where some logs contain personal data or secrets-like material.

There is no universal standard for telemetry depth yet, so best practice is evolving around risk-based coverage. For example, a high-value identity event might need full token and session context, while a low-risk internal application may only need audit trails and denial events. The NIST log management guidance remains useful for retention, integrity, and review practices, but teams still need to adapt it to cloud-native and SaaS realities. In identity-heavy environments, telemetry should also account for privileged access, non-human identities, and service-to-service authentication, because those entities often produce the earliest signal of compromise.

Where this guidance becomes less reliable is in serverless, ephemeral, or vendor-managed platforms, because the underlying execution context disappears before analysts can inspect it. In those environments, detections must lean more heavily on control-plane events, identity records, and application-layer audit data than on traditional host telemetry. When that is not possible, hunting quality degrades quickly and false confidence rises.

Related resources from NHI Mgmt Group

• How should security teams build cloud threat detection for short-lived workloads?
• How should security teams govern telemetry schema drift in AI-driven detection pipelines?
• How should security teams build effective threat hunting hypotheses?
• How should security teams improve detection when telemetry is fragmented across cloud, SaaS, and identity systems?