How to measure incident quality when detection strength varies

At a glance

What this is: Expel proposes scoring incident quality by combining detection diversity, capped signal counts, and dominance ratio to show which incidents are fragile versus robust.

Why it matters: SOC teams can use this to see whether their detections are providing redundant coverage or depending too heavily on a single alert source.

Context

Incident quality is a measurement problem, not just a response metric. If two alerts are investigated at the same speed but one is supported by several independent detections and the other depends on a single noisy source, they do not represent the same level of detection confidence. For SOC teams, the question is whether an incident would still be visible if one detection path failed.

This matters because detection strategy often improves in one dimension at a time, such as adding alert volume, without proving that coverage has become more resilient. In identity-adjacent and broader security programmes, the same logic applies to access, workloads, and attack paths: governance is stronger when evidence is distributed across multiple controls rather than concentrated in one fragile signal.

Key questions

Q: What makes an incident fragile from a detection perspective?

A: An incident is fragile when most of the evidence comes from one detection path, so the case can disappear if that detection fails or is muted. Fragility is not the same as low severity or low volume. It means the incident lacks enough independent corroboration to remain visible under partial control failure.

Q: Why can a high alert count still be a weak signal?

A: A large number of alerts can still come from a single noisy rule, which creates repetition without redundancy. That makes the incident look substantial while leaving it dependent on one observation source. A better measure asks whether the same activity would still be detected if that loud rule stopped firing.

Q: How do security teams know whether detection tuning is actually improving coverage?

A: They should look for a shift over time toward incidents with lower dominance and higher diversity of detections. If tuning is working, more incidents will be corroborated by multiple sources rather than concentrated in one. That trend is stronger evidence of resilience than a rising alert count alone.

Q: What should teams do when one detection accounts for most of an incident?

A: They should treat that case as a concentration risk and ask whether the same activity can be observed through another control, sensor, or analytic path. If not, the incident may be visible today but still vulnerable to future misses. The goal is to reduce dependence on one alert source.

Technical breakdown

Why alert count alone misstates incident quality

Raw alert volume does not tell you whether an incident is well-covered. One detection firing ten times can look substantial, but it still represents a single observation path. A better measure has to distinguish redundancy from repetition, because incident quality depends on whether multiple controls or detections corroborate the same attacker activity. That is why a simple count can overstate resilience or hide fragility. The core technical issue is distribution: if one source disappears, does the incident remain observable through other detections?

Practical implication: measure coverage by diversity of detections, not by total alert count alone.

How capped counts and dominance ratio work together

Capped counts assign extra weight to repeated detections, but only up to a limit, so one loud detection does not dominate the score indefinitely. Dominance ratio then measures how much of the incident is driven by the loudest detection compared with the total alert set. Together, these two measures separate evidence weight from concentration risk. This is useful because an incident can have a strong signal and still be fragile if most of that signal comes from one control or sensor. In practice, that means signal strength and concentration must both be tracked.

Practical implication: combine capped volume with dominance analysis to avoid mistaking noisy incidents for resilient ones.

Why resilience changes with scope of compromise

A dynamic cap can improve interpretation when the number of compromised internal assets is known, because a broader compromise should legitimately produce more evidence. That makes the score more reflective of actual incident scope. But the trade-off is that higher volume can still be concentrated in one detection path, which leaves fragility hidden unless dominance is also measured. This is the key architectural insight: scope increases signal, but redundancy determines survivability if one detection breaks. Security teams need both dimensions to understand whether a detection strategy is broad enough to withstand failure.

Practical implication: tie scoring to compromise scope where possible, but keep dominance checks to expose single-control dependence.

NHI Mgmt Group analysis

Incident quality is an operational resilience metric, not a reporting convenience. A SOC can have plenty of alerts and still miss the attack if those alerts come from one brittle detection path. The article’s core insight is that detection strength should be judged by redundancy and balance, because resilience disappears when one sensor failure erases the incident. Practitioners should treat this as a coverage question, not a volume question.

Detection concentration is the named failure mode: single-point detection dependence. When one rule, sensor, or telemetry source contributes most of the evidence, the incident can look strong while remaining fragile. That failure mode is different from low alert volume. It tells teams that observability is present but not yet distributed across enough control paths to survive a broken detection.

Capped counts are a useful governance compromise, but they only solve one half of the problem. They stop repeated alerts from overstating evidence, yet they also show that more events do not automatically mean better quality. The real governance question is whether the alert set reflects multiple independent views of the same activity. SOC leaders should use this as a tuning discipline, not a scoring vanity metric.

Incident quality scoring belongs in the detection engineering feedback loop. If the distribution of incidents never moves toward lower dominance and higher signal variety, the programme is not reducing blind spots. The value of the method is that it exposes whether tuning is actually creating redundant coverage, which is the practical measure of improved detection governance.

What this signals

Detection concentration is the real governance problem behind incident quality. When a case depends too heavily on one alert source, the SOC has observability but not enough redundancy to survive partial failure. That means detection engineering has to be judged by whether it spreads evidence across multiple paths, not by whether it creates more noise.

The next maturity step is to measure whether incidents are becoming more balanced over time. Quarterly trends that move fragile cases down and robust cases up show that tuning is improving coverage rather than just increasing alert volume. That is the kind of signal programme leaders can use to prove that detection strategy is becoming more resilient.

For practitioners

• Score incidents for detection diversity and concentration Track both the number of unique detections and the dominance ratio so you can tell whether an incident is supported by multiple observations or one repeated alert path.
• Use capped counts for repeated detections Apply a default cap to repeated alerts so a loud rule does not overwhelm the score, then adjust the cap when the incident scope is clearly broader.
• Compare fragile, risky, decent, and robust incidents over time Review incident quality by quarter or other fixed period to see whether tuning is shifting cases away from single-point dependence and toward more balanced coverage.

Key takeaways

• Incident quality is better measured by resilience than by raw alert volume, because repeated alerts from one source can hide a fragile detection posture.
• A useful scoring model combines diversity, capped counts, and dominance ratio so teams can see when one detection is doing too much of the work.
• The operational goal is not perfect balance but fewer single-point failures in detection coverage, which makes incidents harder to miss.

Key terms

• Incident Quality: Incident quality is the degree to which an incident is supported by multiple, balanced detections rather than one brittle alert source. It reflects how well the evidence survives loss of a single control and whether the activity is still recognisable enough for confident triage and response.
• Dominance Ratio: Dominance ratio measures how much the loudest detection contributes to the total incident evidence. A high ratio means the incident depends heavily on one source, which creates fragility if that source fails, degrades, or becomes too noisy to trust.
• Capped Count: A capped count limits how much repeated firing from one detection contributes to the overall incident score. It preserves evidence value without letting one noisy alert path overwhelm the assessment of diversity and resilience.
• Detection Diversity: The extent to which an incident is represented by multiple distinct detections rather than repeated alerts from one source. Higher diversity usually indicates better corroboration and lower dependence on any one control, making the incident more resilient to partial detection failure.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through independent research, practitioner guides, and the NHI Foundation Level course. It is designed for teams that need stronger governance across identities, credentials, and access paths.