Precision vs Recall in Security Detection: tuning alerts without missing attacks

Most detection teams tune for the number they can see, the false positives in the queue, and lose sight of the number they cannot see, the attacks that never fired an alert. Precision tells you how much of what you alert on is real; recall tells you how much of what is real you alert on. They pull against each other through the same threshold, and a programme that only measures one will drift towards silence or towards noise. This guide is for detection engineers, SOC leads and teams evaluating ML detectors or LLM guardrails. It explains how to measure both for rules, models and guardrails, how to read the trade-off curve, what alert fatigue really costs, and how to build a measurement setup that makes the missed attacks visible.

Key takeaways

• Precision and recall answer different questions. Precision is TP / (TP + FP), the share of alerts that are real; recall is TP / (TP + FN), the share of real attacks that alerted. Your queue shows the first; only testing shows the second.
• The base rate decides precision. When malicious events are one in tens of thousands, even a detector with a tiny false positive rate produces mostly false alarms, which is why accuracy and false positive rate alone mislead.
• The threshold is the dial. Raising a threshold or adding exclusions usually buys precision with recall, and every suppression should be recorded as a recall decision, not a clean-up.
• Measure recall deliberately. Emulation tests, replayed incidents and labelled red team activity are the only way to count false negatives; without them recall is a guess.
• Set targets per detection tier. Paging alerts need high precision, hunting and enrichment signals can tolerate low precision, and high-impact techniques need high recall, so one global target is wrong for everything.

The short answer

Optimise precision where a human or an automated response acts on every alert, because each false positive costs analyst time and erodes trust. Optimise recall where a miss is catastrophic and the volume is manageable, such as privileged account abuse, domain controller changes or a guardrail in front of an agent that can move money. Most programmes need both, achieved in layers: a high-recall, low-precision layer that collects weak signals without paging anyone, and a high-precision layer that correlates them into an alert. The Correlation Rules and Risk Based Alerting Guide covers that layering pattern; this page covers how to measure whether each layer is doing its job.

The definitions that matter, side by side

All of these metrics come from the same four counts at a fixed threshold: true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN). Google's Machine Learning Crash Course defines recall as the probability of detection and false positive rate as the probability of false alarm, and notes that accuracy is misleading on imbalanced data: a model that always predicts negative on a dataset with 1% positives scores 99% accuracy while being useless.

A detail that trips teams up: the definition of a "positive" must be fixed before measuring. A benign true positive (the rule fired correctly on real but authorised activity, such as an admin running a sanctioned scan) is a TP for the rule's logic and a non-actionable alert for the analyst. Decide which view each metric uses. The True Positive vs False Positive vs Benign True Positive page covers how those dispositions should be labelled.

Why the base rate wrecks precision

Stefan Axelsson's paper on the base-rate fallacy (ACM Transactions on Information and System Security, 2000) made the point that still governs detection engineering: the false alarm rate, not the detection rate, is the limiting factor. His model assumed about a million audit records a day and two intrusions, a base rate near 2 in 100,000. Even with perfect detection and a false alarm rate of about 1 in 100,000, only around two thirds of alarms would be real; at 1 in 1,000 the share of real alarms fell to around 2%.

A worked illustration with round numbers shows why SOC queues look the way they do. Take 50 million events a day reaching a rule, 20 of them malicious.

The lesson is not that detection is hopeless. It is that a rule must be scoped so that the population it evaluates is small and already enriched for malice (a specific process, a privileged group, a rare API call) before precision can be acceptable. Scoping is how good detection engineering moves the base rate, and it is the reason "run the ML model on everything" so often fails in production.

The trade-off curve and how to read it

Every detector with a tunable threshold, a score cut-off, a count over a time window or a similarity score, has a precision-recall curve. Raising the threshold tends to cut false positives and add false negatives, so precision rises and recall falls. The scikit-learn documentation adds a useful caveat: precision does not necessarily fall as recall rises, because lowering a threshold that was set too high can add only true positives.

• Use precision-recall curves, not accuracy, for imbalanced problems: scikit-learn describes precision-recall as a useful measure when classes are very imbalanced, which is every security problem.
• Pick the operating point from capacity and impact: start from how many alerts per day the responders can actually work for that tier, find the threshold that produces that volume, and read off the recall you are buying. If that recall is unacceptable for the technique, the answer is better scoping or enrichment, not a lower threshold.
• Exclusions move the curve, not just the point: a well-scoped allow-list entry (a named service account on a named host) removes false positives without touching true positives. A broad one (all of a subnet, all of a parent process) can remove the attacker too.

Measuring rules, ML detectors and LLM guardrails

The same two metrics apply to all three detector types, but how you get the labels and what drives each error differs.

Rules

For rules, precision comes free from triage if every closure carries a disposition and the rule ID. Recall needs a test library: for each rule, record the ATT&CK technique it targets (for example T1110.003 Password Spraying or T1059.001 PowerShell) and the emulation test that should trigger it. Atomic Red Team, an MIT-licensed library of small tests mapped to ATT&CK, is a common starting point; the Breach and Attack Simulation (BAS) Guide covers continuous options. The Sigma rule specification gives a useful vocabulary for documenting expected precision: its level field reserves critical for cases where probability borders certainty, says informational rules should trigger no case or alert, and its falsepositives field lists known benign triggers. Its status values (experimental, test, stable) map well to a promotion pipeline gated on measured precision. See the Sigma Rules Guide.

ML and anomaly detectors

Build a precision-recall curve on your own data, not the vendor's. Measure precision per score band (for example, scores above 90, 70 to 90, 50 to 70) rather than overall, because the operating decision is which band pages a human. For recall, inject known-bad behaviour into a test tenant or replay sanitised incident telemetry, and check whether it scores above your threshold. Watch for the specific failure of anomaly detection: an attacker using a valid account (T1078 Valid Accounts) during business hours from a usual location may simply not be anomalous. Record those as known blind spots. The UEBA Guide covers baselining.

LLM guardrails

A guardrail is a classifier, and it has a precision-recall trade-off like any other. The OWASP Top 10 for LLM Applications 2025 lists prompt injection as LLM01, and MITRE ATLAS tracks AML.T0051 LLM Prompt Injection and AML.T0054 LLM Jailbreak. Build two labelled sets: an attack set (direct and indirect injection, encoded and multi-turn variants, jailbreak prompts) to measure recall, and a benign set drawn from real, consented traffic, including awkward but legitimate requests such as security staff asking about attacks, to measure precision as the false refusal rate. Measure both again after every model or guardrail version change. For agents that act, weight recall higher on the action path than on the chat path. See the AI Guardrails and AI Firewalls Guide and Prompt Guardrails vs Policy Engines for AI Agent Actions.

Alert fatigue economics

Precision is an economic quantity. The cost of a detection tier per day is roughly the alert volume multiplied by the minutes to triage each alert, plus the expected loss from the attacks it misses. Low precision does not only waste time: it changes analyst behaviour, so that true positives get closed in bulk with the noise, which quietly lowers effective recall even though the rule still fires.

• Analyst capacity is the budget: if a tier receives 400 alerts a day at 6 minutes each, it consumes 40 analyst hours, five full shifts, before any investigation. Cutting precision from 10% to 5% at the same true positive count doubles that cost.
• The cost of a miss is lumpy: it is low for most techniques and very high for a few, such as identity provider tampering, backup deletion or a privileged service account used from a new location. Those few justify low-precision, high-recall detection with a dedicated, small queue.
• Automation changes the price of a false positive: if enrichment and a playbook can close a benign alert in seconds, a lower-precision rule may be affordable. If automation contains (disables an account, isolates a host), precision must be very high, because each false positive is now an outage.

The Alert Fatigue Reduction Guide covers the programme for cutting noise; the point here is that every cut is a recall decision and should be tested as one.

Building a measurement setup

1. Fix the labels. Adopt a closure taxonomy (true positive, benign true positive, false positive, duplicate, insufficient data) and make it mandatory on every alert, with rule ID and detector version stored alongside.
2. Compute precision per rule and per score band monthly. Use at least 30 closures before trusting a figure; for rare rules, accumulate over a quarter.
3. Build a recall test library. For each production detection, record the technique, the test that should trigger it and the expected alert. Start with the 20 techniques that matter most to your threat model.
4. Run tests on a schedule and after change. Weekly or after every log source, parser, rule or model change, run the library in a controlled environment and count hits. A rule that silently stops firing is the most expensive failure in detection.
5. Replay incidents. After every real incident or red team exercise, list each attacker action and mark whether any detection fired. Those misses are your highest-value recall data.
6. Record suppressions as risk decisions. Each exclusion gets an owner, a justification, an expiry and a test that proves the attack path is still detected.
7. Report both numbers together. Never show precision gains without the recall test pass rate beside them for the same period.

For rollback, keep previous rule and threshold versions in version control (see the Detection as Code Guide) and run a changed rule in shadow or audit mode beside the old one for one to two weeks, comparing alert volume and test hits before switching.

Worked scenarios

Password spraying against the identity provider. A rule fires on more than 10 failed sign-ins across distinct accounts from one source in 10 minutes. Precision is poor because shared egress addresses and misconfigured clients trip it. Raising the count to 50 restores precision but misses low-and-slow sprays. The better move is to change the population: evaluate only failures with error codes for bad passwords on valid accounts, group by source and by user agent, and add a second, high-recall, non-paging signal for slow sprays that feeds a risk score. Recall is checked monthly with a sanctioned low-rate spray test against test accounts.

Anomalous service account behaviour. An anomaly model flags non-human identities that call new APIs. Precision is about 3% because deployments legitimately change behaviour. Joining alerts to the change calendar and excluding service accounts whose pipeline deployed in the past hour lifts precision without hiding a stolen key used from a new network, which is then tested by replaying a sanitised key misuse scenario. The AI Agent Behaviour Baselining and Anomaly Detection Guide covers the same issue for agents.

A guardrail in front of a support agent. The guardrail blocks 2% of legitimate customer messages (low precision on the block action) and catches 85% of the internal injection test set. Rather than loosen it globally, the team routes low-confidence blocks to a lighter control (strip tool access, allow the chat response) and keeps hard blocks for high-confidence detections on prompts that would trigger refunds. Recall on the action path rises, false refusals visible to customers fall.

Decision guidance and checklist

Before changing a threshold or adding an exclusion, check:

• Is the precision figure based on enough closures, and are the dispositions consistent across analysts?
• Which recall test covers this rule, and does it still pass after the change?
• Could the exclusion match attacker activity (broad paths, parent processes, whole subnets)?
• Would better scoping or enrichment fix precision without moving the threshold?
• Is the change recorded with owner, reason and expiry, and can it be rolled back?

Detection and measurement

These metrics show whether the precision and recall balance is holding. The Security Operations Metrics Guide covers broader SOC measures.

• Precision per paging rule: share of closures that were true positives or actionable benign true positives, per rule, monthly.
• Recall test pass rate: share of recall tests that produced the expected alert in the last cycle, per priority technique.
• Silent rule count: production rules that produced zero alerts and zero test hits in 30 days, a proxy for broken telemetry.
• Incident detection coverage: share of attacker actions in real incidents and red team exercises that triggered at least one detection.
• Suppression debt: active exclusions without a passing recall test or past their expiry.
• Guardrail false refusal rate and attack catch rate: on fixed benign and attack sets, after each model or guardrail change.

Common failure patterns

• Tuning by volume alone: a team cuts the top ten noisy rules by widening exclusions and celebrates the drop, with no recall tests, and an exclusion on a common parent process hides the next intrusion.
• Quoting false positive rate as quality: a vendor reports a 0.01% false positive rate, which on 50 million events is 5,000 false alerts a day.
• Measuring recall on the training set: ML detector recall is reported on the same attack samples used to build it, so new variants are missed in production.
• Treating zero alerts as success: a parser change renames a field and a critical rule stops firing for months; only a scheduled recall test would have shown it.
• Inconsistent dispositions: analysts close benign true positives as false positives, so precision looks worse than it is and good rules are retired.
• One threshold for every guardrail action: the same score blocks a chat answer and a payment instruction, so either customers are refused too often or the agent is under-protected.

Framework mapping

Practitioner checklist

• Make disposition and rule ID mandatory on every alert closure, with a benign true positive option.
• Report precision per rule and per score band, never as a single SOC-wide figure.
• Build a recall test library for your top 20 techniques and run it weekly and after every change.
• Replay every incident and red team exercise against your detections and record the misses.
• Give every exclusion an owner, an expiry and a recall test that proves the attack still alerts.
• Set separate precision and recall targets for paging, review and enrichment tiers.
• Evaluate ML detectors and guardrails on your own data with precision-recall curves, not vendor accuracy figures.
• Re-measure guardrails on fixed attack and benign sets after every model or policy update.

Standards and references

• NIST: SP 800-53 Rev. 5 Security and Privacy Controls
• NIST: Cybersecurity Framework 2.0
• Axelsson, ACM TISSEC: The Base-Rate Fallacy and the Difficulty of Intrusion Detection
• Google for Developers: Classification, accuracy, recall, precision and related metrics
• scikit-learn: Precision-Recall
• SigmaHQ: Sigma Rules Specification
• OWASP: LLM01:2025 Prompt Injection
• Red Canary: Atomic Red Team

Related NHI Mgmt Group resources: Detection Engineering Guide · Alert Triage Guide · Correlation Rules and Risk Based Alerting Guide · SOC Detection Coverage Health Check