Threat intelligence that says an actor "may" target you, from a source nobody has graded, gives decision makers nothing they can act on and no way to tell a rumour from a confirmed finding. Intelligence communities solved this decades ago with two separate tools: a grading system for how reliable a source is and how credible a piece of information is (the Admiralty or NATO system), and a fixed vocabulary for likelihood with stated probability ranges (estimative language), plus a separate statement of analytic confidence. This guide explains each scale, how they differ, how to apply them in cyber threat intelligence reports, tickets and platforms such as MISP and STIX 2.1, and how to measure whether your team uses them consistently. It is for CTI analysts, SOC leads who consume intelligence, and the managers who brief leadership. It builds on the Threat Intelligence Guide.
Key takeaways
- Three different questions: source reliability (can I trust who told me?), information credibility (is this claim plausible and corroborated?) and analytic confidence (how solid is my judgement?) are separate and must be graded separately.
- Admiralty grades are two characters: a letter A to F for the source and a number 1 to 6 for the information, so "B2" means a usually reliable source reporting something probably true.
- Fix the likelihood words: ICD 203 ties seven term pairs to ranges from 01-05 percent (almost no chance) to 95-99 percent (almost certain), and asks writers to use one set of terms.
- Do not mix confidence and likelihood in one sentence: ICD 203 requires this so readers do not confuse the probability of an event with the strength of the evidence.
- Make it machine readable: MISP's admiralty-scale and estimative-language taxonomies and the STIX 2.1 confidence property carry the grades into tooling, so automated actions can depend on them.
Why grading matters in cyber threat intelligence
Sherman Kent's essay "Words of Estimative Probability", written in the 1960s for the CIA, starts from a 1951 National Intelligence Estimate that called a Soviet attack on Yugoslavia a "serious possibility". Kent's own odds were about 65 to 35 in favour; when he asked the board members who had agreed the wording, their private odds ranged from about 20 to 80 to 80 to 20, and policy readers took the phrase to mean much lower odds. The same failure is common in cyber reporting today: "likely linked to", "possibly exploited" and "may be targeting" are read differently by every recipient.
In security operations the cost is concrete. An indicator from an unvetted paste site and one from a government advisory can arrive in the same feed with the same weight, and a block list built from both will either break production or miss the real threat. Graded sources and calibrated language let analysts route low-grade information to monitoring and high-grade information to blocking, and let executives see how much weight a judgement can bear. See the Threat Intelligence Feed Evaluation Guide.
The Admiralty system: source reliability and information credibility
The Admiralty Scale, also called the NATO system, rates the reliability of a source and the credibility of an item of information independently. The definitions below follow MISP's admiralty-scale taxonomy, which cites US Army FM 2-22.3 and NATO documents.
| Source reliability | Meaning |
|---|---|
| A Completely reliable | No doubt of authenticity, trustworthiness or competency; a history of complete reliability |
| B Usually reliable | Minor doubt; a history of valid information most of the time |
| C Fairly reliable | Doubt, but has provided valid information in the past |
| D Not usually reliable | Significant doubt, but has provided valid information in the past |
| E Unreliable | Lacking authenticity, trustworthiness and competency; a history of invalid information |
| F Reliability cannot be judged | No basis exists for evaluating the source |
| Information credibility | Meaning |
|---|---|
| 1 Confirmed by other sources | Confirmed by other independent sources; logical in itself; consistent with other information |
| 2 Probably true | Not confirmed; logical in itself; consistent with other information |
| 3 Possibly true | Not confirmed; reasonably logical; agrees with some other information |
| 4 Doubtful | Not confirmed; possible but not logical; no other information on the subject |
| 5 Improbable | Not confirmed; not logical; contradicted by other information |
| 6 Truth cannot be judged | No basis exists for evaluating the information |
MISP's taxonomy also includes a seventh source value, "G Deliberately deceptive", which is not part of the classic six-letter scale. Decide whether your team uses it and document the choice.
Applying the grades to cyber sources
- Grade the source on track record, not reputation: a well-known vendor blog that has twice misattributed activity is not an A. Keep a source register with each source's history of confirmed and refuted reporting.
- Grade each item separately: a reliable source can pass on doubtful information, so "A4" is legitimate and useful.
- Your own telemetry is a source too: a detection from a tuned EDR rule and an alert from an untuned anomaly model deserve different reliability grades.
- Use F and 6 honestly: new sources and first-time claims should be F and 6 until there is a basis to judge. Defaulting everything to B2 destroys the value of the scale.
- Independence matters for 1: two feeds that both copied the same vendor report are one source, not corroboration.
Estimative language: expressing likelihood
US Intelligence Community Directive 203, Analytic Standards (signed 2 January 2015, with technical amendments dated 21 January 2022; published on the ODNI archive site), requires products to express likelihood with one consistent set of terms tied to ranges. MISP's estimative-language taxonomy carries the same ranges.
| Range | ICD 203 terms | Illustrative cyber example |
|---|---|---|
| 01-05% | Almost no chance; remote | Exploitation of a vulnerability with no known exploit and no exposure |
| 05-20% | Very unlikely; highly improbable | Actor known only for one sector targeting an unrelated one |
| 20-45% | Unlikely; improbable | Opportunistic campaign reaching a hardened estate |
| 45-55% | Roughly even chance; roughly even odds | Genuinely balanced evidence |
| 55-80% | Likely; probable | Mass exploitation of an exposed, unpatched edge device |
| 80-95% | Very likely; highly probable | Credential reuse after a confirmed infostealer infection |
| 95-99% | Almost certain; nearly certain | Activity matching multiple confirmed, independent indicators |
Kent's original scheme was similar: almost certain about 93 percent, give or take 6; probable about 75, give or take 12; chances about even, 50 give or take 10; probably not about 30, give or take 10; almost certainly not about 7, give or take 5. He reserved "possible" for things that cannot be given odds and rejected modifiers such as "serious possibility".
UK government assessments, including the NCSC's published threat assessments, use the Professional Head of Intelligence Assessment (PHIA) probability yardstick, with terms such as "realistic possibility", "likely", "highly likely" and "almost certain". Its bands differ from ICD 203, so pick one yardstick for your organisation, publish it in every report, and do not mix terms from both.
Writing rules that make the language work
- State the judgement, the term and the timeframe: "It is likely (55-80 percent) that the actor will attempt to exploit this vulnerability in our estate within 30 days."
- Ban unscaled hedges: remove "may", "could", "possibly" and "it cannot be ruled out" from key judgements; they carry no probability.
- Put the yardstick on the page: include the table as a footer or appendix in every product, so a forwarded report keeps its meaning.
- Separate fact from judgement: ICD 203's tradecraft standards require distinguishing underlying intelligence information from analysts' assumptions and judgements; use "we observed" for facts and "we assess" for judgements.
- Explain change: when a judgement moves from likely to very likely, say what new information moved it.
Analytic confidence
Confidence describes the basis for a judgement, not the probability of the event. MISP's estimative-language taxonomy, based on ICD 203 and US Joint Publication 2-0, gives three levels.
| Confidence | Basis |
|---|---|
| High | Well-corroborated information from proven sources, minimal assumptions, strong logical inferences and methods |
| Moderate | Partially corroborated information from good sources, several assumptions, a mix of strong and weak inferences and methods |
| Low | Uncorroborated information from good or marginal sources, many assumptions, mostly weak logical inferences |
ICD 203 requires that a product does not combine a confidence level and a degree of likelihood in the same sentence. Write two sentences: "We assess it is likely that the intrusion set will target our VPN appliances. We have moderate confidence in this judgement because it rests on two independent B2 reports and one assumption about patch status." This also links confidence back to the Admiralty grades of the underlying sources, which is what makes the chain auditable.
Attribution is where confidence matters most. A low-confidence attribution can still be useful for prioritising hunts, but must never drive sanctions, public statements or legal action. See the Cyber Threat Attribution Guide.
Structured techniques that improve calibration
Grading scales record judgements; structured analytic techniques improve them. The CIA's Tradecraft Primer (March 2009) groups techniques into diagnostic ones (Key Assumptions Check, Quality of Information Check, Indicators or Signposts of Change, Analysis of Competing Hypotheses), contrarian ones (Devil's Advocacy, Team A/Team B, High-Impact/Low-Probability Analysis, "What If?" Analysis) and imaginative ones (Brainstorming, Outside-In Thinking, Red Team Analysis, Alternative Futures Analysis).
- Quality of Information Check: the natural partner of Admiralty grading; review the sources behind each key judgement before publishing.
- Key Assumptions Check: list the assumptions that the confidence statement depends on; many assumptions means moderate or low confidence.
- Analysis of Competing Hypotheses: use for attribution and for "is this incident related to that campaign" questions.
See the Structured Analytic Techniques for CTI Guide.
Encoding grades in tools and workflows
| Tool or format | Mechanism | Practical use |
|---|---|---|
| MISP | admiralty-scale:source-reliability, admiralty-scale:information-credibility, estimative-language:likelihood-probability and estimative-language:confidence-in-analytic-judgment taxonomy tags | Filter exports so only A-B and 1-2 tagged indicators reach blocking controls |
| STIX 2.1 | Optional confidence common property, an integer 0-100; Appendix A of the specification maps common scales to it | Carry confidence across TAXII feeds and platforms |
| Threat intelligence platforms | Per-object confidence and per-user or per-group confidence limits | Stop low-confidence imports overwriting analyst-confirmed data; see the OpenCTI Guide |
| Reports and tickets | Fixed fields for source grade, likelihood term and confidence | Consistent briefing and auditable decisions |
Map your chosen scale to STIX numbers once, document it, and apply it in every integration. Automated actions such as blocking, disabling an account or revoking an API key should require a minimum grade. A credential reported in a low-grade paste should trigger verification and rotation planning, while one confirmed in your own logs should trigger immediate revocation. See the MISP Threat Sharing Guide and the STIX and TAXII Guide.
A worked example
A sharing community reports that an intrusion set is exploiting a remote access appliance your organisation runs. Graded properly, the entry in your intelligence log reads as follows.
- Source: sector sharing community, reliability B (usually reliable; three of its last four advisories were later confirmed).
- Information: credibility 2 (probably true; logical and consistent with a vendor advisory, but not yet independently confirmed). Grade: B2.
- Own telemetry: no matching indicators in 30 days of appliance logs; source reliability A for the logs, credibility 6 for absence of evidence because logging on two appliances is incomplete.
- Key judgement: "We assess it is likely (55-80 percent) that the intrusion set will attempt exploitation of our appliances within 14 days."
- Confidence: "We have moderate confidence in this judgement: it rests on one B2 report, a vendor advisory and an assumption that our exposure matches the reported targets."
- Action: patch within the emergency window, hunt with the reported indicators, and fix the logging gap; regrade when the vendor confirms or refutes the report.
Rolling it out
- Choose and publish the scales: Admiralty for sources and information, one likelihood yardstick (ICD 203 or PHIA), and three confidence levels. Put them in your report template; see the Threat Intelligence Report Template.
- Build the source register: every feed, sharing community, vendor and internal sensor with its current reliability grade and history.
- Train with calibration exercises: have analysts grade the same ten items independently and discuss differences.
- Configure tooling: taxonomy tags in MISP, confidence mapping in STIX exports, minimum grades for automated actions.
- Review quarterly: regrade sources based on how their reporting held up, and adjust thresholds. If a threshold blocks too much, roll it back to monitoring rather than lowering the source grade.
A small team can start with the Admiralty grades and the ICD 203 table in the report template. A large programme should add the source register, calibration scoring and enforced grading fields in its platform. See the Threat Intelligence Maturity Model.
Measuring consistency
- Grading coverage: percentage of published key judgements with a likelihood term and a confidence level (target 100 percent).
- Source register coverage: percentage of ingested items whose source has a current reliability grade.
- Inter-analyst agreement: share of calibration items graded within one step by all analysts.
- Calibration: of judgements marked likely, roughly 55-80 percent should come true over time; track outcomes.
- Hedge rate: unscaled hedges ("may", "could") per report, trending to zero in key judgements.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | ID.RA-02 Cyber threat intelligence is received from information sharing forums and sources | Grading sources on receipt |
| NIST CSF 2.0 | ID.RA-03 Internal and external threats are identified and recorded | Recorded judgements with likelihood and confidence |
| NIST CSF 2.0 | DE.AE-07 Threat intelligence integrated into analysis | Grades that govern automated use |
| NIST SP 800-53 Rev. 5 | PM-16 Threat Awareness Program; RA-3(3) Dynamic Threat Awareness | A defined, consistent intelligence process |
| ICD 203 | Tradecraft standards 1 to 3 (sources, uncertainty, fact versus judgement) | The direct source for the practices above |
Common failure patterns
- Everything is B2: analysts default to a middle grade, so the scale carries no information.
- Collapsing the two axes: treating a reliable source's claim as confirmed because of who said it.
- Circular corroboration: counting several feeds that repeat one original report as independent confirmation.
- Confidence as likelihood: "high confidence the actor will attack" mixes two ideas and is read as near certainty.
- Grades lost in transit: grades held in report text but not in machine-readable fields, so the SIEM and SOAR treat every indicator the same.
- No regrading: sources keep their original grade years after their reporting quality changed.
Practitioner checklist
- Publish one source reliability scale, one likelihood yardstick and one confidence scale.
- Maintain a source register with grades and track record, reviewed quarterly.
- Grade source and information separately for every item used in a key judgement.
- Write likelihood and confidence in separate sentences.
- Remove unscaled hedges from key judgements.
- Tag MISP events with admiralty-scale and estimative-language taxonomies.
- Document the mapping from your scale to the STIX 2.1 confidence integer.
- Set minimum grades for automated blocking and credential revocation.
- Run calibration exercises and track outcomes of past judgements.
Standards and references
- ODNI: Intelligence Community Directive 203, Analytic Standards
- CIA Center for the Study of Intelligence: Words of Estimative Probability (Sherman Kent)
- CIA: A Tradecraft Primer, Structured Analytic Techniques
- MISP Project: Admiralty Scale taxonomy
- MISP Project: Estimative Language taxonomy
- OASIS: STIX Version 2.1
- NCSC: Impact of AI on cyber threat assessment (PHIA yardstick in use)
- NIST: Cybersecurity Framework 2.0
Related NHI Mgmt Group resources: Threat Intelligence Guide · Structured Analytic Techniques for CTI Guide · Cyber Threat Attribution Guide · ISACs and Information Sharing Communities Guide