A method for checking whether an output is factually correct and grounded in reliable information. In LLM evaluation, factuality scoring helps teams detect hallucinations, unsupported claims, and incorrect assertions. It is often used as part of an early evaluation stack before more complex or domain-specific measures are introduced.
Expanded Definition
Factuality scoring is an evaluation method for judging whether a model output is grounded in verifiable information rather than invented detail. In practice, it sits between simple correctness checks and deeper quality evaluation: a response can be fluent, relevant, and still score poorly if it states unsupported facts, confuses entities, or fabricates specifics. For LLM evaluation teams, the term is usually applied to outputs that can be checked against a reference source, a curated knowledge base, or a human review rubric.
The concept is narrower than general model quality assessment. It does not by itself measure helpfulness, style, or task completion, and it is not the same as source citation. A cited answer can still be wrong, and an uncited answer can still be factually sound. Guidance versus consensus is uneven here: some teams treat factuality as a single score, while others split it into claim-level precision, recall, and supportability. That distinction matters because a high-level score can hide whether the model is making one major error or many smaller ones.
A common boundary issue is that factuality scoring depends on what is considered the “ground truth” for the task. In open-ended domains, the evaluation must define whether ambiguity, partial evidence, or outdated sources count as correct.
Examples and Use Cases
Factuality scoring appears in evaluation workflows where teams need a fast signal before more expensive review. It is often used as a screening layer, not a final verdict, because its value is in catching obvious unsupported claims early.
- Scoring a customer support draft against a product knowledge base to find statements that cannot be traced to approved documentation.
- Reviewing a legal or compliance assistant for unsupported claims about policy, deadlines, or obligations before it is exposed to users.
- Comparing model answers to gold-standard reference answers in benchmark datasets, especially where exactness matters more than style.
- Sampling generated summaries for claim-level verification when a workflow needs to separate accurate retrieval from plausible but incorrect paraphrase.
In many teams, factuality scoring is paired with human spot checks because automated scoring can miss subtle errors such as correct facts placed in the wrong context. That tradeoff is especially visible when a system answers from multiple sources and the evaluator must decide whether disagreement reflects a model error or source conflict.
For a broader controls perspective on evaluation governance, NIST’s control catalog shows how organisations usually anchor measurement inside a larger assurance process, and the NIST SP 800-53 Rev 5 Security and Privacy Controls is a useful reference for that context.
Security Implications
When factuality scoring is weak or treated as a box-ticking exercise, false confidence can spread through the rest of the evaluation stack. A model may appear ready for release because it scores well on coherence or tone, while still producing unsupported claims that create operational, legal, or reputational exposure. The practical failure is not just “the model was wrong”; it is that the review process failed to detect which errors were material enough to matter.
Another recurring issue is score drift caused by inconsistent rubrics. If annotators or automated judges interpret supportability differently, teams can see unstable results across datasets, model versions, or prompt variants. That makes it hard to compare releases, spot regressions, or decide whether a change improved reliability. In high-stakes settings, the most dangerous symptom is selective correctness: outputs that are partly right enough to look trustworthy while still containing one critical fabricated detail.
Practitioner observation: factuality problems are often easiest to catch when evaluators score individual claims rather than whole answers, because one unsupported assertion can otherwise be hidden inside a broadly good response.
Domain and Governance Relevance
Factuality scoring matters most in AI evaluation governance, where organisations need a repeatable way to distinguish grounded output from plausible-sounding fabrication. That makes it a measurement problem as much as a model problem: the score only has value if the reference standard, rubric, and review process are clearly defined.
In broader AI security and assurance work, factuality is also a boundary control. It helps teams decide when a system is merely retrieving information, when it is synthesising it, and when it is overreaching beyond evidence. That distinction becomes more important as systems are given broader tool access, because an inaccurate answer can become an action request, a policy recommendation, or a delegated decision. The governance question is not only whether the model sounds right, but whether the organisation can trust the reasoning chain behind the output.
For non-human identity and agentic workflows, factuality scoring can support trust calibration, but it should not be mistaken for authorization, provenance, or access control. A system can be factually accurate and still be inappropriate to execute a task, so factuality is one assurance input rather than a full safety gate.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
NIST AI 600-1, NIST AI RMF, NIST CSF 2.0 and CIS Controls v8 set the technical controls, while ISO/IEC 42001:2023 define the regulatory obligations.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI 600-1 | 1-2 — Evaluation and Validation | Factuality scoring is an AI output evaluation activity. |
| Recommendation — Score model outputs against grounded references and track factual errors across test runs. | ||
| NIST AI RMF | GV-1 — Govern AI Risk | Factuality scoring supports AI risk governance and assurance. |
| Recommendation — Use factuality metrics as part of AI risk oversight and release decisions. | ||
| ISO/IEC 42001:2023 | 9.1 — Monitoring, measurement, analysis and evaluation | Factuality scoring is a measurable AI quality control within governance systems. |
| Recommendation — Define factuality checks as a monitored AI assurance measure with repeatable criteria. | ||
| NIST CSF 2.0 | GV.OC-03 — Mission Objective | Factuality scoring helps align model evaluation with intended business use. |
| Recommendation — Tie factuality thresholds to the system's intended mission and risk tolerance. | ||
| CIS Controls v8 | 8 — Audit Log Management | Scoring workflows benefit from traceable evaluation records and reviewer accountability. |
| Recommendation — Log evaluation inputs, outputs, and reviewer decisions so scoring can be audited later. | ||
