What is the difference between groundedness and answer relevance in RAG evaluation?

Why groundedness and answer relevance measure different failure modes

Groundedness asks whether the model’s claims are supported by the retrieved context. Answer relevance asks whether the model actually answers the user’s question. Those are separate checks: a response can be well supported but off-topic, or directly responsive while smuggling in unsupported details. In RAG, that distinction matters because retrieval quality and response quality can fail independently.

Groundedness is about evidence alignment. If the model states a fact, groundedness evaluates whether that fact appears in, or can be justified from, the retrieved documents. This is the metric that catches unsupported additions, overconfident paraphrases, and plausible but invented detail. It is especially important when the model has enough context to sound credible but not enough to justify every claim.

Answer relevance is about intent alignment. It checks whether the response is actually useful for the user’s prompt, rather than merely being about the same topic. A response can stay tightly tied to the source documents and still miss the user’s ask if it answers a nearby question, over-explains background, or shifts to a different subtopic. That is why relevance is not a substitute for factual support.

How the two metrics diverge in practice

In a RAG pipeline, groundedness and relevance fail differently. If retrieval returns strong evidence for the wrong subtopic, the model may produce a grounded answer that does not resolve the user’s question. If retrieval returns partial evidence and the model fills gaps with inference, the answer may sound relevant while losing groundedness. Both are common when the query is ambiguous, the retrieved set is noisy, or the model over-generalises from nearby context.

This split is easiest to see in evaluation review. A response that accurately summarises a policy document but never addresses the user’s requested exception handling is relevant failure, not grounding failure. A response that directly answers the question but adds version numbers, dates, or causes that are absent from the retrieved text is grounding failure, not relevance failure. Treating them separately helps you diagnose whether the issue is retrieval, generation, or both.

For teams building RAG systems, this also means one metric can improve while the other gets worse. Tightening the prompt to force citation may raise groundedness but produce timid, incomplete answers. Broadening generation can improve topical coverage but increase unsupported detail. The evaluation harness should be able to show both effects, rather than collapsing them into a single “good or bad” score.

What practitioners should measure when both matter

Groundedness is strongest when you score individual claims, not just the full response. That makes it easier to spot which sentence, clause, or detail lacks support. Answer relevance is strongest when you judge the response against the user’s intent, not the retrieved passages alone. That usually means you need one rubric for evidence support and another for task completion, even if the same human or automated reviewer applies both.

For practitioners comparing systems, the useful question is not which metric is “better”, but which failure mode is more expensive. If your RAG system is used for policy, legal, clinical, or operational guidance, unsupported facts are usually the higher-risk failure. If it is used for search, summarisation, or triage, answering the wrong question may be the more common user pain point. The right balance depends on the use case.

Practitioner Guidance

What to verify: Review groundedness and answer relevance separately on the same sample set. If a response fails relevance but passes groundedness, the retrieval or query interpretation layer needs attention. If it passes relevance but fails groundedness, the generation layer is overreaching beyond the evidence.

Decision rule: Treat a high groundedness score as necessary but not sufficient for a good RAG answer. Only trust a response when it both answers the question asked and keeps every material claim inside the evidence boundary.

Practitioner takeaway: Groundedness protects you from unsupported facts, while answer relevance protects you from answering the wrong question, and RAG quality requires both to be true at once.

Related resources from NHI Mgmt Group

• Guide: Permission-Aware RAG Guide
• RAG Evaluation
• What is the difference between policy evaluation and vector filtering in RAG?
• How should teams choose an evaluation framework for RAG applications?
• What breaks when RAG systems are deployed without continuous evaluation?