In RAG evaluation, what is the difference between faithfulness and answer relevance?
Faithfulness asks whether every claim in the answer is supported by the retrieved context. Answer relevance asks whether the answer actually addresses the user's question. An answer can be faithful but off-topic, or on-topic but unsupported.
Explain it simply
Say a student asks, "What is the last date to pay the semester fee?" and the retrieved chunk is about hostel rules.
If the model replies "Hostel gates close at 10 pm", that is faithful, since it is exactly in the context, but not relevant to the question. If it replies "The last date is 15 July" with no support in the context, that is relevant but not faithful: it made it up.
You need both, plus a third check: context relevance, whether retrieval brought back useful chunks in the first place. Separating these metrics tells you where to fix the system. Low context relevance points to chunking or retrieval, low faithfulness points to the prompt or model behaviour, and low answer relevance points to how the question is handled.
Example
Q: Last date for semester fee? Context: hostel gate timings
A1: "Gates close at 10 pm." -> faithful, not relevant
A2: "Last date is 15 July." -> relevant, not faithfulWhat to say in the interview
"Faithfulness checks grounding in the retrieved context, answer relevance checks that the question is addressed, and context relevance checks retrieval quality. Tracking all three localises whether the retriever, prompt or generator needs fixing."