Work

Faithful but Wrong: Validating LLM Faithfulness Judges for Retrieval-Augmented Generation Under Corrupted Context

Year: 2026
Type: preprint
Abstract: LLM-judged faithfulness is widely used to certify RAG answers, yet it is rarely checked against human labels, and it is unclear what it rewards when the retrieved context is wrong. We measure faithfulness, correctness, refusal, and adoption of injected falsehoods on the same answers under controlled context perturbations, and audit the faithfulness judge against two human annotators. From SQuAD v2.0 we build 75 answerable and 20 unanswerable questions, with six context conditions for answerable questions (clean, random distractors, related distractors, heavy noise, contradictory evidence, answer-absent) and three prompt mitigations. Three open-weight generators (Qwen3-8B, DeepSeek-R1-Distill-Qwen-32B, Llama-3.1-8B) produce 4,230 context-conditioned answers, scored with Ragas by a local 27B judge and rule-based checks, plus two cross-family API judges (Gemini, Claude). For Qwen3-8B, distractors had no large effect (equivalent to zero within +/-10 points under lenient matching). In all three models, a contradictory passage was the damaging perturbation: under baseline and refusal prompts, correctness fell by 20-35 points and models repeated the injected falsehood in 23-48% of answers, almost never flagging the conflict. The local judge rated 86-94% of these answers faithful. Human annotators agreed closely with each other (kappa = 0.96 on supported versus not supported). Re-weighted within each condition, the local judge agreed with humans on 91-92% of Qwen3-8B answers (kappa = 0.62-0.69), but its errors concentrated in its unfaithful verdicts. Gemini and Haiku had numerically higher agreement on the same answers, though not significantly so. A judge that mostly agrees with humans about grounding can still certify a wrong answer, so faithfulness judgments cannot stand in for correctness. The effect of prompt mitigations depends on the model.
Language: English
Open Access status: green
OpenAlex ID: W7235229527
Cites:
Cited by:
Related to: 10