Better Scores, Worse Grounding: Hidden Regressions after Fine-Tuning in Dialogue Fact Verification
Year: 2026
Type: conference-paper
Abstract: In dialogue fact verification (DFV), responses often depend on prior turns for correct interpretation, yet systems are still judged mainly by aggregate benchmark scores.We study a hidden grounding regression: aggregate Macro-F1 improves after fine-tuning while previously correct, context-dependent pronoun cases become newly wrong and show stronger premise-side sensitivity than cases that remain correct.On three referent-annotated audit sets constructed from DialFact and FaithDial, we audit six encoder-only verifiers before and after matched source-specific fine-tuning through predictiontransition analysis and the Premise-Preference Score (PPS), a control-adjusted masking diagnostic.Fine-tuning improves Macro-F1 across the six-model/three-evaluation-set panel, yet newly regressed cases show stronger premiseside sensitivity than stable-correct cases under PPS in 17 of 18 evaluated comparisons, showing that aggregate gains can conceal regressions on a controlled dialogue-grounding audit.
Authors: Hyunkyung Park, Arkaitz Zubiaga
Institution: University of London
Topic: Speech and dialogue systems
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7214030851
FWCI:
Cites:
Cited by:
Related to: