Granularity Masking: When Paper-Level Evaluation Misses Passage-Level Degradation
Year: 2026
Type: preprint
Abstract: This study examines when paper-level retrieval metrics mask passage-level degradation. Across 1,309 QASPER questions, a controlled within-paper reversal preserved every paper ranking while reducing mean passage nDCG@10 by 0.175. Paper-level metrics detected the fallback and index-shift interventions in aggregate, although masking persisted for individual questions.
Author: Mamun, S M Abdulla Al
Institution: University of Manchester
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Keywords: question answering, NDCG
Open Access status: green
OpenAlex ID: W7236705877
