Controlled Pre-Reranking Evidence Coverage and the Downstream Value of Cross-Encoder Reranking in Multi-Hop RAG
Year: 2026
Type: preprint
Abstract: Standard Retrieval-Augmented Generation (RAG) evaluations average performance across queries, hiding whether errors come from missing candidates in first-stage retrieval, context-window cutoffs, or generator failures. We isolate when and why cross-encoder reranking helps multi-hop question answering through a controlled paired study of 1,700 HotpotQA queries, holding the first-stage BM25 pool (top-20), prompt, and reasoning generator fixed while comparing top-5 selection with and without a cross-encoder. Although reranking improves overall Exact Match by +8.1 percentage points, 86% of queries end in ties---meaning the aggregate gain comes from a small minority of queries. Specifically, pre-reranking candidate coverage acts as a hard gate: complete candidate pools drive 97% of net gains, whereas incomplete pools see no meaningful improvement. Downstream gains come entirely from promoting missing gold passages across the top-5 boundary rather than reordering documents already inside the context window, while reasoning-token telemetry shows that hidden thinking tokens outnumber visible answer tokens by 63:1.
Author: Tiwari, Rishab Nitin
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225459752