Controlled Pre-Reranking Evidence Coverage and the Downstream Value of Cross-Encoder Reranking in Multi-Hop RAG
Year: 2026
Type: preprint
Abstract: Standard Retrieval-Augmented Generation (RAG) evaluations conflate upstream candidate starvation, boundary displacement, and generation failures. We isolate when and why cross-encoder reranking improves multi-hop QA in a paired study of $N = 1,700$ HotpotQA (dev_distractor) queries ($3,400$ generations) over $|\mathcal{D}| = 66,576$ articles. Fixing the BM25 pool ($K = 20$), prompt, and generator (Google Gemini 2.5 Flash thinking model, logged under harness identifier gemini-3.8-flash; $T = 0.0$, max_tokens $= 1,024$), we compare BM25 top-5 (No-CE) against cross-encoder/ms-marco-MiniLM-L-6-v2 (CE; $K = 20 \to N = 5$). Reranking improves Exact Match by $\Delta\text{EM} = +8.12\text{ pp}$ ($p_{\text{HB}} = 3.52 \times 10^{-20}$) and token $F_1$ by $\Delta F_1 = +10.52\text{ pp}$ ($p_{\text{HB}} = 8.53 \times 10^{-29}$), with $86.1\%$ concordant EM ties. Pre-reranking completeness gates efficacy: Complete Pools ($n = 1,111$) drive $97.1\%$ of net EM wins ($+12.06\text{ pp}$, $p_{\text{HB}} = 5.30 \times 10^{-20}$), whereas Incomplete Pools ($n = 589$) yield $+0.68\text{ pp}$ EM ($p_{\text{HB}} = 0.4240$) and a borderline, family-dependent $+1.23\text{ pp}$ $F_1$ ($p_{\text{raw}} = 0.0340$, $p_{\text{HB}} = 0.0340$ at $m = 2$, $p_{\text{HB,Jt}} = 0.0680$). Observational transitions show symmetric payoffs for PROMOTED ($n = 380$, $+41.58\text{ pp}$ EM) vs. Bridge DEMOTED ($n = 34$, $-41.18\text{ pp}$ EM), while REORDERED_ONLY ($n = 93$) yields $0.00\text{ pp}$. Finally, telemetry reveals a $63:1$ thinking-to-visible token ratio ($782,227$ vs. $12,411$), requiring ample output budget to prevent silent truncation.
Author: Tiwari, Rishab Nitin
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225459752