Work

DeAR: Dual-Stage Document Reranking with Reasoning Agents via LLM Distillation

Year: 2025
Type: conference-paper
Abstract: Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets, yet single models often struggle to balance fine-grained relevance scoring with holistic cross-document analysis.We propose DeepAgentRank (DEAR), an open-source framework that decouples these tasks through a dual-stage approach, achieving superior accuracy and interpretability.In Stage 1, we distill token-level relevance signals from a frozen 13B LLaMA teacher into a compact {3, 8}B student model using a hybrid of cross-entropy, RankNet, and KL divergence losses, ensuring robust pointwise scoring.In Stage 2, we attach a second LoRA adapter and fine-tune on 20K GPT-4o-generated chain-of-thought permutations, enabling listwise reasoning with natural-language justifications.Evaluated on TREC-DL19/20, eight BEIR datasets, and NovelEval-2306, DEAR surpasses opensource baselines by +5.1 nDCG@5 on DL20 and achieves 90.97 nDCG@10 on NovelEval, outperforming GPT-4 by +3.09.Without fine-tuning on Wikipedia, DeAR also excels in open-domain QA, achieving 54.29 Top-1 accuracy on Natural Questions, surpassing baselines like MonoT5, UPR, and RankGPT.Ablations confirm that dual-loss distillation ensures stable calibration, making DEAR a highly effective and interpretable solution for modern reranking systems. 1 .
Language: English
Open Access status: hybrid
OpenAlex ID: W4416035045
FWCI: 3.151
Cites: 29
Cited by: 6
Related to: 10