Work

Deliberative Searcher: Improving LLM Reliability via Reinforcement Learning with Constraints

Year: 2026

Type: conference-paper

Abstract: Large language models with search capabilities frequently exhibit miscalibrated confidence, producing incorrect answers with high certainty.We present Deliberative Searcher, a reasoning-primary framework that integrates search operations into chain-of-thought generation while maintaining explicit confidence calibration.Our method employs constrained reinforcement learning with adaptive Lagrangian multipliers to jointly optimize correctness and reliability.Experiments across five benchmarks demonstrate substantial improvements: our 7B model reduces average false-certain rates from 54% in baselines to 2%, while our 72B variant achieves competitive accuracy with closedsource models and reduces false-certain rates to 9%.The well-calibrated confidence scores also enable more efficient test-time compute: instead of standard majority voting, we use confidence-weighted aggregation and match the performance of 16-sample majority voting with only 4 samples, a 4× reduction in inference compute.These results establish calibrated confidence as a foundation for both trustworthy outputs and adaptive test-time compute, demonstrating the value of the proposed constrained RL framework in search-augmented language models.

Source: Proceedings of the conference - Association for Computational Linguistics. Meeting

DOI: 10.18653/v1/2026.acl-long.199

Authors: Zhenyun Yin, Shujie Wang, Xuhong Wang, Xingjun Ma, Yingchun Wang

Institutions: Fudan University, Shanghai Artificial Intelligence Laboratory

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Natural Science Foundation of China, National Science and Technology Major Project

Award: G5650502627