Year: 2025
Type: conference-paper
Abstract: Recent studies have demonstrated the effectiveness of LLM test-time scaling.However, existing approaches to incentivize LLMs' deep thinking abilities generally require large-scale data or significant training efforts.Meanwhile, it remains unclear how to improve the thinking abilities of less powerful base models.In this work, we introduce S 2 R, an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.Specifically, we first initialize LLMs with iterative self-verification and self-correction behaviors through supervised fine-tuning on carefully curated data.The self-verification and self-correction skills are then further strengthened by outcome-level and process-level reinforcement learning with minimized resource requirements.Our results demonstrate that, with only 3.1k behavior initialization samples, Qwen2.5-math-7Bachieves an accuracy improvement from 51.0% to 81.6%, outperforming models trained on an equivalent amount of long-CoT distilled data.We also discuss the effect of different RL strategies on enhancing LLMs' deep reasoning.Extensive experiments and analysis based on three base models across both in-domain and out-of-domain benchmarks validate the effectiveness of S 2 R 1 .
Authors: Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, Jia Li
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4412944795