Work

Post-Training for Reasoning LLMs with Reinforcement Learning: A Stability–Efficiency Perspective

Year: 2026
Type: conference-paper
Abstract: Reinforcement Learning (RL)-based post-training has become a key mechanism for improving reasoning Large Language Models (LLMs), especially when automated verifiers enable Reinforcement Learning with Verifiable Reward (RLVR). Recent methods, however, are increasingly modular: many apparent algorithmic advances recombine a small set of recurring components, including group-relative baselines, trust-region control, dense or process-level feedback, replay, and throughput-oriented data routing. This survey organises this literature through a unified design map with three axes: learning-signal source and density, optimisation regime and trust-region control, and structural granularity and agentic scope. We further use a compact method-by-motif view to expose common stability–efficiency trade-offs across representative methods. Rather than treating stability as a formal convergence guarantee, we focus on auditable failure signatures, including reward degeneracy, ratio-tail growth, entropy collapse, length bias, and off-policy mismatch. We conclude with practical evaluation guidance based on matched compute, controlled length, and diagnostic reporting, and identify open problems in dense-signal causality, critic-free off-policy RL, and long-horizon credit assignment.
Language: English
Open Access status: hybrid
OpenAlex ID: W7216413247
FWCI:
Cites: 8
Cited by:
Related to: 10