Year: 2026
Type: conference-paper
Abstract: Reinforcement learning with verifiable rewards (RLVR) has delivered impressive gains in mathematical and multimodal reasoning and has become a standard post-training paradigm for contemporary language and vision-language models.However, the RLVR recipe introduces a significant risk of capability regression, where models forget foundational skills after prolonged training without employing regularization strategies.We empirically confirm this concern, observing that open-source reasoning models suffer performance degradation on core capabilities such as perception and faithfulness.While imposing regularization terms like KL divergence can help prevent deviation from the base model, these terms are calculated for the current task; thus, they do not guarantee broader knowledge.Meanwhile, commonly used experience replay across heterogeneous domains makes it nontrivial to decide how much training focus each objective should receive.To address this, we propose RE-CAP-a replay strategy with dynamic objective reweighting for general knowledge preservation.Our reweighting mechanism adapts in an online manner using short-horizon signals of convergence and instability, shifting the posttraining focus away from saturated objectives and toward underperforming or volatile ones.Our method is end-to-end and readily applicable to existing RLVR pipelines without training additional models or heavy tuning.Extensive experiments on benchmarks using Qwen2.5-VL-3Band Qwen2.5-VL-7Bdemonstrate the effectiveness of our method, which not only preserves general capabilities but also improves reasoning by enabling more flexible trade-offs among in-task rewards.A-OKVQA AesBench VStar VisOnly OCRBench
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166860602