Year: 2026
Type: conference-paper
Abstract: Small language models (SLMs) struggle with complex reasoning because exploration is expensive under tight compute budgets.We introduce Semantic Diversity -Exploration-Exploitation (SD-E 2 ), a reinforcement learning framework that makes exploration explicit by optimizing semantic diversity in generated reasoning trajectories.Using a frozen sentence-embedding model, SD-E 2 assigns a diversity reward that captures (i) the coverage of semantically distinct solution strategies and (ii) their average pairwise dissimilarity in embedding space, rather than surface-form novelty.This diversity reward is combined with outcome correctness and solution efficiency in a z-score-normalized multi-objective objective that stabilizes training.On GSM8K, SD-E 2 surpasses the base Qwen2.5-3B-Instructand strong GRPO baselines (GRPO-CFL and GRPO-CFEE) by +27.4,+5.3, and +1.5 percentage points, respectively, while discovering on average 9.8 semantically distinct strategies per question.We further improve MedM-CQA to 49.64% vs 38.37 for base and show gains on the harder AIME benchmark , reaching 13.28% vs. base 6.74%.These results indicate that rewarding semantic novelty yields a more compute-efficient exploration-exploitation signal for training reasoning-capable SLMs.By introducing cognitive adaptation (adjusting the reasoning process structure rather than per-token computation), SD-E 2 offers a complementary path to efficiency gains in resource-constrained models.
Authors: Kshitij Mishra, Nils Lukas, Salem Lahlou
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140095155