Year: 2026
Type: conference-paper
Abstract: Recently, latent reasoning has been introduced into large language models (LLMs) to leverage rich information within a continuous space.However, without stochastic sampling, these methods inevitably collapse to deterministic inference, failing to discover diverse reasoning paths.To bridge the gap, we inject controllable stochasticity into latent reasoning via Gumbel-Softmax, restoring LLMs' exploratory capacity and enhancing their compatibility with Reinforcement Learning (RL).Building on this, we propose Latent Reasoning Policy Optimization (LEPO), a novel framework that applies RL directly to continuous latent representations.Specifically, in rollout stage, LEPO maintains stochasticity to enable diverse trajectory sampling, while in optimization stage, LEPO constructs a unified gradient estimation for both latent representations and discrete tokens.Extensive experiments show that LEPO significantly outperforms existing RL methods for discrete and latent reasoning.1
Institutions: Zhejiang University, Shanghai Jiao Tong University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166828694