Work

Learning to Plan with Reinforcement Learning

Year: 2025
Type: dissertation
Abstract: This thesis investigates the domain of learning to plan within reinforcement learning. Traditional planning algorithms, such as Monte Carlo Tree Search, are typically handcrafted and lack the flexibility inherent in biological planning systems. To address this limitation, the Thinker algorithm is proposed. A central theme of this work is the shift in focus from designing specialized planning algorithms to augmenting the environment. By treating planning as an explicit action within an augmented environment, the agent is enabled to learn to plan using standard model-free reinforcement learning. Additionally, this thesis examines the interpretability of planning behaviour of RL agents trained by different algorithms. Finally, this thesis extends the Thinker algorithm to Large Language Models trained on mathematical reasoning tasks, demonstrating that similarly structured augmented environments significantly reduce computational costs while maintaining competitive performance on reasoning tasks.
Language: English
Open Access status: green
OpenAlex ID: W7159592709
Cites: 91
Cited by:
Related to: 10