Work

Provably Efficient Long-Horizon Exploration in Monte Carlo Tree Search through State Occupancy Regularization

Year: 2024

Type: preprint

Abstract: Monte Carlo tree search (MCTS) has been successful in a variety of domains, but faces challenges with long-horizon exploration when compared to sampling-based motion planning algorithms like Rapidly-Exploring Random Trees. To address these limitations of MCTS, we derive a tree search algorithm based on policy optimization with state occupancy measure regularization, which we call {\it Volume-MCTS}. We show that count-based exploration and sampling-based motion planning can be derived as approximate solutions to this state occupancy measure regularized objective. We test our method on several robot navigation problems, and find that Volume-MCTS outperforms AlphaZero and displays significantly better long-horizon exploration properties.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2407.05511

Authors: Schramm, Liam, Boularias, Abdeslam

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Awards: RI: CAREER: Task-Oriented Model Identification for Robust Robotic Manipulation, NRI: Robust and Efficient Physics-based Learning and Reasoning in Degraded Environments

OpenAlex ID: W4400480650

Cites:

Cited by:

Related to: 10