Work

Efficient Frontier Selection via Reinforcement Learning for Exploring Unstructured Environments with Minimal Sensing

Year: 2026

Type: article

Abstract: In recent years, reinforcement learning (RL) has been applied to frontier-based exploration to enhance a robot’s decision-making policy and improve exploration performance. In this work, we address this scenario with the aim of pushing forward the finding of the optimal frontier selection policy in unknown, unstructured environments, with RL deployed for a minimal sensing drone setup. We propose a novel policy architecture, featuring an attention module that uses the global map features captured by a convolutional neural network together with local frontier features in the form of scalar values, trained end-to-end with a scoring network using the Proximal Policy Optimization algorithm over a 2D randomized unstructured environment. Our approach demonstrates improved exploration efficiency in the evaluated scenarios, as it surpasses purely heuristic-based frontier selection strategies used as baselines for other RL methods, achieving shorter paths than the Nearest Frontier, the Hybrid Approach, and the TARE local horizon, as well as one-shot sim-to-real policy deployment.

Source: Technologies

DOI: 10.3390/technologies14060365

Authors: Javier Melero Deza, Pedro Arias Perez, Guillermo GP-Lenza, Martín Molina, Pascual Campoy

Institutions: Centre for Automation and Robotics, Universidad Politécnica de Madrid, Universidade de Vigo

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $1,930

Funder: Comunidad de Madrid