Human-in-the-Loop Reinforcement Learning to Track Excavation Paths for a Hydraulic Excavator in a Short Period of Time
Year: 2025
Type: article
Abstract: The construction industry faces a severe shortage of skilled operators. One promising approach to mitigate this problem is to automate construction machinery, particularly hydraulic excavators, which have a wide range of applications. However, it is challenging to automate hydraulic excavators due to the difficulty of modeling their nonlinear dynamics and soil interactions, which makes difficult to design a controller that tracks excavation paths precisely and quickly. Reinforcement learning (RL) is promising solution because it can learn to track excavation paths in a purely data-driven way, but simply applying RL may lead to an impractically long data collection time due to its low data efficiency. Moreover, since controlling the hydraulic excavator is a safety-critical task, the safety of RL is also a considerable challenge. In light of this background, we propose a path-tracking controller based on a human-in-the-loop RL system. While some human intervention is required, we can collect data safely and efficiently. Our experimental results demonstrate that the path-tracking controller assisted by a skilled human operator learned to excavate gravelly, sandy, and cobbly soils in 12, 18, and 21 minutes of data collection, respectively, and dispensed with human assistance afterward. Moreover, the results of safety analysis experiments show that the trained controller succeeded in performing 128 successive excavations (38 minutes) without any safety violation or human assistance.
Source: IEEE Access
Authors: Tatsuya Mori, Tatsuya Yoshimoto, Hiroshi Yoshida
Institution: NEC (Japan)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,075
OpenAlex ID: W4411949195