Multi-USV cooperative target encirclement through learning-based distributed transferable policy and experimental validation
Year: 2024
Type: article
Abstract: Coordinated strategies for unmanned surface vehicles (USVs) to encircle an escaping target is a key research focus in maritime operations. In this paper, we propose a distributed, transferable multi-agent reinforcement learning (MARL) algorithm to address this challenge. The algorithm achieves the target encirclement for multi-agent swarm based on limited observational information, while there are random obstacles in the scenario. A centralized training decentralized execution (CTDE) deep reinforcement learning (DRL) framework is introduced. Also, the framework facilitates transfer of policies to larger swarms. To balance individual and collective benefits, we propose a knowledge-embedded compound reward function. Nonlinear mapping terms are added to the reward function to meet the requirements of the encirclement task at different stages. Furthermore, to improve training effectiveness, we implement a parallel training method and a multi-agent policy-sharing (MAPS) mechanism for isomorphic agents. Multiple tests indicate the effectiveness of the proposed algorithm. We developed a framework for deploying the learning-based policies in real-world scenarios. The MARL-based algorithm proposed in this paper has been successfully deployed in an indoor pool.
Source: Ocean Engineering
Authors: Chenming Zhang, Rijie Zeng, Bin Lin, Yibo Zhang, Wei Xie, Weidong Zhang
Institution: Shanghai Jiao Tong University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $4,030
OpenAlex ID: W4405812101