Moniagenttinen vahvistusoppiminen autonomisten miehittämättömien pinta-alusten kohteen etsintä- ja torjuntatehtävissä
Year: 2026
Type: dissertation
Abstract: Unmanned surface vehicles (USVs) have become prominent in naval warfare during the Russo-Ukrainian war. USVs can be operated remotely or with different levels of autonomy, either individually or as part of a swarm. A swarm can be described as a decentralized group that acts in coordination. Advances in machine learning (ML) and more powerful onboard computing for ML inference enable greater autonomy and more advanced swarming behavior for USVs. One approach for enabling autonomous swarming behavior is multi-agent reinforcement learning (MARL). MARL is a sub-area of ML in which multiple agents learn through trial and error to choose actions that maximize rewards provided by the environment. It has been widely applied in robotics and has been explored for the control of USVs. This thesis investigates the applicability of MARL for controlling a swarm of USVs in a challenging two-phase mission in which the swarm must first locate and then destroy an adversarial surface combatant capable of hard-kill measures and jamming. The thesis reviews the relevant literature on MARL, swarming, naval warfare, and USVs. It then designs and implements a training framework based on multi-agent proximal policy optimization (MAPPO) for the task. Finally, the resulting controller is evaluated against an idealized rule-based reference controller, given full knowledge of the target. The aim is to assess whether MARL can provide a feasible and robust solution to the task, how its performance compares with that of the reference controller, and whether it can discover behaviors beyond those built into the rule-based approach. The resulting MARL-based controller was able to locate and destroy the target with a success rate of 0.88, while the ideal reference controller achieved 1.00. The difference can be attributed to the agents not learning an effective search pattern and thus occasionally failing to locate the target. Under jamming, the MARL-based controller maintained a success rate of 0.78, demonstrating robustness. Although trained only against a static target, the controller retained a 0.05 success rate against a moving target. The training did not yield novel tactics, but it did produce emergent evasive behavior. The results indicate that MAPPO can successfully train a large number of agents for a challenging, two-phase long-horizon task. At the same time, they reveal clear limitations in tactical refinement and search behavior.
Source: Aaltodoc (Aalto University)
Author: Leskinen, Niki
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7161919734