Deep Reinforcement Learning based tracking behavior for underwater vehicles
Year: 2018
Type: dissertation
Abstract: This thesis introduces the use of Machine Learning, specifically Reinforcement\nLearning, to create a model-free tracking property for Remotely Operated Vehicles\n(ROV). In detail, the ROV is trained by a RL algorithm to track an aruco\nmarker, using online implementation of a Computer Vision (CV) algorithm as a\ndetection property. The main motivation behind this enterprise is the contribution\nto increased autonomy in underwater operations, by introducing model-free\nautonomous tracking behavior to underwater vehicles. This approach of implementation\nrequires minimal human intervention during operation, while significantly\nreducing prior human control programming effort. Firstly, a simulator based tracking\nbehavior training of the ROV was done prior to conducting physical experiments\nwith a real ROV in the MC-laboratory at NTNU. The ROV used for the experimental\ntests is a BlueROV2, which is highly customizable and fitting for R&D\npurposes.\n\nThe theory presented in this thesis lays the groundwork for the many reasonings\ndone in this project s course, including the choice of RL method. The RL algorithm\nchosen for training the tracking behavior is a online Python implementation of\nthe type Proximal Policy Optimization (PPO) algorithm. The tracking behavior is\ntrained on a simulator, which is a Python script based on typical OpenAI s simulator\narchitecture. The resulting tracking performance is then evaluated by studying the\nevolution of accumulated rewards and ROV s trajectory plots. While the resulting\nperformance did show to have some weak sides, it was, however, feasible enough to\ntest the trained model in a real-world setting.\n\nHowever, the real-world experiments did not yield positive tracking results, considering\nthe ROV performed in a random manner instead of favorably moving towards\nthe aruco marker. Several challenges described in the theory-section proved to be\nprevalent during the lab experiments, which caused the disruption in the real-world\ntracking performance. Nonetheless, based on experience gained from both the simulations\nand real-world experiments, various proposals for further work was devised\nand highlighted. Especially, is the importance of appropriate reward function design\nunderlined.
Author: Abir Khan
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2899046323
