Year: 2023
Type: dissertation
Abstract: In this thesis, we investigate the use of search based algorithms with deep neural \nnetworks to tackle a wide range of problems ranging from board games to video \ngames and beyond. Drawing inspiration from AlphaGo, the first computer program \nto achieve superhuman performance in the game of Go, we developed a new algorithm AlphaZero. AlphaZero is a general reinforcement learning algorithm that \ncombines deep neural networks with a Monte Carlo Tree search for planning and \nlearning. Starting completely from scratch, without any prior human knowledge \nbeyond the basic rules of the game, AlphaZero managed to achieve superhuman \nperformance in Go, chess and shogi. Subsequently, building upon the success of AlphaZero, we investigated ways to extend our methods to problems in which the rules \nare not known or cannot be hand-coded. This line of work led to the development \nof MuZero, a model-based reinforcement learning agent that builds a deterministic \ninternal model of the world and uses it to construct plans in its imagination. We \napplied our method to Go, chess, shogi and the classic Atari suite of video-games, \nachieving superhuman performance. MuZero is the first RL algorithm to master \na variety of both canonical challenges for high performance planning and visually complex problems using the same principles. Finally, we describe Stochastic \nMuZero, a general agent that extends the applicability of MuZero to highly stochastic environments. We show that our method achieves superhuman performance in \nstochastic domains such as backgammon and the classic game of 2048 while matching the performance of MuZero in deterministic ones like Go.
Author: Antonoglou, Ioannis
Institution: University College London
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: DeepMind
OpenAlex ID: W7011227822