Reinforcement Learning with Chromatic Networks
Year: 2019
Type: preprint
Abstract: We present a new algorithm for finding compact neural networks encodingreinforcement learning (RL) policies. To do it, we leverage in the novel RL
setting the theory of pointer networks and ENAS-type algorithms for
combinatorial optimization of RL policies as well as recent evolution
strategies (ES) optimization methods, and propose to define the combinatorial
search space to be the the set of different edge-partitionings (colorings) into
same-weight classes. For several RL tasks, we manage to learn colorings
translating to effective policies parameterized by as few as 17 weight
parameters, providing 6x compression over state-of-the-art compact policies
based on Toeplitz matrices. We believe that our work is one of the first
attempts to propose a rigorous approach to training structured neural network
architectures for RL problems that are of interest especially in mobile
robotics with limited storage and computational resources.
Source: arXiv (Cornell University)
Authors: Xingyou Song, Krzysztof Choromański, Jack Parker-Holder, Yunhao Tang, Wenbo Gao, Aldo Pacchiano, Tamás Sarlós, Deepali Jain, Yuxiang Yang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2957460253