Work

Quantifying the effects of environment and population diversity in multi-agent reinforcement learning

Year: 2022

Type: article

Abstract: Generalization is a major challenge for multi-agent reinforcement learning. How well does an agent perform when placed in novel environments and in interactions with new co-players? In this paper, we investigate and quantify the relationship between generalization and diversity in the multi-agent domain. Across the range of multi-agent environments considered here, procedurally generating training levels significantly improves agent performance on held-out levels. However, agent performance on the specific levels used in training sometimes declines as a result. To better understand the effects of co-player variation, our experiments introduce a new environment-agnostic measure of behavioral diversity. Results demonstrate that population size and intrinsic motivation are both effective methods of generating greater population diversity. In turn, training with a diverse set of co-players strengthens agent performance in some (but not all) cases.

Source: Autonomous Agents and Multi-Agent Systems

DOI: 10.1007/s10458-022-09548-8

Authors: Kevin R. McKee, Joel Z. Leibo, Charlie Beattie, Richard Everett

Institution: Google DeepMind (United Kingdom)

Language: English

Topic: Reinforcement Learning in Robotics

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,780

Funder: DeepMind