Work

Covariance Matrix Adaptation MAP-Annealing: Theory and Experiments

Year: 2024

Type: article

Abstract: Single-objective optimization algorithms search for the single highest quality solution with respect to an objective. Quality diversity (QD) optimization algorithms, such as Covariance Matrix Adaptation MAP-Elites (CMA-ME), search for a collection of solutions that are both high quality with respect to an objective and diverse with respect to specified measure functions. However, CMA-ME suffers from three major limitations highlighted by the QD community: prematurely abandoning the objective in favor of exploration, struggling to explore flat objectives, and having poor performance for low-resolution archives. We propose a new QD algorithm, CMA MAP-Annealing (CMA-MAE), and its differentiable QD variant, CMA-MAE via a Gradient Arborescence (CMA-MAEGA), that address all three limitations. We provide theoretical justifications for the new algorithm with respect to each limitation. Our theory informs our experiments, which support the theory and show that CMA-MAE achieves state-of-the-art performance and robustness on standard QD benchmark and reinforcement learning domains.

Source: ACM Transactions on Evolutionary Learning and Optimization

DOI: 10.1145/3665336

Authors: Shihan Zhao, Bryon Tjanaka, Matthew C. Fontaine, Stefanos Nikolaidis

Institution: University of Southern California

Language: English

Topic: Evolutionary Algorithms and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Science Foundation Graduate Research Fellowship Program

Award: G6695183303