Year: 2026
Type: preprint
Abstract: We show that the effective dynamics of the elitist $(1+M)$ genetic algorithm is, in the limit of small mutations, normalized gradient descent on the loss in the presence of anisotropic Gaussian white noise. In expectation, therefore, a simple mutation-selection genetic algorithm follows the gradient of the loss, without explicit calculation of gradients and without averaging over loss evaluations. The genetic algorithm is slower than gradient descent because of the noise that acts in directions transverse to the gradient. However, this slowdown is controlled not by the number of parameters of the search space but by the effective rank of the Hessian of the loss function. For the concentrated Hessian spectra observed in neural-network loss functions the effective rank can be far smaller than the number of parameters, which may explain why genetic algorithms can scale to large search spaces.
Source: arXiv (Cornell University)
Author: Whitelam, Stephen
Institutions: Lawrence Berkeley National Laboratory, Molecular Foundry
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166660130