Work

Expansive natural neural gradient flows for energy minimization

Year: 2026

Type: article

Abstract: This paper develops expansive gradient dynamics in deep neural network (DNN)-induced mapping spaces. Specifically, we introduce a framework for minimizing a broad class of energy functionals in an abstract Hilbert space setting, with applications spanning PDE-based problems and supervised learning. The approach hinges on a Hilbert space metric in the full diffeomorphism mapping space, which can be viewed as a generalized Wasserstein-2 metric. Within this framework, we study a projected gradient descent method operating on DNN-parameterized sets. More importantly, we develop an adaptive expansion strategy to dynamically enlarge the DNN architecture. This expansion mechanism aims to enhance the alignment of the neural manifold-induced natural gradient direction, as well as possible, with the ideal Hilbert space gradient descent direction by leveraging the fact that we can evaluate projections of the Hilbert space gradient. We demonstrate the efficacy of the proposed strategy on several simple model problems involving energies arising in the context of function approximation, physics informed learning, and model reduction. Additionally, we highlight the importance of assembling the neural flow matrix based on the inner product of the ambient Hilbert space. The presented algorithms represent the simplest specifications of the broader framework, with a detailed analysis deferred to future work.

Source: Journal of Computational Physics

DOI: 10.1016/j.jcp.2026.115117

Authors: Wolfgang Dahmen, Wuchen Li, Yuankai Teng, Zhu Wang

Language: English

Topic: Model Reduction and Neural Networks

Subfield: Statistical and Nonlinear Physics

Field: Physics and Astronomy

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $3,980

Funders: National Science Foundation, Deutsche Forschungsgemeinschaft, Air Force Office of Scientific Research