Year: 2026
Type: article
Abstract: This paper proposes a method called Neural Energy Descent (NED) via neural network (NN) evolution equations for a wide class of deep learning (DL) problems. We demonstrate that the training process of DL can be reformulated as the evolution of the network parameters governed by a partial differential equation (PDE) where the steady-state solution yields a solution for DL. This equation corresponds to a gradient descent flow of a variational problem and hence the proposed PDE solves an energy minimization problem to obtain a global minimizer of DL. This gives a novel interpretation and solution for DL optimization. The computational complexity of the proposed energy descent method can be enhanced by randomly sampling the spatial domain of the PDE. Numerical examples are provided to demonstrate the numerical advantages of NED over stochastic gradient descent (SGD) method and Adam.
Authors: Wenrui Hao, Chunmei Wang, Xingjian Xu, Haizhao Yang
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Awards: CAREER: Deep Learning Based Scientific Computing: Mathematical Theory and Algorithms, Collaborative Research: Personalized Modeling of Alzheimer’s Disease, CAREER: Primal-Dual Weak Galerkin Finite Element Methods, Collaborative Research: Friedrichs Learning: Mathematical Foundation and Applications, G4666933312, Collaborative Research: Friedrichs Learning: Mathematical Foundation and Applications, G8876996369, G9000014436
OpenAlex ID: W4321592971