Work

On the existence of infinitely many realization functions of non-global local minima for artificial neural networks with ReLU activation

Year: 2026

Type: article

Abstract: Gradient descent (GD) type optimization schemes are standard tools for training fully connected feedforward artificial neural networks (ANNs) with rectified linear unit (ReLU) activation and can be considered as temporal discretizations of solutions of gradient flow (GF) differential equations. It is known that, for shallow ReLU ANNs, the risk of every bounded GF trajectory converges under suitable assumptions to the risk of a critical point, a zero of the corresponding gradient function. This motivates studying necessary and sufficient conditions in GF and GD convergence analysis for critical points and their dependence on problem parameters, such as the target function.In the first main result, we prove for shallow ReLU ANNs that for every dimension $ d \in \mathbb{N} $, every $

Source: Discrete and Continuous Dynamical Systems - B

DOI: 10.3934/dcdsb.2026.104221

Authors: Shokhrukh Ibragimov, Arnulf Jentzen, Timo Kröger, Adrian Riekert

Institutions: University of Münster, Chinese University of Hong Kong, Chinese University of Hong Kong, Shenzhen

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: European Commission, Deutsche Forschungsgemeinschaft, National Natural Science Foundation of China

Awards: G1652504919, EXC 2044: Mathematik Münster: Dynamik – Geometrie – Struktur, G6938060324, G8724757341