Work

On the Growth of the Parameters of a Class of Approximating ReLU Neural Networks

Year: 2026
Type: article
Abstract: This work focuses on the analysis of a class of fully connected feed forward ReLU neural networks as they approximate a given, smooth function. In contrast to conventionally studied universal approximation properties under increasing architectures, e.g., in terms of width or depth of the networks, we are concerned with the asymptotic growth of the norm of the parameters of approximating networks. Such results are of interest, e.g., for error analysis or consistency results for neural network training. The main result of our work is that, for a class of ReLU architectures with state of the art approximation error, the norm of the realizing parameters grows at most polynomially. The obtained rate with respect to a normalized network size is compared to existing results and is shown to be superior in most cases, in particular for high dimensional input.
Institution: University of Graz
Language: English
Open Access status: hybrid
Estimated APC paid: $3,090
OpenAlex ID: W4399992053
FWCI: 1.884
Cites: 20
Cited by: 4
Related to: 9