An in-depth look at approximation via deep and narrow neural networks
Year: 2025
Type: article
Abstract: In 2017, Hanin and Sellke showed that the class of arbitrarily deep, real-valued, feed-forward and ReLU-activated networks of width forms a dense subset of the space of continuous functions on , with respect to the topology of uniform convergence on compact sets, if and only if holds. To show the necessity, a concrete counterexample function was used. In this note we actually approximate this very by neural networks in the two cases and around the aforementioned threshold. We study how the approximation quality behaves if we vary the depth and what effect (spoiler alert: dying neurons) cause that behavior.
Source: Neurocomputing
Authors: Joris Dommel, Sven A. Wegner
Institution: Universität Hamburg
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,670
Funder: Universität Hamburg
OpenAlex ID: W4415215890