Work

Neural network integral representations with the ReLU activation function

Year: 2019

Type: preprint

Abstract: In this effort, we derive a formula for the integral representation of a shallow neural network with the ReLU activation function. We assume that the outer weighs admit a finite $L_1$-norm with respect to Lebesgue measure on the sphere. For univariate target functions we further provide a closed-form formula for all possible representations. Additionally, in this case our formula allows one to explicitly solve the least $L_1$-norm neural network representation for a given function.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1910.02743

Authors: Petrosyan, Armenak, Dereventsov, Anton, Webster, Clayton

Institutions: Oak Ridge National Laboratory, University of Tennessee at Knoxville

Language: English

Topic: Neural Networks and Applications

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, U.S. Department of Energy, UT-Battelle, Office of Science, Division of Mathematical Sciences, Advanced Scientific Computing Research, Oak Ridge National Laboratory

Awards: G1026679911, G121915436, G1645119126, G203625172, G2411683797, G2487984123, G313676650, G3809081345, G4370606643, G4423657506, G6231974842, G6344208662, G6677662844, G675320460, G8269158468, G8943143067

OpenAlex ID: W3115614799

Cites: 4

Cited by: 17

Related to: 10