Year: 2020
Type: preprint
Abstract: We present a novel machine learning architecture that uses the exponential of a single input-dependent matrix as its only nonlinearity. The mathematical simplicity of this architecture allows a detailed analysis of its behaviour, providing robustness guarantees via Lipschitz bounds. Despite its simplicity, a single matrix exponential layer already provides universal approximation properties and can learn fundamental functions of the input, such as periodic functions or multivariate polynomials. This architecture outperforms other general-purpose architectures on benchmark problems, including CIFAR-10, using substantially fewer parameters.
Source: arXiv (Cornell University)
Authors: Fischbacher, Thomas, Comsa, Iulia M., Potempa, Krzysztof, Firsching, Moritz, Versari, Luca, Alakuijala, Jyrki
Institutions: Google (United States), University of Pisa
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W3047657192