Year: 2014
Type: preprint
Abstract: It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), and regularization. In this paper we revisit the computational complexity of training neural networks from a modern perspective. We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training certain types of neural networks.
Source: arXiv (Cornell University)
Authors: Livni, Roi, Shalev-Shwartz, Shai, Shamir, Ohad
Institutions: Hebrew University of Jerusalem, Weizmann Institute of Science
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Award: G6207652689
OpenAlex ID: W2143859313
