Year: 2019
Type: preprint
Abstract: The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive. Numerous rigorous attempts have been made to explain generalization, but available bounds are still quite loose, and analysis does not always lead to true understanding. The goal of this work is to make generalization more intuitive. Using visualization methods, we discuss the mystery of generalization, the geometry of loss landscapes, and how the curse (or, rather, the blessing) of dimensionality causes optimizers to settle into minima that generalize well.
Source: arXiv (Cornell University)
Institution: University of Maryland, College Park
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funders: National Science Foundation, Alfred P. Sloan Foundation, National Institutes of Health, Defense Advanced Research Projects Agency, Multidisciplinary University Research Initiative, Office of Naval Research, Division of Electrical, Communications and Cyber Systems, Air Force Office of Scientific Research
Awards: SpecEES: Spatio-Spectral Sensing with Wideband Feature Extraction Arrays, AitF: EXPL: Collaborative Research: Approximate Discrete Programming for Real-Time Systems, Collaborative Research: BAMM: Baseband Accelerators for Massive Multiple-Input Multiple-Output (MIMO) Technology, CAREER: Hardware Accelerated Bayesian Inference via Approximate Message Passing: A Bottom-Up Approach, Harmonic Analysis and Machine Learning for Emergency Response, NeTS: Small: Collaborative Research: BRICK: Breaking the I/O and Computation Bottlenecks in Massive MIMO Base Stations
OpenAlex ID: W2948920178