Work

Exploring Generalization in Deep Learning

Year: 2017

Type: preprint

Abstract: With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1706.08947

Authors: Neyshabur, Behnam, Bhojanapalli, Srinadh, McAllester, David, Srebro, Nathan

Institutions: Toyota Technological Institute at Chicago, Toyota Technological Institute

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W2963739978

Cites: 25

Cited by: 844

Related to: 10