Work

When does gradient descent with logistic loss find interpolating two-layer networks?

Year: 2020

Type: preprint

Abstract: We study the training of finite-width two-layer smoothed ReLU networks for binary classification using the logistic loss. We show that gradient descent drives the training loss to zero if the initial loss is small enough. When the data satisfies certain cluster and separation conditions and the network is wide enough, we show that one step of gradient descent reduces the loss sufficiently that the first result applies.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2012.02409

Authors: Chatterji, Niladri S., Long, Philip M., Bartlett, Peter L.

Institutions: University of California, Berkeley, Google (United States)

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: National Science Foundation

Awards: G2647087871, Collaboration on the Theoretical Foundations of Deep Learning, Foundations of Data Science Institute

OpenAlex ID: W3112962553

Cites: 60

Cited by: 5

Related to: 20