Work

Gradient Descent Converges to Minimizers

Year: 2016

Type: preprint

Abstract: We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.1602.04915

Authors: Lee, Jason D., Simchowitz, Max, Jordan, Michael I., Recht, Benjamin

Institution: University of California, Berkeley

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Science Foundation, U.S. Department of Energy, Cisco Systems, VMware, Directorate for Computer and Information Science and Engineering, Defense Advanced Research Projects Agency, Samsung, Office of Naval Research, Army Research Office

Awards: G1519837837, G3026391708, G4332978919, G4654920669, G4713059963, G4793493537, Making Sense at Scale with Algorithms, Machines, and People, G674549868, CIF: Small: Decoding Error-Correcting Codes using Large-Scale Decomposition Methods, G7103680314, G7452299184, CAREER: Efficient Atomic Decompositions of Massive Data Sets, G862988727, G8876996369, G932903589

OpenAlex ID: W2283214199

Cites: 28

Cited by: 123

Related to: 20