Work

Online estimation of the inverse of the Hessian for stochastic optimization with application to universal stochastic Newton algorithms

Year: 2024

Type: preprint

Abstract: This paper addresses second-order stochastic optimization for estimating the minimizer of a convex function written as an expectation. A direct recursive estimation technique for the inverse Hessian matrix using a Robbins-Monro procedure is introduced. This approach enables to drastically reduces computational complexity. Above all, it allows to develop universal stochastic Newton methods and investigate the asymptotic efficiency of the proposed approach. This work so expands the application scope of secondorder algorithms in stochastic optimization.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2401.10923

Authors: Godichon-Baggioni, Antoine, Lu, Wei, Portier, Bruno

Institutions: Laboratoire de Probabilités, Statistique et Modélisation, Laboratoire de Mathématiques de l'INSA de Rouen, Institut National des Sciences Appliquées Rouen Normandie

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4390895039

Cites: 31

Cited by: 6

Related to: 10