Variance Reduction and Low Sample Complexity in Stochastic Optimization via Proximal Point Method
Year: 2024
Type: preprint
Abstract: High-probability guarantees in stochastic optimization are often obtained only under strong noise assumptions such as sub-Gaussian tails. We show that such guarantees can also be achieved under the weaker assumption of bounded variance by developing a stochastic proximal point method. This method combines a proximal subproblem solver, which inherently reduces variance, with a probability booster that amplifies per-iteration reliability into high-confidence results. The analysis demonstrates convergence with low sample complexity, without restrictive noise assumptions or reliance on mini-batching.
Source: arXiv (Cornell University)
Author: Liang, Jiaming
Institution: University of Rochester
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Award: G1523888516
OpenAlex ID: W4391871589