Work

Randomized smoothing for (parallel) stochastic optimization

Year: 2012
Type: conference-paper
Abstract: By combining randomized smoothing techniques with accelerated gradient methods, we obtain convergence rates for stochastic optimization procedures, both in expectation and with high probability, that have optimal dependence on the variance of the gradient estimates. To the best of our knowledge, these are the first variance-based rates for non-smooth optimization. A combination of our techniques with recent work on decentralized optimization yields order-optimal parallel stochastic optimization algorithms. We give applications of our results to several statistical machine learning problems, providing experimental results (in the full version of the paper) demonstrating the effectiveness of our algorithms.
Language: English
Open Access status: bronze
OpenAlex ID: W2002517365
FWCI: 0.913
Cites: 35
Cited by: 123
Related to: 10