Work

Decentralized and parallel primal and dual accelerated methods for stochastic convex programming problems

Year: 2021

Type: article

Abstract: We introduce primal and dual stochastic gradient oracle methods for decentralized convex optimization problems. Both for primal and dual oracles, the proposed methods are optimal in terms of the number of communication steps. However, for all classes of the objective, the optimality in terms of the number of oracle calls per node takes place only up to a logarithmic factor and the notion of smoothness. By using mini-batching technique, we show that the proposed methods with stochastic oracle can be additionally parallelized at each node. The considered algorithms can be applied to many data science problems and inverse problems.

Source: Journal of Inverse and Ill-Posed Problems

DOI: 10.1515/jiip-2020-0068

Authors: Darina Dvinskikh, Alexander Gasnikov

Institutions: Moscow Institute of Physics and Technology, Weierstrass Institute for Applied Analysis and Stochastics, Institute for Information Transmission Problems

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Russian Foundation for Basic Research, Russian Science Foundation, Ministry of Science and Higher Education of the Russian Federation