Work

Decentralized Projected Riemannian Stochastic Recursive Momentum Method for Nonconvex Optimization

Year: 2025

Type: conference-paper

Abstract: This paper studies decentralized optimization over a compact submanifold within a communication network of n nodes, where each node possesses a smooth non-convex local cost function, and the goal is to jointly minimize the sum of these local costs. We focus particularly on the online setting, where local data is processed in real-time as it streams in, without the need for full data storage. We propose a decentralized projected Riemannian stochastic recursive momentum (DPRSRM) method that employs local hybrid stochastic gradient estimators and uses the network to track the global gradient. DPRSRM achieves an oracle complexity of O(epsilon^(-3/2)), outperforming existing methods that have at most O(epsilon^(-2)) complexity. Our method requires only O(1) gradient evaluations per iteration for each local node and does not require restarting with a large batch gradient. Furthermore, we demonstrate the effectiveness of our proposed methods compared to state-of-the-art ones through numerical experiments on principal component analysis problems and low-rank matrix completion.

Source: Proceedings of the AAAI Conference on Artificial Intelligence

DOI: 10.1609/aaai.v39i11.33218

Authors: Kangkang Deng, Jiang Hu

Institutions: National University of Defense Technology, University of California, Berkeley

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

Funders: National Natural Science Foundation of China, Office of Naval Research

Awards: G3277760478, G3633144423, G8876996369