Year: 2019
Type: conference-paper
Abstract: Asynchronous parallel stochastic optimization for non-convex problems becomes more and more important in machine learning especially due to the popularity of deep learning. The Frank-Wolfe (a.k.a. conditional gradient) algorithms has regained much interest because of its projection-free property and the ability of handling structured constraints. However, our understanding of asynchronous stochastic Frank-Wolfe algorithms is extremely limited especially in the non-convex setting. To address this challenging problem, in this paper, we propose our asynchronous stochastic Frank-Wolfe algorithm (AsySFW) and its variance reduction version (AsySVFW) for solving the constrained non-convex optimization problems. More importantly, we prove the fast convergence rates of AsySFW and AsySVFW in the non-convex setting. To the best of our knowledge, AsySFW and AsySVFW are the first asynchronous parallel stochastic algorithms with convergence guarantees for solving the constrained non-convex optimization problems. The experimental results on real high-dimensional gray-scale images not only confirm the fast convergence of our algorithms, but also show a near-linear speedup on a parallel system with shared memory due to the lock-free implementation.
Authors: Bin Gu, Wenhan Xian, Heng Huang
Institution: University of Pittsburgh
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
Awards: SCH: EXP: Collaborative Research: Privacy-Preserving Framework for Publishing Electronic Healthcare Records, BIGDATA: Collaborative Research: IA: Big Imaging-Omics Data Mining Framework for Precision Medicine, G3143537441, SCH: INT: New Machine Learning Framework to Conduct Anesthesia Risk Stratification and Decision Support for Precision Health, III: Medium: Collaborative Research: Robust Large-Scale Electronic Medical Record Data Mining Framework to Conduct Risk Stratification for Personalized Intervention, III: Small: Robust Large-Scale Data Mining for Knowledge Discovery in Depression Thought Records, G6462069701, BIGDATA: IA: Collaborative Research: Asynchronous Distributed Machine Learning Framework for Multi-Site Collaborative Brain Big Data Mining, ABI Innovation: A New Automated Data Integration, Annotations, and Interaction Network Inference System for Analyzing Drosophila Gene Expression
OpenAlex ID: W2964607944