Year: 2022
Type: dissertation
Abstract: Distributed optimization has proven to be an effective scheme to tackle numerous emerging problems that are surged by the development of digital technologies and mobile smart devices.Examples include control of robot swarms, distributed and large scale machine learning, estimation in sensor networks, and management of smart grids.In such setups, the individual interests of agents (machines) have to be collectively optimized under stringent restrictions on communication bandwidth, computation resources, and security concerns.To balance the efforts on optimizing local objectives and cooperation among the network is challenging while necessary to achieve a global solution.Many endeavors have been made to design efficient algorithms that can harness the computation power of a heterogeneous network, in terms of both machine hardware and data distribution.The majority of existing algorithms can be categorized as first-order methods, where the computation process is guided only by the objective (sub) gradient information.Such practices have found many successes due to their economical computation costs and uninvolved implementation.Nevertheless, first-order algorithms suffer from a common drawback: slow convergence rate, especially when the objective curvature is skewed.This induces large overall iteration numbers and prevents the implementation of first order algorithms to applications where accurate solutions are sought in few rounds, such as real time computation on embedded systems and communication expensive learning practices.In this dissertation, we shift our attention to second-order methods that aim to accelerate the iterative process through use of objective curvature.The main challenge of designing second-order methods in large scale networks is threefold: (i) distributed implementation, (ii) computation and communication balance, and (iii) network synchronization.These challenges are not unique to devising decentralized second-order methods, but become more pronounced due to the involvement of Hessian matrices, solving linear systems, backtracking line search, and approximation of curvature information etc.We address these challenges in several steps.Chapter 3 tackles the distributed convex composite problem by presenting two algorithms that distributedly approximate the curvature information from the perspective of Method of Multipliers and Alternating Direction Method of Multipliers.By increasing the accuracy of approximation, we show both theoretically and empirically that the proposed algorithms achieve faster convergence rate, and thus Dušan introduced me to Petros and that's when we started talking business.After a little while, Nick joined in and little did I know at the time that I completed the best advisor collection anyone could ask.
Author: Li, Yichuan
Institution: University of Illinois Urbana-Champaign
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Keywords: distributed optimization, second-order methods, Hessian approximation, communication efficiency, large-scale networks, alternating direction method of multipliers, augmented Lagrangian method, convergence rate, network synchronization, computation-communication trade-off, backtracking line search
Open Access status: green
OpenAlex ID: W7220760802
