Work

Weighted Gradient Coding with Leverage Score Sampling

Year: 2020

Type: conference-paper

Abstract: A major hurdle in machine learning is scalability to massive datasets. Approaches to overcome this hurdle include compression of the data matrix and distributing the computations. Leverage score sampling provides a compressed approximation of a data matrix using an importance weighted subset. Gradient coding has been recently proposed in distributed optimization to compute the gradient using multiple unreliable worker nodes. By designing coding matrices, gradient coded computations can be made resilient to stragglers, which are nodes in a distributed network that degrade system performance. We present a novel weighted leverage score approach, that achieves improved performance for distributed gradient coding by utilizing an importance sampling.

Source: IEEE International Conference on Acoustics Speech and Signal Processing

DOI: 10.1109/icassp40776.2020.9054153

Authors: Neophytos Charalambides, Mert Pilanci, Alfred O. Hero

Institutions: University of Michigan, Stanford University

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funder: Army Research Office

Awards: G408915191, G5385673730, G7452299184, G8257659198