Year: 2016
Type: preprint
Abstract: We propose a novel coding theoretic framework for mitigating stragglers in distributed learning. We show how carefully replicating data blocks and coding across gradients can provide tolerance to failures and stragglers for Synchronous Gradient Descent. We implement our schemes in python (using MPI) to run on Amazon EC2, and show how we compare against baseline approaches in running time and generalization error.
Source: arXiv (Cornell University)
Institution: The University of Texas at Austin
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: CIF: Medium: Collaborative Research: Content Delivery over Heterogeneous Networks: Fundamental Limits and Distributed Algorithms, CIF: Small: Index Coding and Matrix Factorizations, CIF: Small: Sparsity in Quadratic Optimization through Low-Rank Approximations, CIF: Small: Collaborative Research: Design and Analysis of Novel Compressed Sensing Algorithms via Connections with Coding Theory
OpenAlex ID: W4394644480