Work

Deep Learning at Scale

Year: 2019

Type: conference-paper

Abstract: This work presents a novel approach to distributed training of deep neural networks (DNNs) that aims to overcome the issues related to mainstream approaches to data parallel training. Established techniques for data parallel training are discussed from both a parallel computing and deep learning perspective, then a different approach is presented that is meant to allow DNN training to scale while retaining good convergence properties. Moreover, an experimental implementation is presented as well as some preliminary results.

Source: Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)

DOI: 10.1109/empdp.2019.8671552

Authors: Paolo Viviani, Maurizio Drocco, Daniele Baccega, Iacopo Colonnelli, Marco Aldinucci

Institutions: Noesis Solutions (Belgium), Pacific Northwest National Laboratory, University of Turin

Language: English

Topic: Stochastic Gradient Optimization Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Università degli Studi di Torino, Horizon 2020 Framework Programme, European Regional Development Fund

Award: G1127086505