Keyword

gradient compression

Description: A communication-reduction technique in distributed machine learning that encodes or sparsifies model gradients exchanged during training.

Primary topic: Privacy-Preserving Technologies in Data (37%)

Related topics: Privacy-Preserving Technologies in Data (37%), Stochastic Gradient Optimization Techniques (34%)

OpenAlex ID: keywords/gradient-compression

QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

2016 · Alistarh, Dan, Grubic, Demjan, et al. · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · 1,189

Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

2017 · Lin, Yujun, Han, Song, et al. · arXiv (Cornell University) · 1,134

Communication-efficient federated learning via knowledge distillation

2022 · Chuhan Wu, Fangzhao Wu, et al. · Nature Communications · 623

Deep Anomaly Detection for Time-Series Data in Industrial IoT: A Communication-Efficient On-Device Federated Learning Approach

2020 · Yi Liu, Sahil Garg, et al. · IEEE Internet of Things Journal · 592

Sparse Communication for Distributed Gradient Descent

2017 · Alham Fikri Aji, Kenneth Heafield · Conference on Empirical Methods in Natural Language Processing (EMNLP) · 584

signSGD: Compressed Optimisation for Non-Convex Problems

2018 · Bernstein, Jeremy, Wang, Yu-Xiang, et al. · arXiv (Cornell University) · 541

Error Feedback Fixes SignSGD and other Gradient Compression Schemes

2019 · Karimireddy, Sai Praneeth, Rebjock, Quentin, et al. · arXiv (Cornell University) · 316

PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

2019 · Vogels, Thijs, Karimireddy, Sai Praneeth, et al. · arXiv (Cornell University) · 176

AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Training

2018 · Chia-Yu Chen, Jungwook Choi, et al. · Proceedings of the AAAI Conference on Artificial Intelligence · 167

CREAT: Blockchain-Assisted Compression Algorithm of Federated Learning for Content Caching in Edge Computing

2020 · Laizhong Cui, Xiaoxin Su, et al. · IEEE Internet of Things Journal · 146

Federated Learning with Compression: Unified Analysis and Sharp Guarantees

2020 · Haddadpour, Farzin, Kamani, Mohammad Mahdi, et al. · arXiv (Cornell University) · 140

Magnamosis: magnetic compression anastomosis with comparison to suture and staple techniques

2009 · Ramin Jamshidi, Jacob T. Stephenson, et al. · Journal of Pediatric Surgery · 136

Peak compression in reversed-phase gradient elution

2006 · Uwe D. Neue, Daniel H. Marchand, et al. · Journal of Chromatography A · 117

A Framework for Evaluating Client Privacy Leakages in Federated Learning

2020 · Wenqi Wei, Ling Liu, et al. · Lecture notes in computer science · 104

2018 · Jiawei Jiang, Fangcheng Fu, et al. · Proceedings - ACM-SIGMOD International Conference on Management of Data · 102

8-Bit Approximations for Parallelism in Deep Learning

2015 · Dettmers, Tim · arXiv (Cornell University) · 101

Adaptive Federated Learning With Gradient Compression in Uplink NOMA

2020 · Haijian Sun, Xiang Ma, et al. · IEEE Transactions on Vehicular Technology · 98

Distributed Learning with Compressed Gradient Differences

2019 · Mishchenko, Konstantin, Gorbunov, Eduard, et al. · arXiv (Cornell University) · 98

Communication-Efficient Federated Learning for Anomaly Detection in Industrial Internet of Things

2020 · Yi Liu, Neeraj Kumar, et al. · Globecom · 92

A Framework for Evaluating Gradient Leakage Attacks in Federated Learning

2020 · Wei, Wenqi, Liu, Ling, et al. · arXiv (Cornell University) · 92

Communication-efficient distributed SGD with Sketching

2019 · Ivkin, Nikita, Rothchild, Daniel, et al. · arXiv (Cornell University) · 86

A Network-Centric Hardware/Algorithm Co-Design to Accelerate Distributed Training of Deep Neural Networks

2018 · Youjie Li, Jongse Park, et al. · Proceedings of the ... annual International Symposium on Microarchitecture/Proceedings of the annual International Symposium on Microarchitecture · 86

Natural Compression for Distributed Deep Learning

2019 · Horvath, Samuel, Ho, Chen-Yu, et al. · arXiv (Cornell University) · 81

Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients

2019 · Sun, Jun, Chen, Tianyi, et al. · arXiv (Cornell University) · 80

Cost and Cost Effectiveness of Venous and Pressure Ulcer Protocols of Care

2001 · Morris D. Kerstein, Eric Gemmen, et al. · Disease Management & Health Outcomes · 79

Works count: 914

Citation count: 12,370