Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
gradient accumulation
Description:
A deep-learning training technique that sums gradients across multiple smaller batches before updating model parameters, reducing memory requirements.
Related topics:
Stochastic Gradient Optimization Techniques (11%),
Advanced Neural Network Applications (10%),
Parallel Computing and Optimization Techniques (8%),
Topic Modeling (7%)
OpenAlex ID:
keywords/gradient-accumulation
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
2015
·
Courbariaux, Matthieu
, Bengio, Yoshua
, et al.
·
arXiv (Cornell University)
·
3,368
PDF
Extraction and recognition of artificial text in multimedia documents
2004
·
C. Wolf
, J.-M. Jolion
·
Pattern Analysis and Applications
·
224
FedSel: Federated SGD Under Local Differential Privacy with Top-k Dimension Selection
2020
·
Ruixuan Liu
, Yang Cao
, et al.
·
Lecture notes in computer science
·
181
PDF
MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth
2022
·
Cao, Chenjie
, Ren, Xinlin
, et al.
·
Transactions on machine learning research
·
162
Boosting Adversarial Transferability through Enhanced Momentum
2021
·
Xiaosen Wang
, Jiadong Lin
, et al.
·
British Machine Vision Conference (BMVC)
·
132
PDF
SWALP : Stochastic Weight Averaging in Low-Precision Training
2019
·
Yang, Guandao
, Zhang, Tianyi
, et al.
·
arXiv (Cornell University)
·
115
PDF
Accelerating Neural Network Training: A Brief Review
2024
·
Sahil Nokhwal
, Priyanka Chilakalapudi
, et al.
·
International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence (ISMSI)
·
89
PDF
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful
2025
·
Martin Marek
, Sanae Lotfi
, et al.
·
neural information processing systems
·
66
PDF
Automatic segmentation of brain metastases using T1 magnetic resonance and computed tomography images
2021
·
Dylan G Hsu
, Åse Ballangrud
, et al.
·
Physics in Medicine and Biology
·
47
Rapid Method to Account for Process Variation in Full-Chip Capacitance Extraction
2004
·
A. Labun
·
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
·
43
Mitigating Popularity Bias in Recommendation with Unbalanced Interactions: A Gradient Perspective
2022
·
Weijieying Ren
, Lei Wang
, et al.
·
Proceedings
·
38
PDF
Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
2021
·
Yuan, Kun
, Alghunaim, Sulaiman A.
, et al.
·
arXiv (Cornell University)
·
37
PDF
HybridGNN: A Self-Supervised Graph Neural Network for Efficient Maximum Matching in Bipartite Graphs
2024
·
Chun-Hui Pan
, Yi Qu
, et al.
·
Symmetry
·
34
PDF
A Comprehensive Performance Study of Large Language Models on Novel AI Accelerators
2023
·
Emani, Murali
, Foreman, Sam
, et al.
·
arXiv (Cornell University)
·
31
PDF
SortedNet: A Scalable and Generalized Framework for Training Modular Deep Neural Networks
2023
·
Valipour, Mojtaba
, Rezagholizadeh, Mehdi
, et al.
·
arXiv (Cornell University)
·
27
PDF
Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance
2023
·
Xinyu Piao
, Doangjoo Synn
, et al.
·
IEEE Access
·
25
PDF
Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
2024
·
Huang, Wenke
, Liang, Jian
, et al.
·
arXiv (Cornell University)
·
23
PDF
Parallel Computing of Spatio-Temporal Model Based on Deep Reinforcement Learning
2021
·
Zhiqiang Lv
, Jianbo Li
, et al.
·
Lecture notes in computer science
·
22
Improving adversarial transferability by temporal and spatial momentum in urban speaker recognition systems
2022
·
Hao Tan
, Zhaoquan Gu
, et al.
·
Computers & Electrical Engineering
·
21
Learning Rate Dropout
2022
·
Huangxing Lin
, Weihong Zeng
, et al.
·
IEEE Transactions on Neural Networks and Learning Systems
·
21
PDF
P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
2026
·
Hui, Mude
, Huang, Xin
, et al.
·
arXiv (Cornell University)
·
20
PDF
A Study on Energy Consumption in AI-Driven Medical Image Segmentation
2025
·
R. Prajwal
, S. J. Pawan
, et al.
·
Journal of Imaging
·
20
PDF
MAMGD: Gradient-Based Optimization Method Using Exponential Decay
2024
·
Nikita Sakovich
, Dmitry Aksenov
, et al.
·
Technologies
·
20
PDF
End-to-end Multiple Instance Learning with Gradient Accumulation
2022
·
Axel Andersson
, Nadezhda Koriakina
, et al.
·
IEEE International Conference on Big Data (BigData)
·
20
PDF
Measuring the Impact of Gradient Accumulation on Cloud-based Distributed Training
2023
·
Zimeng Huang
, Bo Jiang
, et al.
·
IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid)
·
19
Show more works
View as search
Works count:
269
Citation count:
5,330
Year
Topic
Type