Year: 2017
Type: preprint
Abstract: We demonstrate that training ResNet-50 on ImageNet for 90 epochs can be achieved in 15 minutes with 1024 Tesla P100 GPUs. This was made possible by using a large minibatch size of 32k. To maintain accuracy with this large minibatch size, we employed several techniques such as RMSprop warm-up, batch normalization without moving averages, and a slow-start learning rate schedule. This paper also describes the details of the hardware and software of the system used to achieve the above performance.
Source: arXiv (Cornell University)
Authors: Akiba, Takuya, Suzuki, Shuji, Fukuda, Keisuke
Institution: Preferred Networks (Japan)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Keywords: ResNet50, ImageNet, deep neural network training
Open Access status: green
OpenAlex ID: W2769856846
