And the Bit Goes Down: Revisiting the Quantization of Neural Networks
Year: 2020
Type: conference-paper
Abstract: In this paper, we address the problem of reducing the memory footprint of convolutional network architectures. We introduce a vector quantization method that aims at preserving the quality of the reconstruction of the network outputs rather than its weights. The principle of our approach is that it minimizes the loss reconstruction error for in-domain inputs. Our method only requires a set of unlabelled data at quantization time and allows for efficient inference on CPU by using bytealigned codebooks to store the compressed weights. We validate our approach by quantizing a high performing ResNet-50 model to a memory size of 5 MB (20 X compression factor) while preserving a top-1 accuracy of 76:1% on ImageNet object classification and by compressing a Mask R-CNN with a 26 X factor.1
Source: HAL
Institution: Meta (United States)
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2996390082