Year: 2025
Type: article
Abstract: Vision Transformer (ViT) has established itself as the leading model in image processing, and quantization offers a practical way to deploy ViTs on memory-constrained devices. This technique maps float32 inference to lower bits such as int8. However, existing studies often use uniform bit-width across ViT encoders, leading to sub-optimal allocation and accuracy loss. This study aims to optimize bit-width allocation by leveraging the observed variation in quantization sensitivity among different ViT encoders. We propose PoMQ-ViT, a method that dynamically assigns bit-width to ViT encoders based on their sensitivity. It uses Pareto Optimization to balance accuracy and inference speed. Experiments on the ImageNet Dataset show PoMQ-ViT outperforms uniform bit-width methods across DeiT and Swin models. Within the same computation budget, it also achieves 0.1–1.5 percentage points higher accuracy than other mixed-precision paradigms across different models. This work demonstrates PoMQ-ViT’s effectiveness in ViT quantization, providing a practical solution for resource-constrained deployments.
Source: Applied Sciences
DOI: 10.3390/app15189856
Authors: Zhiqiang Wu, Zhong Zhao
Institution: Northeastern University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,896
OpenAlex ID: W4414093498