LOAD: Enabling Low-Bit Group Quantization for Vision Transformers via Adaptive Data Formats
Year: 2026
Type: article
Abstract: Vision Transformers (ViTs) achieve state-of-the-art performance across various vision tasks but pose significant deployment challenges due to their high computational complexity and memory footprint. Post-training quantization (PTQ) offers an efficient, retraining-free compression strategy; however, existing solutions typically enforce a uniform quantization scheme (fixed bit-width and data format) across all layers, constraining the potential accuracy-efficiency trade-off. We propose LOAD-ViT , a novel PTQ framework that leverages both mixed-precision and mixed-data-format quantization to allocate resources at a fine granularity based on layer sensitivity. LOAD-ViT optimizes per-layer configurations via a calibration-based sensitivity metric and a constrained search algorithm, minimizing accuracy loss under strict hardware budgets. To support this scheme, we design a dedicated hardware accelerator featuring heterogeneous compute engines and lightweight format-conversion units, facilitating the efficient execution of mixed formats. Extensive evaluations on various ViT models and benchmarks demonstrate that LOAD-ViT achieves up to 5.3 × speedup and 4.2 × energy savings compared to state-of-the-art quantization accelerators, with negligible accuracy degradation.
DOI: 10.1145/3857350
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7221860195