Year: 2026
Type: conference-paper
Abstract: Post-training quantization (PTQ) has emerged as a promising approach for reducing the memory footprint and computational cost of large language models (LLMs), enabling efficient deployment without full model retraining.However, existing PTQ methods struggle to simultaneously support weight-activation joint quantization and extreme low-bit weight quantization.This limitation primarily arises from the depth of LLMs and their strong cross-layer dependencies, which cause quantization errors to propagate and accumulate across layers, ultimately leading to significant performance degradation.In this paper, we present ACBQ, a simple yet effective framework that simultaneously addresses weight-activation joint quantization and extreme weight quantization.We first propose a granular quantization strategy that treats self-attention and FFN as separate quantization units with module-specific optimization objectives.To mitigate the propagation and accumulation of quantization errors across layers, we introduce an adaptive cross-block quantization strategy that explicitly accounts for crosslayer dependencies by encouraging consistency across blocks.Extensive experiments across diverse LLMs, including OPT and the LLaMA family, demonstrate that ACBQ achieves superior performance under both W4A4 and highly aggressive W2 settings, while incurring negligible additional computational overhead.
Institution: Northeastern University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166881448