pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training
Year: 2026
Type: conference-paper
Abstract: Quantization-Aware Training from scratch has emerged as a promising approach for building efficient large language models (LLMs) with extremely low-bit weights (sub 2-bit), which can offer substantial advantages for edge deployment.However, existing methods still fail to achieve satisfactory accuracy and scalability.In this work, we identify a parameter democratization effect as a key bottleneck: the sensitivity of all parameters becomes homogenized, severely limiting expressivity.To address this, we propose pQuant, a method that decouples parameters by splitting linear layers into two specialized branches: a dominant 1-bit branch for efficient computation and a compact highprecision branch dedicated to preserving the most sensitive parameters.Through tailored feature scaling, we explicitly guide the model to allocate sensitive parameters to the highprecision branch.Furthermore, we extend this branch into multiple, sparsely-activated experts, enabling efficient capacity scaling.Extensive experiments indicate our pQuant achieves stateof-the-art performance in extremely low-bit quantization.
Authors: Wenzheng Zhang, Bingzheng Liu
Institutions: Peking University, Fudan University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166802776