Year: 2026
Type: conference-paper
Abstract: Deploying large language models (LLMs) in resource-constrained environments is hindered by heavy computational and memory requirements.We present LBLLM, a lightweight binarization framework that achieves effective W(1+1)A4 quantization through a novel threestage quantization strategy.The framework proceeds as follows: (1) initialize a high-quality quantized model via PTQ; (2) quantize binarized weights, group-wise bitmaps, and quantization parameters through layer-wise distillation while keeping activations in full precision; and (3) training learnable activation quantization factors to dynamically quantize activations to 4 bits.This decoupled design mitigates interference between weight and activation quantization, yielding greater training stability and better inference accuracy.LBLLM, trained only using 0.016B tokens with a single GPU, surpasses existing state-of-the-art binarization methods on W2A4 quantization settings across tasks of language modeling, commonsense QA, and language understanding.These results demonstrate that extreme low-bit quantization of LLMs can be both practical and highly effective without introducing any extra high-precision channels nor rotational matrices commonly used in recent PTQ-based works, offering a promising path toward efficient LLM deployment on resource-limited situations.
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166841334