Work

QSLR: Post-Training Compression via Quantized Sparse and Low-Rank Factorization

Year: 2025

Type: article

Abstract: The growing scale and complexity of transformer-based foundation models have led to significant challenges in deploying them efficiently, particularly on resource-constrained environments. While recent methods such as post-training pruning or sparse + low-rank decomposition reduce model size without retraining, they still rely on full-precision weights, limiting gains in memory bandwidth and latency. In this paper, we propose QSLR, a unified post-training quantization (PTQ) framework that combines outlier-aware pruning via sparse and low-rank decomposition with component-wise Hessian-aware quantization. Each decomposed component (sparse matrix, low-rank left factor, low-rank right factor) is quantized independently using projected Hessians, with an efficient approximation to eliminate redundant Hessian computations. We further optimize the quantization parameters via Hessian-weighted grid search to minimize second-order quantization loss. Experiments on LLaMA2-7B and ViT-Base demonstrate that QSLR achieves up to 5×model compression with minimal accuracy degradation, consistently outperforming state-of-the-art pruning or quantization methods.

Source: IEEE Access

DOI: 10.1109/access.2025.3615473

Author: Eunji Kwon

Institution: Kookmin University

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,075

Funder: National Research Foundation of Korea