Year: 2026
Type: conference-paper
Abstract: Low-rank decomposition, particularly Singular Value Decomposition (SVD), is a pivotal technique for mitigating the storage and computational demands of Large Language Models (LLMs).However, prevalent SVD-based approaches overlook the critical phenomenon that decomposition errors exhibit significant disparity across different components of the parameter matrix, often leading to suboptimal approximation.Furthermore, existing methods lack a direct metric to evaluate the importance of individual weight matrices.To address these limitations, we propose Duo-SVD (Duallevel Optimization SVD), a novel training-free framework that synergizes optimization at both the column and the module levels.First, Duo-SVD incorporates a Column-Preserving Strategy that explicitly retains columns exhibiting high decomposition errors, while applying lowrank approximation solely to those with lower errors.Second, at the module level, we employ a Module-Adaptive Allocation Strategy that formulates ratio allocation as a global constrained optimization problem based on perturbation-induced model deviation.Extensive experiments demonstrate that Duo-SVD consistently outperforms state-of-the-art SVDbased baselines and structured pruning methods, establishing it as a superior paradigm for efficient LLM compression.
Institution: Shanghai Jiao Tong University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4416058052