Year: 2025
Type: conference-paper
Abstract: Existing pruning methods for large language models (LLMs) focus on achieving high compression rates while maintaining model performance.Although these methods have demonstrated satisfactory performance in handling a single user's compression request, their processing time increases linearly with the number of requests, making them inefficient for realworld scenarios with multiple simultaneous requests.To address this limitation, we propose a Univeral Model for Customized Compression (UniCuCo) for LLMs, which introduces a Strat-Net that learns to map arbitrary requests to their optimal pruning strategy.The challenge in training StratNet lies in the high computational cost of evaluating pruning strategies and the non-differentiable nature of the pruning process, which hinders gradient backpropagation for StratNet updates.To overcome these challenges, we leverage a Gaussian process to approximate the evaluation process.Since the gradient of the Gaussian process is computable, we can use it to approximate the gradient of the non-differentiable pruning process, thereby enabling StratNet updates.Experimental results show that UniCuCo is 28 times faster than baselines in processing 64 requests, while maintaining comparable accuracy to baselines.
Authors: Rongguang Ye, Ming Tang
Institution: Southern University of Science and Technology
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Awards: G2800729717, G522179584
OpenAlex ID: W4412888740