Exploring Model Compression Techniques for Efficient Inference of Large Language Models
Year: 2025
Type: preprint
Abstract: The growing size and complexity of large language models (LLMs) have driven significant advances in natural language processing tasks. However, the extensive computational, memory, and energy demands of these models pose challenges for deployment in resource-constrained environments. Model compression techniques offer a promising solution by reducing model size and inference latency while maintaining high performance. This survey provides a comprehensive overview of the key compression strategies applied to LLMs, including pruning, quantization, knowledge distillation, low-rank factorization, and emerging hybrid approaches. We discuss the theoretical foundations, practical implementations, and trade-offs associated with each method. Additionally, we present prominent evaluation metrics and benchmark datasets used to assess compressed models, ensuring fair and reproducible comparisons. Furthermore, we outline key challenges and promising future research directions, such as adaptive compression techniques, hardware-aware optimizations, and sustainable AI practices. By consolidating recent advancements in model compression, this survey aims to guide researchers and practitioners toward developing efficient LLMs capable of delivering robust performance in diverse real-world applications.
Source: HAL
Authors: Leanne, Mercia, Brody, Stephany, Cass, Nicola
Institution: University of Edinburgh
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4414908785