A Systematic Mapping Study of Parameter-Efficient Fine-Tuning Approaches for Large Language Models
Year: 2026
Type: dissertation
Study design: Systematic Review
Abstract: Background: The rapid development of large language models (LLMs) has fueled the demand of efficient adaptation methods for downstream tasks. Traditional full fine-tuning methods are computationally expensive and resource intensive, especially for large-scale transformer models. Hence, parameter-efficient fine-tuning (PEFT) methods have drawn more attention, which can adapt efficiently by lightweight trainable components and selective parameter updates. Objective: This study aims to systematically investigate and classify existing PEFT approaches for LLMs, with emphasis on adaptation techniques, target architectures, application areas, evaluation benchmarks, and reported limitations. Method: In accordance with systematic mapping study (SMS) guidelines, an extensive literature search was conducted across scientific databases. The final set comprised 24 selected studies for detailed analysis to evaluate methodological patterns and to discuss the implications of the findings in relation to the defined research questions. Results: The field of PEFT has grown rapidly due to the computational challenge of large transformer models. The most popular approaches are Low-Rank Adaptation (LoRA) and reparameterisation, especially for decoder-only architectures, with a focus on memory efficiency and scalability. Quantisation aware methods, in particular Quantised Low-Rank Adaptation (QLoRA), could have strong performance with reduced memory requirements. Conclusion: PEFT has emerged as a core paradigm for scalable adaptation of LLMs. The field is moving towards more efficient, modular and hardware-aware to allow Artificial Intelligence (AI) to be more widely used. Theoretical gaps, consistency of evaluation and robustness in different settings should be filled in future research.
Source: Doria (University of Helsinki)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7163296531