LoRA-Adv: Boosting Text Classification in Large Language Models Through Adversarial Low-Rank Adaptations
Year: 2025
Type: article
Abstract: Low-rank adaptation (LoRA), a paradigm bridging the gap between large language models and fine-tuning, has demonstrated effectiveness across various natural language processing tasks. The LoRA algorithm updates only a small number of model parameters, significantly reducing the consumption of computational resources. Although the LoRA algorithm achieves feasible performance, it is susceptible to the influence of training samples. To enhance the stability of the LoRA algorithm and improve the performance of classification models, this paper introduces a novel adversarially enhanced LoRA algorithm, named LoRA-Adv. Specifically, in the LoRA-Adv algorithm, we leverage carefully designed adversarial perturbations to expand the diversity of training samples. Minimizing the adversarial training loss for both normal and adversarial samples significantly reinforces the model’s robustness. To verify the performance of the LoRA-Adv algorithm, multiple state-of-the-art large language models were employed, and all experimental results confirmed that LoRA-Adv significantly enhances the classification performance of the models. Therefore, the LoRA-Adv algorithm represents a significant advancement in fine-tuning large language models, enhancing both their stability and classification accuracy.
Source: IEEE Access
Authors: Hong Ye, Xialin Xie, Fenlong Xie, Jun Zuo, Chunyan Bu
Institution: State Grid Corporation of China (China)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,075
OpenAlex ID: W4411270486