Work

RankMean: Module-Level Importance Score for Merging Fine-tuned LLM Models

Year: 2024

Type: conference-paper

Abstract: Traditionally, developing new language models (LMs) capable of addressing multiple tasks involves fine-tuning pre-trained LMs using a wide collection of datasets, a process that often incurs significant computational expenses.Model merging emerges as a cost-effective alternative, allowing the integration of existing models fine-tuned on different tasks into a single model that performs well across all tasks, eliminating the need for additional training.In this paper, we propose RankMean, an algorithm for merging fine-tuned LMs without requiring any downstream data.RankMean determines merging coefficients based on the relative rankings of weight change magnitudes and applies these coefficients for module-wise integration of various fine-tuned models.Our experimental results demonstrate that RankMean outperforms existing baseline methods on multiple benchmarks.The code is available at github.com/VITA-Group/RankMean.

DOI: 10.18653/v1/2024.findings-acl.104

Authors: Gabriel Perin, Xuxi Chen, Shusen Liu, Bhavya Kailkhura, Zhangyang Wang, Brian Gallagher

Institutions: Universidade de São Paulo, The University of Texas at Austin, Lawrence Livermore National Laboratory

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: U.S. Department of Energy, Fundação de Amparo à Pesquisa do Estado de São Paulo, Laboratory Directed Research and Development, Lawrence Livermore National Laboratory