Work

Group-Merger: A LoRA-based Framework for Multilingual Continual Learning

Year: 2026

Type: conference-paper

Abstract: Multilingual continual learning (MCL) is crucial for enabling language models to adapt across diverse linguistic environments while retaining knowledge over time.Existing parameter isolation methods allocate language-specific modules but fail to leverage cross-lingual transfer, leading to inefficient parameter growth and poor generalization.Model merging based approaches suffer from severe performance degradation as the number of language-specific tasks increases, due to interference between linguistic and task-specific knowledge.To address these challenges, we propose Group-Merger, a framework that employs group-wise merging to balance parameter efficiency and continual learning performance.Our framework mitigates catastrophic forgetting across languages while enabling knowledge transfer.Extensive experiments on multilingual evaluation benchmarks demonstrate superior performance compared to existing methods.

Source: Workshop on Multilinguality in the Era of Large Language Models (MeLLM)

DOI: 10.18653/v1/2026.mellm-1.30

Authors: Weijian yi, Hongliang Li, Jinan Xu

Institution: Beijing Jiaotong University

Language: English

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funder: National Natural Science Foundation of China