How to Improve LLMs’ Performance on Specific Languages: A Perspective on LLM-Derived Language Similarity
Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) exhibit uneven performance across languages.In languagespecific applications, practitioners often rely on target-language corpora or cross-lingual transfer to achieve better performance.However, traditional linguistic typology, commonly used as a transfer language selection strategy in previous studies, may not align with LLM's perception of language similarity.This work proposes LLM-based language similarity as a novel perspective for selecting effective fine-tuning languages.We construct a framework to quantify the similarity within each language pair through both the lenses of language-specific performance patterns and cross-lingual transferability, ultimately deriving three similarity score matrices.Moreover, we observe a counterintuitive phenomenon: super-additive transfer effect, where finetuning on a certain language yields higher performance than fine-tuning directly on the target language.Additionally, due to the absence of an existing dataset meeting our experimental requirements, we construct and release the M4CQ-Pro dataset, which features domain-diverse distribution of 135 tasks and content consistency across 31 languages (including over 20 medium-and low-resource languages), with 61518 manually reviewed highquality questions per language.We evaluate our approach on representative multilingual LLMs and results show that all three LLM-based similarity measures effectively guide fine-tuning language selection, outperforming traditional linguistic similarity, with the integrated measure achieving the best results.Our approach provides not only a novel perspective on language similarity, but also practical baselines for selecting fine-tuning languages.
Authors: Xinhe Shi, Qingcheng Zeng, Weihao Xuan, Linchao Zhu
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Award: G339005287
OpenAlex ID: W7166891439