Cross-Lingual Transfer Learning in Large Language Models: Multilingual Representations and Low-Resource Adaptation
Year: 2026
Type: preprint
Abstract: Over the past two decades, the emergence of the Transformer architecture has led to explosive development in the field of multilingual machine translation. And in the past five years, breakthroughs in large language models (LLMs) have made this field more professional and efficient. This paper provides a comprehensive synthesis of recent developments in cross-lingual transfer learning for multilingual natural language processing, with a focus on LLMs. It examines how linguistic knowledge is transferred across languages, the evolution of pretrained multilingual models, zero-shot and few-shot transfer techniques, and prompt-based adaptation strategies. The paper further analyses technical challenges related to multilingual neural machine translation, lifelong learning in cross-lingual settings, and the evaluation metrics used to assess transferability. Key findings highlight the influence of linguistic similarity on transfer performance, the partial success of multilingual training in aligning lexical and grammatical representations, and the persistent difficulties faced in achieving deep cross-lingual knowledge integration. Strategies such as self-distillation, translation-based methods, and prompt tuning are reviewed, alongside emerging trends in low-resource language support and multilingual representation learning. By identifying the limitations of current approaches and proposing directions for future research, this survey contributes to the broader goal of building more robust, equitable, and linguistically versatile language models.
Institutions: Dalhousie University, University of Sussex, Xiamen University, University of Liverpool, Beijing University of Technology, Emory University, Guangdong Medical College, Imperial College London, University of Bologna, Georgia Institute of Technology, University of Illinois Urbana-Champaign, National Chung Hsing University, The University of Texas at Dallas, Purdue University West Lafayette, University College Dublin, University of Malaya, University of California System, East China Normal University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7119645940