Work

Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention

Year: 2025

Type: conference-paper

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in natural language processing but exhibit significant performance gaps among different languages.Most existing approaches to address these disparities rely on pretraining or fine-tuning, which are resourceintensive.To overcome these limitations without incurring significant costs, we propose Inference-Time Cross-Lingual Intervention (INCLINE), a novel framework that enhances LLM performance on low-performing (source) languages by aligning their internal representations with those of high-performing (target) languages during inference.INCLINE initially learns alignment matrices using parallel sentences from source and target languages through a Least-Squares optimization, and then applies these matrices during inference to transform the low-performing language representations toward the high-performing language space.Extensive experiments on nine benchmarks with five LLMs demonstrate that IN-CLINE significantly improves performance across diverse tasks and languages, compared to recent strong baselines.Our analysis demonstrates that INCLINE is highly cost-effective and applicable to a wide range of applications.In addition, we release the code to foster research along this line.

Source: Annual Meeting of the Association for Computational Linguistics (ACL)

DOI: 10.18653/v1/2025.acl-long.270

Authors: Weixuan Wang, Minghao Wu, Barry Haddow, Alexandra Birch

Institutions: University of Edinburgh, Monash University

Language: English

Topic: Topic Modeling

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Reduced inequalities

Open Access status: hybrid