Year: 2026
Type: conference-paper
Abstract: Although Large Language Models (LLMs) have achieved remarkable success in Machine Translation (MT), a significant performance gap persists between high-resource and lowresource languages due to imbalanced pretraining data.In this paper, we first investigate the internal mechanisms driving this performance disparity from a layer-wise perspective.We propose a metric termed Activation Disparity (∆R) to quantify the activation divergence between high-and low-resource MT.Based on this metric, we distinguish between Task-Adaptive Layers (TAL, ∆R > 0) that encode task-specific signals and Legacy-Inert Layers (LIL, ∆R < 0) dominated by pre-trained bias.Leveraging this finding, we propose the Layeraware Dual-directional Modulation (LaDM).Integrated with Low-Rank Adaptation (LoRA), LaDM employs a sparse strategy to bidirectionally modulate optimization dynamics.Specifically, it amplifies contributions from TAL to accelerate feature consolidation while inhibiting LIL to dampen misaligned legacy biases.Extensive experiments on Chinese-toseven low-resource language translation using Llama-3.1,Qwen2.5, and Gemma-2 demonstrate that LaDM significantly outperforms standard LoRA fine-tuning, achieving an average improvement of 1.73 spBLEU.Code is available at https://github.com/zzssqqq/ LaDM.
Institution: Kunming University of Science and Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166873783