DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation
Year: 2026
Type: conference-paper
Abstract: This paper describes the submission of Team DevLake for the LoResMT 2026 Shared Task on Russian-Bashkir machine translation.We conducted a comprehensive comparative study of three distinct neural architectures: NLLB-200 (1.3B), M2M-100 (418M), and MarianMT (77M).Our primary goal was to evaluate whether parameter-efficient fine-tuning (PEFT) of massive models outperforms full training of compact architectures in a low-resource setting.To achieve this, we employed QLoRA for large models and vocabulary expansion techniques for the smaller MarianMT.We also implemented a rigorous data filtering pipeline using a domain-specific BERT classifier.Our results demonstrate that model scale and "native" pre-training coverage are decisive factors: our best system, a fine-tuned NLLB-200-1.3B model, achieved a CHRF++ score of 52.67, significantly outperforming the compact baseline (43.15) despite the latter's extensive training on a larger dataset.We release our code and trained models to facilitate further research.
Author: Vyacheslav Tyurin
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W7140091404
FWCI:
Cites:
Cited by:
Related to: