Year: 2024
Type: article
Abstract: Large language models (LLMs) have shown significant potential in multilingual machine translation (MMT).This paper systematically explores the advantages and challenges of LLMs in this domain by addressing two key questions: 1) How effectively do LLMs translate a wide range of languages?2) What factors influence their translation performance?We assess popular LLMs, including XGLM, OPT, BLOOMZ, and ChatGPT, across 102 languages.Our empirical findings reveal that even the top-performing model, ChatGPT, falls short of the supervised baseline NLLB in 83.33% of translation tasks.Further analysis uncovers new patterns in LLM behavior during MMT.First, the semantics of prompts can be largely disregarded when using in-context examples, as LLMs maintain strong performance even with nonsensical prompts.Second, cross-lingual examples often provide superior task guidance for low-resource translations compared to same-language pairs.Lastly, we find that BLOOMZ's performance on the Flores-101 dataset may be overestimated, highlighting potential risks associated with using public datasets for evaluation. I.INTRODUCTIONAs large language models (LLMs) grow in scale and complexity, they have developed a remarkable ability to perform a wide range of tasks through human-written instructions and in-context learning (ICL) (Brown et al., 2020).ICL enables these models to learn tasks by using a few provided examples
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4403798322