Year: 2024
Type: conference-paper
Abstract: This study investigates the factors influencing the performance of multilingual large language models (MLLMs) across diverse languages.We study 6 MLLMs, including masked language models, autoregressive models, and instruction-tuned LLMs, on the SIB-200 dataset, a topic classification dataset encompassing 204 languages.Our analysis considers three scenarios: ALL languages, SEEN languages (present in the model's pretraining data), and UNSEEN languages (not present or documented in the model's pretraining data in any meaningful way).We examine the impact of factors such as pretraining data size, general resource availability, language family, and script type on model performance.Decision tree analysis reveals that pretraining data size is the most influential factor for SEEN languages.However, interestingly, script type and language family are crucial for UNSEEN languages, highlighting the importance of crosslingual transfer learning.Notably, model size and architecture do not significantly alter the most important features identified.Our findings provide valuable insights into the strengths and limitations of current MLLMs and hope to guide the development of more effective and equitable multilingual NLP systems. 1
Authors: Sina Bagheri Nezhad, Ameeta Agrawal
Institution: Portland State University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: National Science Foundation
OpenAlex ID: W4401042060