Work

Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM

Year: 2023

Type: preprint

Abstract: The NLP community recently saw the release of a new large open-access multilingual language model, BLOOM (BigScience et al., 2022) covering 46 languages. We focus on BLOOM's multilingual ability by evaluating its machine translation performance across several datasets (WMT, Flores-101 and DiaBLa) and language pairs (high- and low-resourced). Our results show that 0-shot performance suffers from overgeneration and generating in the wrong language, but this is greatly improved in the few-shot setting, with very good results for a number of language pairs. We study several aspects including prompt design, model sizes, cross-lingual transfer and the use of discursive context.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2303.01911

Authors: Bawden, Rachel, Yvon, François

Institutions: Institut national de recherche en sciences et technologies du numérique, ALMANACH: Modélisation et analyse linguistique automatique et humanités computationnelles, Université Paris-Saclay, Laboratoire Interdisciplinaire des Sciences du Numérique

Language: English

Topic: Natural Language Processing Techniques

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Agence Nationale de la Recherche, Centre National de la Recherche Scientifique, Grand Équipement National De Calcul Intensif, Sorbonne Université

Awards: G3010527862, PaRis Artificial Intelligence Research InstitutE

OpenAlex ID: W4323323154

Cites:

Cited by: 17

Related to: 10