Work

GENDEROUS: Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset

Year: 2025

Type: conference-paper

Abstract: Contributing to research on gender beyond the binary, this work introduces GENDEROUS, a dataset of gender-ambiguous sentences containing gender-marked occupations and adjectives, and sentences with the ambiguous or nonbinary pronoun their.We cross-linguistically evaluate how machine translation (MT) systems and large language models (LLMs) translate these sentences from English into four grammatical gender languages: Greek, German, Spanish and Dutch.We show the systems' continued default to male-gendered translations, with exceptions (particularly for Dutch).Prompting for alternatives, however, shows potential in attaining more diverse and neutral translations across all languages.An LLMas-a-judge approach was implemented, where benchmarking against gold standards emphasises the continued need for human annotations.

Source: Workshop on Gender Bias in Natural Language Processing (GeBNLP)

DOI: 10.18653/v1/2025.gebnlp-1.27

Authors: Janiça Hackenbuchner, Eleni Gkovedarou, Joke Daems

Institution: Ghent University

Language: English

Topic: Gender Studies in Language

Subfield: Gender Studies

Field: Social Sciences

Domain: Social Sciences

SDGs: Reduced inequalities, Gender equality

Open Access status: hybrid

Funders: Vlaams Supercomputer Centrum, Fonds Wetenschappelijk Onderzoek, Universiteit Gent, Vlaamse regering