Work

ArGAN: Arabic Gender, Ability, and Nationality Dataset for Evaluating Biases in Large Language Models

Year: 2025

Type: conference-paper

Abstract: Large language models (LLMs) are pretrained on substantial, unfiltered corpora, assembled from a variety of sources.This risks inheriting the deep-rooted biases that exist within them, both implicit and explicit.This is even more apparent in low-resource languages, where corpora may be prioritized by quantity over quality, potentially leading to more unchecked biases, particularly in low-resource languages, where all available data is leveraged solely to expand volume due to inherent scarcity.More specifically, we address the biases present in the Arabic language in both general-purpose and Arabic-specialized architectures in three dimensions of demographics: gender, ability, and nationality.We introduce ArGAN, a dataset for evaluating the fairness of these models across three demographic axes: gender, ability and nationality.Where we experiment with bias-revealing, template-based prompts and measure performance and bias using existing and evaluation metrics, and propose adaptations to others.

Source: Workshop on Gender Bias in Natural Language Processing (GeBNLP)

DOI: 10.18653/v1/2025.gebnlp-1.23

Authors: Ranwa Aly, Yara Allam, Rana Gaber, Christine Basta

Institution: Alexandria University

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

SDGs: Reduced inequalities, Gender equality

Open Access status: hybrid

OpenAlex ID: W4412944355