Work

Delving into Multilingual Ethical Bias: The MSQAD with Statistical Hypothesis Tests for Large Language Models

Year: 2025

Type: conference-paper

Abstract: Despite the recent strides in large language models, studies have underscored the existence of social biases within these systems.In this paper, we delve into the validation and comparison of the ethical biases of LLMs concerning globally discussed and potentially sensitive topics, hypothesizing that these biases may arise from language-specific distinctions.Introducing the Multilingual Sensitive Questions & Answers Dataset (MSQAD), we collected news articles from Human Rights Watch covering 17 topics, and generated socially sensitive questions along with corresponding responses in multiple languages.We scrutinize the biases of these responses across languages and topics, employing two statistical hypothesis tests.The results suggest that the null hypotheses are rejected in most cases, indicating biases arising from cross-language differences.It indicates that ethical biases in responses are widespread across various languages, and notably, these biases are prevalent even among different LLMs.By making the proposed MSQAD openly available, we aim to facilitate future research endeavors focused on examining cross-language biases in LLMs and their variant models 1 .

Source: Annual Meeting of the Association for Computational Linguistics (ACL)

DOI: 10.18653/v1/2025.acl-long.15

Authors: Seunguk Yu, Juhwan Choi, YoungBin Kim

Institution: Chung-Ang University

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

SDG: Reduced inequalities

Open Access status: hybrid

Funders: Ministry of Science and ICT, South Korea, Institute for Information and Communications Technology Promotion