Through the Looking Glass of Multilingual AI: Contrasting Language- and Name Script-Dependent Ethnic Hierarchies in GPT and DeepSeek
Year: 2026
Type: conference-paper
Abstract: Large language models (LLMs) are increasingly used as evaluative tools across languages, yet bias research remains overwhelmingly Anglocentric, with most studies conducted in English using Latin-script names.It remains unclear whether bias patterns generalize across linguistic contexts.We investigate this question and introduce the stereotype perceptual map, a framework for analyzing how ethnic groups are positioned along evaluative dimensions.We find that prompt language and writing script alter the hierarchical ordering of ethnic groups and reveal model behaviors that monolingual evaluations fail to detect.Using 900,000 model responses over 45,000 name variations spanning 9 ethnicities, we evaluate model behavior across prompt languages (English, Chinese, Thai), writing scripts (Latin, Chinese, Thai), evaluative domains (competence, warmth), and models (GPT, DeepSeek).We find that ethnic bias hierarchies are jointly shaped by local linguistic context and model origin and differ substantially between Westerncentric and Sinocentric models.DeepSeek exhibits highly stable rankings across conditions in math competence judgments, consistently placing Chinese at the top, followed by Russian, and White, Hispanic, and Black names at the bottom.GPT, by contrast, shows strong script-dependent reordering: Latin-script conditions form one stable cluster, while nativescript conditions form another, with substantially lower cross-cluster correlations.We term this script-gated bias: transliterating the same names into a non-Latin script can activate a different evaluative frame and produce rankings that are sometimes inversely correlated with Latin-script results.Warmth evaluations are less stable than competence in both models.Our study demonstrates that multilingual bias cannot be characterized by single-language, single-writing system audits.An organization auditing for bias only in English with romanized names may observe some levels of eth-nic variation, yet the same models, deployed in another language with transliterated names, can produce substantially different or even inversely correlated ethnic hierarchies.For multilingual users, code-switching between languages unknowingly toggles between different bias regimes, with potential consequences for any application where LLMs assess human competence and character.Fairness evaluations for multilingual LLMs should therefore test across deployment languages, writing scripts, evaluation domains, and model origins to capture the full range of potentially harmful bias these systems carry.
Authors: Annabella Sakunkoo, Jonathan Sakunkoo
Institutions: Stanford University, University of Oxford
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: hybrid
Funder: University of Oxford
OpenAlex ID: W7166879185