Year: 2026
Type: article
Abstract: Word embeddings have become powerful tools for detecting social biases encoded in language, yet research on measuring race bias through embeddings remains underdeveloped compared to studies on gender bias. This gap largely stems from the complexity of constructing race dimensions, which involve socially contested meanings and less clear semantic oppositions. Existing studies on race bias often rely on intuition and context-specific approaches when choosing anchor terms. In this paper, we address this methodological gap by providing statistical metrics to evaluate the quality and adaptability of race categories in embeddings. We apply these metrics to race categories across three embeddings-Google News (U.S.-centric), South African News (South African context), and Wikipedia (neutral, general-purpose). We find that names are effective for constructing race dimensions, with sub-Saharan African/European name categories producing more stable and generalisable dimensions than other categories, while American names were less generalisable. Validation shows that SSA/European name embeddings correlate most strongly with human ratings and demonstrate that our metrics capture human-perceived semantic structure of race. This research provides a framework for constructing robust race dimensions for measuring race bias in word embeddings. Supplementary Information: The online version contains supplementary material available at 10.1007/s42001-025-00449-w.
Institution: University of Johannesburg
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: hybrid
Estimated APC paid: $3,290
Awards: Developmental Change and Individual Differences in Sensory Discrimination and Working Memory: The Impact and Interplay of Chronological Age and Human Intelligence, PONs - People-Opinion Networks: A study of polarization in word embeddings and social networks in Switzerland and Southern Africa., G8208409376
OpenAlex ID: W7124450993