What Are Chatbots’ Stereotypes About? A Data-Driven Analysis of Large Language Models’ Content Associations with Social Categories
Year: 2025
Type: conference-paper
Abstract: This study introduces a data-driven taxonomy of stereotype content in contemporary large language models (LLMs). We prompt ChatGPT 4.5, ChatGPT 3.5, Llama 3, and Mixtral 8x7B, four recent and powerful LLMs, for the characteristics associated with 87 social categories (e.g., gender, race, occupations). We show that these prompts are reliable and valid, predicting unrelated tasks such as storytelling about the targets. Using text embeddings and cluster analyses, we identify 14 dimensions (Ability, Appearance, Assertiveness, Beliefs, Deviance, Emotion, Family, Geography, Health, Morality, Occupations, Social categories, Sociability, and Status) in LLMs’ stereotypes. This high-dimensional taxonomy reveals both similarities (e.g., same set of dimensions) and differences (e.g., variation in prevalence of content) with human stereotypes. In addition, we find that highly overlapping taxonomies emerge from analyses of personal and cultural stereotypes, as well as across various LLMs. However, again, some prompts and LLMs differ in how frequently specific dimensions appear in association with social categories. Our findings suggest that LLMs’ stereotypes are high-dimensional and auditing and debiasing would benefit from considering this complexity to minimize unidentified harm from reliance in low-dimensional views of bias in LLMs.
Authors: Gandalf Nicolas, Aylin Caliskan
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: bronze
Funder: National Science Foundation
OpenAlex ID: W4415230764