Can a Fine-Tuned Language Model Assess Personality Equitably Across Race in Older Adults?
Year: 2026
Type: preprint
Abstract: Artificial intelligence (AI) enables large-scale personality assessment from language. Differential performance across racial groups should be evaluated before applied use. The present study systematically examines bias in language-based personality prediction from life narrative interviews across Black and White American community older adults. Linguistic Inquiry and Word Count (LIWC), BERTopic, and fine-tuned RoBERTa language model predictions of FFM traits were evaluated in a sample of n = 450 Black and n = 450 White older adults matched on education and income. LIWC and BERTopic associations differed little across race, with the few significant differences near chance levels. Fine-tuned RoBERTa predictive accuracy differed significantly for extraversion: the model performed well for White and near chance for Black participants (r = .41 vs. .17). Calibration likewise differed significantly across race only for extraversion. Openness showed significant directional bias, underpredicting scores for Black participants relative to White participants, without a significant calibration difference. These findings indicate that strong aggregate performance across race can mask differential subgroup performance. Keywords: language, natural language processing (NLP), artificial intelligence (AI), personality, five-factor model, big five, racial differences
Institution: Washington University in St. Louis
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: green
OpenAlex ID: W7204394772