Persona Diversity and Error Structure in LLM Crowds
Year: 2026
Type: preprint
Abstract: Can psychologically different roles make a crowd of large language model responses more accurate? We examined demographic personas, cognitive personas, and instructions inspired by the wisdom of the inner crowd. An exploratory reanalysis used six conditions on 15 MARBLES estimation items presented as text or with photographs. We calculated error after aggregating responses and distinguished two comparisons: role-based crowds against neutral responses of the same number, and heterogeneous against homogeneous crowds reconstructed from the same response pools. For text, the 12-response stacked condition had lower mean relative absolute error than a matched hierarchical neutral control, 0.766825 versus 0.958008. The difference was −0.191183, with a 95% item-bootstrap interval of [−0.518117, 0.015682], and one item accounted for much of the mean advantage. The corresponding image difference was −0.004316 [−0.026710, 0.019478]. Within the same response pools, composition effects were generally small and depended on the comparison and error measure. Image responses frequently underestimated the true quantities, while persona error patterns were highly correlated in both presentations. A supplementary Fermi analysis documents how rounding changed an earlier condition ranking. These findings distinguish changes associated with role-based prompting from the additional value of combining different roles, and identify the error structure that each comparison can assess.
Source: PsyArXiv (OSF Preprints)
Author: Itsuki Fujisaki
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225385537