Evaluating the Use of Large Language Models as Synthetic Social Agents in Social Science Research
Year: 2025
Type: preprint
Abstract: Large Language Models (LLMs) are being increasingly used as synthetic agents in social science, in applications ranging from augmenting survey responses to powering multi-agent simulations. This paper outlines cautions that should be taken when interpreting LLM outputs and proposes a pragmatic reframing for the social sciences in which LLMs are used as high-capacity pattern matchers for quasi-predictive interpolation under explicit scope conditions and not as substitutes for probabilistic inference. Practical guardrails such as independent draws, preregistered human baselines, reliability-aware validation, and subgroup calibration, are introduced so that researchers may engage in useful prototyping and forecasting while avoiding category errors.
Source: arXiv (Cornell University)
Author: Madden, Emma Rose
Institution: University of Oxford
Language: English
Subfield: General Social Sciences
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7084447313
Cites:
Cited by:
Related to: