Year: 2026
Type: conference-paper
Abstract: Large Language Models (LLMs) are increasingly used as proxies for human subjects in social science surveys, but their reliability and susceptibility to known human-like response biases, such as central tendency, opinion floating and primacy bias are poorly understood.This work investigates the response robustness of LLMs in normative survey contexts-we test 18 LLMs on questions taken from the World Values Survey (WVS), applying a comprehensive set of ten perturbations to both question phrasing and answer option structure, resulting in over 334,800 simulated survey interviews.In doing so, we not only reveal LLMs' vulnerabilities to perturbations but also show that almost all tested models exhibit a consistent recency bias, disproportionately favoring the lastpresented answer option.While larger models are generally more robust, all models remain sensitive to semantic variations like paraphrasing and to combined perturbations.This underscores the critical importance of prompt design and robustness testing when using LLMs to generate synthetic survey data.
Institution: University of Mannheim
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: hybrid
OpenAlex ID: W7166889499