Evaluating Cross-National Value Alignment in Large Language Models: Challenges and Limitations of Survey-Based Approaches
Year: 2024
Type: article
Abstract: Large language models (LLMs) often exhibit cultural biases, raising questions about their alignment with human values across different countries. This study examines the cross-national value alignment of LLMs, focusing on responses from the United States, China, and Egypt using the World Values Survey (WVS) dataset. We evaluate the alignment of LLM responses to national value distributions and analyze linguistic features such as point of view, justification, and hedging tendencies. Our findings reveal that LLMs display a default alignment towards US-centric values, with varying degrees of success in reflecting the values of other nations. Furthermore, we highlight the methodological challenges and potential ecological fallacies in applying survey-based cultural research to LLM evaluation. This research underscores the need for nuanced approaches in assessing and enhancing the cultural alignment of LLMs.
Source: Stanford Digital Repository
DOI: 10.25740/sb536dz5755
Author: Qu, Carolyn
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
SDG: Reduced inequalities
Open Access status: green
OpenAlex ID: W6964971736
FWCI:
Cites:
Cited by:
Related to: