Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking
Year: 2026
Type: article
Abstract: Large language models (LLMs) shape human–AI interaction, yet capability and safety benchmarks provide limited insight into their expressed value priorities. Across three studies, we developed an output-based approach to evaluating correspondence between LLM priorities and human judgements of optimal AI functioning. Study 1 identified six themes through inductive qualitative analysis: Performance, Adaptive Capacity, Social Good, Ethics and Responsibility, Relational Integration, and Agency. Study 2 found stable rankings within models and a shared priority structure across models. Study 3 compared 75 LLMs with 376 human respondents using a profile-fidelity metric combining profile shape and calibration of between-priority differences. Many models reproduced the human profile while exaggerating differences between priorities. Fidelity varied across models, with no clear pattern linking fidelity to more recent releases. Both humans and LLMs assigned Agency low priority. This framework offers a standardized method for auditing expressed value profiles across models alongside capability, safety, and output evaluations.
Authors: Gabriel R. Lau, Wei Yan Low, Seow Min Koh, Fiona Fui-Hoon Nah, Andree Hartanto
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Funder: Singapore Management University
OpenAlex ID: W4415249921