Exploring generalizability from the bottom up: Applying DTS to demonstrate an item-level approach to personality assessment
Year: 2025
Type: article
Abstract: Evidence for measurement invariance across groups is regarded as a prerequisite for administering a personality measure in diverse samples and an important criterion for generalizability. However, popular statistical techniques for establishing measurement invariance focus predominantly on broad, scale-level constructs, and there is growing recognition among personality psychologists of the value of item-level assessment. This poses a challenge: while the higher degree of precision offered by more narrowly defined constructs can be beneficial for advancing our scientific understanding of topics studied in personality psychology, it can come at the expense of conducting culturally diverse and inclusive studies. This paper explores the role of measurement invariance evidence in studies that focus on narrower personality constructs––particularly at the item-level––using a sample from the Synthetic Aperture Personality Assessment (SAPA) project consisting of participants ( N = 1025,954) residing in 13 culturally and geographically diverse nations. We address the overall lack of available statistical techniques for addressing invariance-related questions at the item-level and demonstrate an approach that compares the item-level empirical cumulative distribution functions (ECDFs) of two groups using 2623 items from the International Personality item Pool (IPIP). Findings point to nations and items with the most significant differences in ECDFs, which may reflect true cultural differences or potential psychometric challenges. ECDF-based tests capture distribution-wide differences that similar, more standard tests cannot detect, making them potentially useful for addressing a variety of research questions.
Authors: Kendall Mather, Sara J. Weston, David M. Condon
Institution: University of Oregon
Language: English
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $2,390
OpenAlex ID: W4417080328