Work

Calibrating the experimental measurement of psychological attributes

Year: 2020

Type: review

Abstract: Behavioural researchers often seek to experimentally manipulate, measure and analyse latent psychological attributes, such as memory, confidence or attention. The best measurement strategy is often difficult to intuit. Classical psychometric theory, mostly focused on individual differences in stable attributes, offers little guidance. Hence, measurement methods in experimental research are often based on tradition and differ between communities. Here we propose a criterion, which we term ‘retrodictive validity’, that provides a relative numerical estimate of the accuracy of any given measurement approach. It is determined by performing calibration experiments to manipulate a latent attribute and assessing the correlation between intended and measured attribute values. Our approach facilitates optimising measurement strategies and quantifying uncertainty in the measurement. Thus, it allows power analyses to define minimally required sample sizes. Taken together, our approach provides a metrological perspective on measurement practice in experimental research that complements classical psychometrics. How can we determine the best way of measuring a psychological construct? Bach et al. propose a ‘retrodictive validity’ approach, in which candidate methods are ranked based on their sensitivity to detect known effects, with the most sensitive then being favoured for use in novel scenarios.

Source: Nature Human Behaviour

DOI: 10.1038/s41562-020-00976-8

Authors: Dominik R. Bach, Filip Melinščak, Stephen M. Fleming, Manuel C. Voelkle