Year: 2025
Type: review
Study designs: Systematic Review, Meta-Analysis
Abstract: Demand characteristics are a fundamental methodological concern in experimental psychology. Yet, little is known about the direction, magnitude, and consistency of their effects. We conducted a three-level meta-analysis of 253 effect sizes from 53 studies that manipulated explicit demand characteristics (EDCs). On average, EDCs led to small overall increases in hypothesis-consistent responding (g = 0.21, 95% CI [0.11, 0.30]). However, the effects were heterogeneous (between-study τ = 0.29; within-study σ = 0.18), with the prediction interval ranging from g = 0.89 (a large increase in hypothesis-consistent responding) to g = -0.48 (a moderate decrease in hypothesis-consistent responding). Consistent with previous theorizing, the observed and estimated distribution of effects suggest that demand characteristics can create false positives, false negatives, upward bias, and downward bias. These unpredictable inferential consequences suggest that further research is needed to test mechanisms theorized to underlie the effects of demand characteristics.
Source: Collabra Psychology
Authors: Nicholas A. Coles, Morgan Wyatt, Michael C. Frank
Institutions: University of Florida, Stanford University
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: gold
Estimated APC paid: $975
Funder: John Templeton Foundation
OpenAlex ID: W4413152707