Registered Replication Report: Johns, Schmader, & Martens (2005)
Year: 2025
Type: preprint
Abstract: Stereotype threat refers to the fear of being judged based on negative stereotypes about the performance of a certain group one identifies with. Numerous published studies have found that stereotype threat might lower mathematics test performance among women. However, many studies used suboptimal designs and analyses; the literature might be subject to publication bias; and the cross-cultural generalizability of the effect remains unknown. This registered replication report describes the result of eight direct replications (total N = 1502) of a representative stereotype-threat study by Johns et al. (2005), who found that threatened women (but not men) underperform when they are confronted with a mathematics test that is presented to measure gender differences, and that this effect can be alleviated by altering test instructions. A multilevel analysis showed no significant interaction effect between stereotype-threat condition and gender, and only an observed gender gap on the mathematics test, with women performing worse than men (d = -0.37; 95%CI[-0.49; -0.25]). The stereotype-threat effect among women was virtually null and not significant (N = 822, d = -0.006; 95% CI[-0.15; 0.14]), and considerably smaller than the original study (N = 45, d = -0.82; 95 %CI[-1.45; -0.18]). Secondary analyses yielded a main effect of gender composition and a random effect of lab, but neither variable affected the interaction between stereotype threat and gender. The current results fail to replicate the stereotype-threat effect by Johns et al. (2005), hence casting doubt on the generalizability of the effect of stereotype threat on women’s mathematics performance.
Authors: Andrea Helena Stoevenbelt, Paulette Carien Flore, Jelte M. Wicherts, Matus Adamkovic, Gabriel Baník, Robert Calin-Jageman, Rachel Bergers, Luis Angel Gomez, Joachim Hüffmeier, Megan N Imundo, Scott Martin, Steven C. Pan, Lorraine Phillips, Jakob Pietschnig, Tilli Ripp, Jan Philipp Röer, Ivan Ropovik, Ann-Kathrin Torka, Bruno Verschuere, Joyce Elena Schleu, Martin Voracek, Luca-Alexander Serafin Weiss
Institution: Purdue University West Lafayette
Language: English
Subfield: Sociology and Political Science
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W4417208817