Work

Selective reporting of placebo tests in top economics journals

Year: 2024

Type: article

Abstract: Placebo tests provide incentives to underreport statistically significant tests, a form of reversed p‐hacking. We test for such underreporting in 11 top economics journals between 2009 and 2021 based on a pre‐registered analysis plan. If the null hypothesis is true in all tests, 2.5% of them should be significant at the 5% level with an effect in the same direction as the main test (and 5% in total). The actual fraction of statistically significant placebo tests with an effect in the same direction is 1.29% (95% CI [0.83, 1.63]), and the overall fraction of statistically significant placebo tests is 3.10% (95% CI [2.2, 4.0]).

Source: Economic Inquiry

DOI: 10.1111/ecin.13217

Authors: Anna Dreber, Magnus Johannesson, Yifan Yang

Institutions: Stockholm School of Economics, Universität Innsbruck

Language: English

Topic: Meta-analysis and systematic reviews

Subfield: Statistics, Probability and Uncertainty

Field: Decision Sciences

Domain: Social Sciences

Open Access status: hybrid

Estimated APC paid: $3,400

Funders: Jan Wallanders och Tom Hedelius Stiftelse samt Tore Browaldhs Stiftelse, Knut och Alice Wallenbergs Stiftelse, Riksbankens Jubileumsfond, Marcus och Amalia Wallenbergs minnesfond