Work

Better P-curves: Making P-curve analysis more robust to errors, fraud, and ambitious P-hacking, a Reply to Ulrich and Miller (2015).

Year: 2015

Type: article

Abstract: When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the possibility that researchers report only the smallest significant p value (as conjectured by Ulrich & Miller, 2015), concluding that it is a very uncommon problem. We then consider more common problems, including (a) p-curvers selecting the wrong p values, (b) fake data, (c) honest errors, and (d) ambitiously p-hacked (beyond p < .05) results. We evaluate the impact of these common problems on the validity of p-curve analysis, and provide practical solutions that substantially increase its robustness.

Source: Journal of Experimental Psychology General

DOI: 10.1037/xge0000104

Authors: Uri Simonsohn, Joseph P. Simmons, Leif D. Nelson

Institutions: Esade, Universitat Ramon Llull, University of Pennsylvania, University of California, Berkeley

Language: English

Topic: Meta-analysis and systematic reviews

Subfield: Statistics, Probability and Uncertainty

Field: Decision Sciences

Domain: Social Sciences

Open Access status: green

OpenAlex ID: W2339493379

FWCI: 12.09

Cites: 20

Cited by: 362

Related to: 10