Incentives, Assessment, and the Reliabiliity of Statistical Significance Assessments of Evidence
Year: 2025
Type: conference-paper
Abstract: This analysis evaluates how researcher hypothesis selection incentives on the inferential value of empirical analyses. It illustrates the strength with which a “statistically significant” outcome objective incentivizes researcher aversion to testing possibly true null hypotheses in a comparatively unconstrained general setting. Left unfettered, it suggests that settings wherein researchers almost always opt to test false null hypotheses are common. That is, studies routinely produce reliable “falsifications” of a priori false hypotheses, a practice that transparently lacks inferential relevance. An outcome that aligns with the longstanding perspective that in many fields and settings the null hypotheses that researchers test are commonly false (e.g., Cohen, 1992). Collectively, the analysis illustrates the importance of comprehensive uncontrived understandings of researcher incentives and research assessment practices when evaluating the reliability and relevance of findings obtained from Null Hypothesis Significance Test based examinations of evidence.
Author: Cready, William
Institution: The University of Texas at Dallas
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7084391658
FWCI:
Cites:
Cited by:
Related to: