NSF Science of Science Proposal: Randomized Trial of Registered Reports
Year: 2020
Type: preprint
Abstract: In theory, Registered Reports eliminate publication bias against negative results because publication decisions are made without knowledge of the results; increase clarity between planned (hypothesis testing; confirmatory) and unplanned (hypothesis generating; exploratory) analyses, thereby increasing the diagnosticity of statistical inferences; and, leverage peer review expertise more effectively to foster better research methodology and more informative outcomes--regardless of whether they are positive or negative results, or are consistent or inconsistent with hypothesized outcomes. With the RRs published to date, there is observational evidence for some of these expectations. But, given the potentially transformative impact of RRs on the research process, we believe that it is essential to conduct a randomized trial to evaluate the model’s qualities. By collaborating with journals, we can conduct a randomized trial of RRs in an ecologically valid context. Authors who receive revise-and-resubmit decision letters asking for an additional study will be invited to participate in the trial. Those that opt-in will be randomized to either a status quo (i.e., standard of care) or RR condition for their resubmission with the additional study. We will examine the impact of RRs on publication (e.g., acceptance rate), research outcomes (e.g., rate of positive results), and qualities of methodology (e.g., sample size, transparency, rigor).
Authors: Brian A. Nosek, David Thomas Mellor, Timothy M. Errington, Courtney K. Soderberg, Tina Krall, Fielding Grasty, Ronald E Brooks, Richard Davis, Stavroula Kousta, Sowmya Swaminathan, Veronique Kiermer, Jacqueline Thompson, wendy berry mendes, Derek Heim, Douglas B Samuel, Nidhi Bansal, Jeremy Wolfe, Richard E. Lucas, Christopher D Chambers, David Rapp, Mitja D Back, Martha Arterberry, Ioannis Nikolaou, Candice Coker Morey, Kevin Arceneaux, Jonas W. B. Lang
Institutions: Center for Open Science, University of Virginia, Ghent University
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: gold
OpenAlex ID: W4233339953