Reproducibility and Replicability in a Fast-paced Methodological World
Year: 2018
Type: preprint
Abstract: Methodological developments and software implementations progress in increasingly faster time frames. The introduction and widespread acceptance of preprint archived reports and open-source software make state-of-the-art statistical methods readily accessible to researchers. At the same time, researchers increasingly emphasize that their results should be reproducible (obtaining the same numeric results at a later stage), which is a basic requirement for assessing the replicability (obtaining the same conclusions at a later stage) of results. Although this age of fast-paced methodology greatly facilitates reproducibility and replicability, it also undermines them in ways not often realized by researchers. The goal of this paper is to draw researchers’ attention to these threats and to propose guidelines to help minimize their impact. Reproducibility may be influenced by software development and change over time: a problem that is greatly compounded by the rising dependency between software packages. Replicability is impacted by rapidly changing standards, researcher degrees of freedom, and possible bugs or errors in a code made by software developers or empirical researchers implementing an analysis. This paper concludes with a list of recommendations for methodologists to improve the reproducibility and replicability of their results.
Author: Sacha Epskamp
Institution: University of Amsterdam
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W4289543569