Work

The logical structure of experiments lays the foundation for a theory of reproducibility

Year: 2023

Type: article

Abstract: The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current 'crisis' reflects inadequate attention to a theoretical understanding of results reproducibility.

Source: Royal Society Open Science

DOI: 10.1098/rsos.221042

Authors: Erkan O. Buzbas, Berna Devezer, Bert Baumgaertner

Institution: University of Idaho

Language: English

Topic: Scientific Computing and Data Management

Subfield: Information Systems and Management

Field: Decision Sciences

Domain: Social Sciences

Open Access status: gold

Estimated APC paid: $1,471

Funders: National Institutes of Health, National Institute of General Medical Sciences