Risk of bias in robustness reports
Year: 2025
Type: preprint
Abstract: Bartoš et al. propose a new article format — Synchronous Robustness Reports (SRRs) — in which journals invite researchers to conduct alternative analyses of recently accepted studies. The goal is to assess the robustness of the original results by exploring how they vary across different analysis choices. However, the proposal does not distinguish between SRRs planned before versus after analysts have seen the data. This distinction is crucial because data-dependent analyses introduce a risk of bias from selective reporting. In this commentary, I argue that without safeguards, SRRs could inadvertently create more confusion than clarity about robustness.
Author: Tom Elis Hardwicke
Language: English
Subfield: Statistics, Probability and Uncertainty
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W4416206612