Exploratory and Confirmatory Empirical Research on Algorithms: Implications for Methodological Practice and Education—A Comment on “On ‘Confirmatory’ Methodological Research in Statistics and Related Fields”
Year: 2026
Type: article
Abstract: draw timely attention to a persistent problem in empirical research on statistical and algorithmic methods: the frequent failure to distinguish between exploratory and confirmatory investigations.This conceptual ambiguity has important methodological consequences.It encourages the presentation of case-based empirical demonstrations as if they provided generalizable evidence and obscures the inferential standards that should govern methods research.While Lange et al. [1] focus primarily on editorial and incentive-related remedies, we argue that this problem also reflects a more fundamental educational gap that deserves explicit consideration-particularly in light of Statistics in Medicine's long-standing emphasis on design, inference, and interpretation.In clinical and epidemiological research, the exploratoryconfirmatory distinction is well established [2].Exploratory analyses generate hypotheses and insights, whereas confirmatory studies are designed to test pre-specified claims under clearly articulated assumptions and targets of inference.In contrast, empirical evaluations of statistical and algorithmic methods-especially in prediction modeling and machine learning-often lack comparable clarity.New methods are frequently assessed using convenience datasets or ad hoc benchmark collections, without explicit justification of dataset selection, comparison structure, or intended scope of inference.
Source: Statistics in Medicine
DOI: 10.1002/sim.70388
Author: Ulrich Mansmann
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7130506362