Work

Defining Replicability of Prediction Rules

Year: 2023

Type: article

Abstract: In this article, I propose an approach for defining replicability for prediction rules. Motivated by a recent report by the U.S.A. National Academy of Sciences, I start from the perspective that replicability is obtaining consistent results across studies suitable to address the same prediction question, each of which has obtained its own data. I then discuss concept and issues in defining key elements of this statement. I focus specifically on the meaning of “consistent results” in typical utilization contexts, and propose a multi-agent framework for defining replicability, in which agents are neither allied nor adversaries. I recover some of the prevalent practical approaches as special cases. I hope to provide guidance for a more systematic assessment of replicability in machine learning.

Source: Statistical Science

DOI: 10.1214/23-sts891

Author: Giovanni Parmigiani

Institutions: Harvard University, Dana-Farber Cancer Institute, Dana-Farber/Harvard Cancer Center

Language: English

Topic: Scientific Computing and Data Management

Subfield: Information Systems and Management

Field: Decision Sciences

Domain: Social Sciences

Open Access status: bronze

Funder: National Science Foundation

Award: Advancing Statistical Methods for Multi-Study Predictions