Evaluating biomarkers and prediction models: predictive utility analysis with E2P Simulator
Year: 2026
Type: preprint
Abstract: Precision psychiatry aims to identify biomarkers and develop prediction models to improve diagnosis, prognosis, and treatment selection. Despite extensive research, translating findings into clinically useful tools remains challenging. Here we argue that one important contributor to this translational gap is a pervasive misalignment between routine statistical analytic practices and the requirements for real-world decision making. To help address this, we introduce predictive utility analysis: a statistical workflow for estimating the practical significance of biomarkers and prediction models by accounting for outcome base rates and measurement reliability. Starting from effect size or model performance estimates, the workflow simulates expected real-world performance in a target population and summarizes it across complementary metrics including positive and negative predictive value, Precision-Recall AUC, and Net Benefit. Analogous to how power analysis is used for determining the conditions needed to achieve statistical significance, predictive utility analysis can help determine the conditions needed to achieve practical significance. We implement and illustrate this workflow using E2P Simulator, a recently developed open-source web tool that makes predictive utility analysis accessible through interactive simulations and publication-ready visualizations. We demonstrate its application using specific examples in three key areas: diagnostic (depression, Alzheimer’s disease), treatment response (antidepressants), and risk (suicide attempts and psychosis onset) prediction, highlighting conditions under which we may expect clinically meaningful advances. While we focus on translational challenges in psychiatry, the workflow and tools presented here address general statistical challenges and are broadly applicable across biomedical and behavioral sciences.
Source: PsyArXiv (OSF Preprints)
Language: English
Subfield: Statistics and Probability
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7214114751