Developing a clinical utility framework to evaluate prediction models in radiogenomics
Year: 2015
Type: conference-paper
Study design: Observational Study
Abstract: Combining imaging and genetic information to predict disease presence and behavior is being codified into an emerging discipline called “radiogenomics.” Optimal evaluation methodologies for radiogenomics techniques have not been established. We aim to develop a clinical decision framework based on utility analysis to assess prediction models for breast cancer. Our data comes from a retrospective case-control study, collecting Gail model risk factors, genetic variants (single nucleotide polymorphisms-SNPs), and mammographic features in Breast Imaging Reporting and Data System (BI-RADS) lexicon. We first constructed three logistic regression models built on different sets of predictive features: (1) Gail, (2) Gail+SNP, and (3) Gail+SNP+BI-RADS. Then, we generated ROC curves for three models. After we assigned utility values for each category of findings (true negative, false positive, false negative and true positive), we pursued optimal operating points on ROC curves to achieve maximum expected utility (MEU) of breast cancer diagnosis. We used McNemar’s test to compare the predictive performance of the three models. We found that SNPs and BI-RADS features augmented the baseline Gail model in terms of the area under ROC curve (AUC) and MEU. SNPs improved sensitivity of the Gail model (0.276 vs. 0.147) and reduced specificity (0.855 vs. 0.912). When additional mammographic features were added, sensitivity increased to 0.457 and specificity to 0.872. SNPs and mammographic features played a significant role in breast cancer risk estimation (p-value < 0.001). Our decision framework comprising utility analysis and McNemar’s test provides a novel framework to evaluate prediction models in the realm of radiogenomics.
DOI: 10.1117/12.2081954
Authors: Yirong Wu, Jie Liu, Alejandro Munoz del Rio, David C. Page, Oguzhan Alagoz, Peggy Peissig, Adedayo A. Onitilo, Elizabeth S. Burnside
Language: English
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: green
Funders: Institute for Clinical and Translational Research, University of Wisconsin, Madison, National Cancer Institute, National Institute of General Medical Sciences, National Institute of Allergy and Infectious Diseases, National Institute of Environmental Health Sciences, U.S. National Library of Medicine
Awards: The Center for Predictive Computational Phenotyping-1 Overall, Machine Learning for Identifying Adverse Drug Events, Secure Sharing of Clinical History & Genetic Data: Empowering Predictive Pers. Me, Machine Learning for Improved Mammography Screening, Breast Cancer GWAS: Function and Environmental Interactions, UW Comprehensive Cancer Center Support
OpenAlex ID: W1985682123
