Work

Experimental Design and Data Analysis in Receiver Operating Characteristic Studies: Lessons Learned from Reports inRadiologyfrom 1997 to 2006

Year: 2009

Type: article

Abstract: PURPOSE: To provide a broad perspective concerning the recent use of receiver operating characteristic (ROC) analysis in medical imaging by reviewing ROC studies published in Radiology between 1997 and 2006 for experimental design, imaging modality, medical condition, and ROC paradigm. MATERIALS AND METHODS: Two hundred ninety-five studies were obtained by conducting a literature search with PubMed with two criteria: publication in Radiology between 1997 and 2006 and occurrence of the phrase "receiver operating characteristic." Studies returned by the query that were not diagnostic imaging procedure performance evaluations were excluded. Characteristics of the remaining studies were tabulated. RESULTS: Two hundred thirty-three (79.0%) of the 295 studies reported findings based on observers' diagnostic judgments or objective measurements. Forty-three (14.6%) did not include human observers, with most of these reporting an evaluation of a computer-aided diagnosis system or functional data obtained with computed tomography (CT) or magnetic resonance (MR) imaging. The remaining 19 (6.4%) studies were classified as reviews or meta-analyses and were excluded from our subsequent analysis. Among the various imaging modalities, MR imaging (46.0%) and CT (25.7%) were investigated most frequently. Approximately 60% (144 of 233) of ROC studies with human observers published in Radiology included three or fewer observers. CONCLUSION: ROC analysis is widely used in radiologic research, confirming its fundamental role in assessing diagnostic performance. However, the ROC studies reported in Radiology were not always adequate to support clear and clinically relevant conclusions.

Source: Radiology

DOI: 10.1148/radiol.2533081632

Authors: Junji Shiraishi, Lorenzo L. Pesce, Charles E. Metz, Kunio Doi

Institution: University of Chicago

Language: English

Topic: Radiology practices and education

Subfield: Radiology, Nuclear Medicine and Imaging

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: green

Funders: National Cancer Institute, National Institute of Biomedical Imaging and Bioengineering