Work

Public Covid-19 X-ray datasets and their impact on model bias – A systematic review of a significant problem

Year: 2021

Type: review

Abstract: Computer-aided-diagnosis and stratification of COVID-19 based on chest X-ray suffers from weak bias assessment and limited quality-control. Undetected bias induced by inappropriate use of datasets, and improper consideration of confounders prevents the translation of prediction models into clinical practice. By adopting established tools for model evaluation to the task of evaluating datasets, this study provides a systematic appraisal of publicly available COVID-19 chest X-ray datasets, determining their potential use and evaluating potential sources of bias. Only 9 out of more than a hundred identified datasets met at least the criteria for proper assessment of risk of bias and could be analysed in detail. Remarkably most of the datasets utilised in 201 papers published in peer-reviewed journals, are not among these 9 datasets, thus leading to models with high risk of bias. This raises concerns about the suitability of such models for clinical use. This systematic review highlights the limited description of datasets employed for modelling and aids researchers to select the most suitable datasets for their task.

Source: Medical Image Analysis

DOI: 10.1016/j.media.2021.102225

Authors: Beatriz Garcia Santa Cruz, Matías Nicolás Bossa, Jan Sölter, Andreas Dominik Husch

Institutions: Centre Hospitalier de Luxembourg, University of Luxembourg, Luxembourg Centre for Systems Biomedicine, Vrije Universiteit Brussel

Language: English

Topic: COVID-19 diagnosis using AI

Subfield: Radiology, Nuclear Medicine and Imaging

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: hybrid

Estimated APC paid: $3,970