Impact of Leakage on Data Harmonization in Machine Learning Pipelines in Class Imbalance Across Sites
Year: 2024
Type: preprint
Abstract: Due to the cost and complexity of data collection in biomedical domains, it is a common practice to combine data from multiple sites to obtain large datasets required for machine learning. However, undesired site-specific variability presents challenges. Data harmonization aims to address this issue by removing site-specific variance while preserving biologically relevant information. We show that the widely used ComBat-based harmonization improvements are driven by data leakage due to illicit use of target information when class labels are imbalanced across sites, a common scenario in biomedical domains. We propose a novel approach, PrettYharmonize, which leverages subtle differences in data harmonized using different pretended target values. Using controlled benchmark datasets and real-world magnetic resonance imaging and clinical ICU data, we demonstrate that our leakage-free PrettYharmonize method achieves performance comparable to leakage-prone methods. As such, it is a viable method to integrate ComBat-based methods into machine learning applications.
Source: arXiv (Cornell University)
Authors: Nieto, Nicolás, Eickhoff, Simon B., Jung, Christian, Reuter, Martin, Diers, Kersten, Kelm, Malte, Lichtenberg, Artur, Raimondo, Federico, Patil, Kaustubh R.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funders: U.S. Department of Defense, Alzheimer's Association, Bristol-Myers Squibb, Eli Lilly and Company, Pfizer, Biogen, University of Southern California, Meso Scale Diagnostics, Alzheimer's Disease Neuroimaging Initiative, BioClinica, Novartis Pharmaceuticals Corporation, Northern California Institute for Research and Education, Eisai, Servier, H. Lundbeck A/S, IXICO, National Institutes of Health, Genentech, Canadian Institutes of Health Research
Awards: G1298807586, G1554142629, G1593117279, G1650819308, G1880935189, Alzheimer's Disease Neuroimaging Initiative, G2100670906, G2152643862, G2189010064, G2340207221, G2423387114, G2471646051, G2522063444, G2835601961, G29951154, G3131374986, G3216574877, G3553156159, G3710456697, G3752451908, G3870774570, G3908963977, G4119534169, G43180419, G4475667458, G4501223628, G4800177743, G5207404921, G5209990037, G528601746, G5324093878, G5365730499, EFFECTS OF TRAUMATIC BRAIN INJURY (TBI) AND POST TRAUMATIC STRESS DISORDER (PTSD) ON ALZHEIMER'S DISEASE (AD) IN VETERANS USING IMAGING AND BIOMARKERS IN THE AD NEUROIMAGING INITIATIVE (ADNI)", G5654639750, G5740349702, G5850682332, G613619302, G6173816828, G6210868510, G6371410416, G6412110892, G6447063303, G6719828067, G6792506723, G6899163935, G72415811, G7298697282, G764699806, G7799614282, G781725486, G8228018387, G8281378126, G839208290, G8852128998, G8895907669, G8897063270, G8957659042, G9000039018, G9000122335
OpenAlex ID: W4404313145