Work

Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias

Year: 2024

Type: conference-paper

Abstract: influence the outputs of LLMs. We expose and quantify discrepancies by juxtaposing these biases against actual disease prevalences in various U.S. demographic groups. Our results highlight substantial misalignment between LLM representation of disease prevalence and real disease prevalence rates across demographic subgroups, indicating a pronounced risk of bias propagation and a lack of real-world grounding for medical applications of LLMs. Furthermore, we observe that various alignment methods minimally resolve inconsistencies in the models' representation of disease prevalence across different languages. For further exploration and analysis, we make all data and a data visualization tool available at: www.crosscare.net.

Source: neural information processing systems

DOI: 10.52202/079017-0749

Authors: Shan Chen, Jack Gallifant, Mingye Gao, Pedro Moreira, Nikolaj Munch, Ajay Muthukkumar, Arvind Rajan, Jaya Kolluri, Amelia Fiske, Janna Hastings, Hugo Aerts, Brian Anthony, Leo Celi, William La Cava, Danielle Bitterman

Institutions: Boston Children's Hospital, Harvard University, Mass General Brigham, Universitat Pompeu Fabra, Aarhus University, Technical University of Munich, University of Zurich, University of St.Gallen, Maastricht University, Beth Israel Deaconess Medical Center

Language: English

Topic: Ethics and Social Impacts of AI

Subfield: Safety Research

Field: Social Sciences

Domain: Social Sciences

SDGs: Good health and well-being, Reduced inequalities

Open Access status: green

Funders: National Science Foundation, European Commission, National Institutes of Health, National Cancer Institute, Fogarty International Center, H2020 European Research Council

Awards: A Learning Ecosystem for Teaching High School Students Machine Learning Concepts and Data Science Skills in Healthcare and Medicine, Bridge2AI: Patient-Focused Collaborative Hospital Repository Uniting Standards (CHoRUS) for Equitable AI, Deep Learning for Automated Quantification of Radiographic Tumor Phenotypes, G4403456761, Critical Care Informatics, G5024937287, Real-time Monitoring and Correction of Clinical Decision Support Systems to Reduce Health Disparities, G9000000961, G9000013229, G9000027136, G9000081776, G9000308345, MUST Data Science Research Hub (MUDSReH), Informatics strategies to improve immune-related adverse event detection in cancer patients

OpenAlex ID: W4396822082

FWCI: 8.754

Cites:

Cited by: 19

Related to: 10