Evaluating human-in-the-loop strategies for artificial intelligence-enabled translation of patient discharge instructions: a multidisciplinary analysis
Year: 2025
Type: article
Abstract: Machine translation supported by artificial intelligence (AI) may enhance linguistically-concordant care for patients speaking languages other than English. This assessment of free-text inpatient discharge instructions in Arabic, Armenian, Bengali, simplified Chinese, Somali, and Spanish compared linguist, clinician, and family caregiver evaluations of translations generated by (1) ChatGPT-4o, (2) professional linguists, and (3) human-in-the-loop (AI-generated, professional linguist post-edited). Likert scales (1-5; higher is better) evaluated linguistic and clinical characteristics of each translation. ChatGPT-4o exhibited variable performance relative to professional translations, with poorest ratings for digitally underrepresented languages (Armenian and Somali). Conversely, human-in-the-loop translations achieved comparable, often better, outcomes to professional translations for all languages, (e.g., Armenian mean overall quality: 3.9 [95% CI 3.7-4.2] vs. professional 3.6 [3.4-3.9], p = 0.01), were most frequently preferred (46.5% vs. 28.4%) and had shorter mean translation time (7.1 [5.4-8.8] vs. 16.8 [13.7-19.9] min, p < 0.001). Human-in-the-loop strategies may enable safe, efficient, equitable machine translation application in clinical practice.
Source: npj Digital Medicine
Authors: Ryan CL Brewster, Gabriel Tse, Angela L. Fan, Marwa Elborki, Maiah Newell, Priscilla Gonzalez, Amitra Hoq, Crystal Chang, Maksud Chowdhury, Adiba Geeti, Marlin Hana, Hoda Hassan, Osama Ibrahim, Lucine Keseyan, Qing Li, Md Mamoon, Maymona Nageye, Arthur Ohannessian, Ilan Rozen Eisenberg, Mohammad Sallam, Giordano Sosa Soto, Christina Su, Raffi Tachdjian, Mondira Ray, Hannah Lev, Jonathan D. Hron, Nate Shaar, Nicholas Kuzma, Alisa Khan
Institutions: Boston Medical Center, Boston Children's Hospital, Beth Israel Deaconess Medical Center, Harvard University, Stanford Medicine, Stanford University, New York University, Kaiser Permanente Bernard J. Tyson School of Medicine, SUNY Downstate Health Sciences University, NYU Langone Health, Virginia Commonwealth University, Children's Hospital of Richmond at VCU, The Ohio State University Wexner Medical Center, The Ohio State University, Cedars-Sinai Medical Center, University of Mississippi Medical Center, Icahn School of Medicine at Mount Sinai, Avalon University School of Medicine, University of California, Los Angeles, Mercy Medical Center, Saint John's Health Center, St. Christopher's Hospital for Children, Drexel University
Language: English
Subfield: General Health Professions
Field: Health Professions
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $4,090
Award: G1550866921
OpenAlex ID: W4415525289