Work

Human-in-the-loop large language model framework for identification of cutaneous immune-related adverse events

Year: 2026

Type: article

Abstract: This study evaluated a retrieval-augmented, multi-agent large language model (LLM)–driven, human-in-the-loop framework for detecting cutaneous immune-related adverse events (cirAEs) from clinical notes. Compared with the unassisted manual-review group, the LLM-assisted group demonstrated higher accuracy (F1 = 0.88 vs 0.77) and inter-rater agreement measured by Cohen’s κ ( κ = 0.82 vs 0.50), and reported an approximately one-third shorter average review time. This framework pilots how LLMs can be applied to identify immune-related toxicities and, more broadly, support accurate, scalable, and transparent adverse event data extraction.

Source: npj Precision Oncology

DOI: 10.1038/s41698-026-01737-y

Authors: Charles Lu, Olivia Burke, Debby Cheng, Adam Kashlan, Caitlyn Duffy, Zeyun Lu, Lirit Fuksman, Jin Ning Tian, Andrew Sedlack, Priya Katyal, Eudora Lee, Ralina Karagenova, Chuck Lin, Kun-Hsing Yu, Nicole LeBoeuf, Alexander Gusev, Yevgeniy R. Semenov

Institutions: Massachusetts General Hospital, Brigham and Women's Hospital, Harvard University, Dana-Farber Cancer Institute, Dana-Farber/Harvard Cancer Center, The University of Texas MD Anderson Cancer Center

Language: English

Topic: Pharmacovigilance and Adverse Drug Reactions

Subfield: Toxicology

Field: Pharmacology, Toxicology and Pharmaceutics

Domain: Life Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $4,090

Funders: U.S. Department of Defense, Melanoma Research Foundation, Melanoma Research Alliance, National Heart, Lung, and Blood Institute, National Institute of General Medical Sciences, National Institute of Arthritis and Musculoskeletal and Skin Diseases

Awards: RISK PREDICTION MODELS FOR RARE MELANOMAS, Predictive modeling of cutaneous immune checkpoint inhibitor toxicities

OpenAlex ID: W7170657348

FWCI:

Cites:

Cited by:

Related to: