An agentic AI system enhances clinical detection of immune-related adverse events in a multi-phase validation study
Year: 2026
Type: article
Abstract: Immune-related adverse events (irAEs) affect up to 40% of patients receiving immune checkpoint inhibitors, yet their identification depends on laborious and inconsistent manual chart review. Here we developed and evaluated an agentic large language model system to extract the presence, temporality, severity grade, attribution, and certainty of six irAE types from clinical notes. Retrospectively (263 notes), the system achieved macro-averaged F1 of 0.92 for detection and 0.66 for multi-class severity grading; self-consistency improved F1 by 0.14. The best-performing configuration cost approximately $0.02 per note. In prospective silent deployment over three months (884 notes), detection F1 was 0.72-0.79. In a randomized crossover study of clinical trial staff (17 participants, 316 observations), agentic assistance reduced annotation time by 40% (P < 0.001), increased complete-match accuracy (OR 1.45; 95% CI 1.01-2.09; P = 0.045), and improved inter-annotator agreement (Krippendorff's α from 0.22-0.51 to 0.82-0.85). These results demonstrate that agentic AI coupled with human verification could enhance efficiency, performance, and consistency for irAE assessment.
Source: Nature Cancer
Authors: Jack Gallifant, Shan Chen, Kee-Young Shin, Katherine C. Kellogg, Patrick F. Doyle, Joyce Guo, Bingyang Ye, Andrew Warrington, Bingxue K. Zhai, Matthew J. Hadfield, Alexander Gusev, Biagio Ricciuti, David C. Christiani, Hugo JWL Aerts, Benjamin H. Kann, Raymond H. Mak, Tanna L. Nelson, Paul Nguyen, Jonathan D. Schoenfeld, Umit Topaloglu, Paul Catalano, Harry Hochheiser, Jeremy L. Warner, Elad Sharon, David E. Kozono, Guergana K. Savova, Danielle S. Bitterman
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $12,690
OpenAlex ID: W7133189263