Get started
Work

Large language models require a new form of oversight: capability-based monitoring

Year: 2026
Type: article
Abstract: Large language models (LLMs) have been rapidly adopted in healthcare, but oversight strategies are lacking. We propose capability-based monitoring, motivated by the fact that LLMs are generalist systems whose overlapping internal capabilities are reused across numerous downstream tasks. This approach organizes monitoring around shared capabilities to enable cross-task detection of systemic weaknesses, long-tail errors, and emergent behaviors. We describe considerations for developers, organizational leaders, and professional societies. and policymakers.
DOI: 10.1038/s41746-026-02740-0
Authors: Katherine C. Kellogg, Bingyang Ye, Yifan Hu, Guergana K. Savova, Byron Wallace, Danielle S. Bitterman
Institutions: Massachusetts Institute of Technology, Brigham and Women's Hospital, Harvard University, Dana-Farber Cancer Institute, Dana-Farber Brigham Cancer Center, Mass General Brigham, Brandeis University, Boston Children's Hospital, Northeastern University, Dana-Farber/Harvard Cancer Center
Language: English
Field: Medicine
Open Access status: gold
Estimated APC paid: $4,090
Funders: American Cancer Society, Patient-Centered Outcomes Research Institute, Harvard University, Harvard Catalyst, Division of Cancer Prevention, National Cancer Institute, National Institutes of Health, National Cancer Institute, National Center for Advancing Translational Sciences
Awards: Artificial Intelligence to Identify and Characterize Cancer Therapy-Induced Adverse Sequelae, G2182112902, G451927975, G6735605043, G8446248655, Radiation Oncology at the Interface of Pediatric Cancer Biology and Data Science, Cancer Deep Phenotype Extraction from Electronic Medical Records (renewal), Informatics strategies to improve immune-related adverse event detection in cancer patients, Harvard Clinical and Translational Science Center
OpenAlex ID: W4416018535
FWCI: 1.947
Cites:
Cited by: 1
Related to: 10