Year: 2026
Type: article
Abstract: This study evaluates the quality of artificial intelligence (AI) clinical note summarization by analyzing physician qualitative feedback on a large language model (LLM) chart review tool integrated into the electronic health record (EHR). Physicians provided free-text feedback on AI-generated chart summaries, which physician informaticists analyzed using MAXQDA. Feedback from 10 physicians was collected on 147 AI-generated summaries. Positive feedback was common (n = 71), but users identified omissions (n = 46), confusing content (n = 20), token limitations (n = 27), hallucinations (n = 5), and bias (n = 1). Cohen's Kappa was 0.64, indicating substantial reviewer agreement. Physician feedback on the tool revealed overall positive impressions, though omissions raised concerns about summary completeness. AI-assisted chart review technology is not infallible, but physicians found this tool acceptable for use in clinical workflows.
Source: npj Health Systems
Institution: University of California San Diego
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,790
OpenAlex ID: W7125974338