Challenges for human oversight in detecting system malfunctions in AI-assisted medical diagnostics
Year: 2026
Type: article
Study design: Clinical Trial
Abstract: Increasingly, physicians must oversee AI systems to detect system malfunctions. We studied 48 physicians in a randomized controlled study using an increasingly malfunctioning AI for cardiac auscultation. Diagnostic performance declined alongside AI reliability, even when physicians were cognitively forced to make an initial classification, and when human performance exceeded AI reliability. Physicians' confidence declined faster than their trust in AI. Results call for specification of oversight goals and processes ensuring patient safety.
Source: npj Digital Medicine
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $4,090
Funder: Deutsche Forschungsgemeinschaft
OpenAlex ID: W7226001971
