Work

Challenges for human oversight in detecting system malfunctions in AI-assisted medical diagnostics

Year: 2026
Type: article
Study design: Clinical Trial
Abstract: Increasingly, physicians must oversee AI systems to detect system malfunctions. We studied 48 physicians in a randomized controlled study using an increasingly malfunctioning AI for cardiac auscultation. Diagnostic performance declined alongside AI reliability, even when physicians were cognitively forced to make an initial classification, and when human performance exceeded AI reliability. Physicians' confidence declined faster than their trust in AI. Results call for specification of oversight goals and processes ensuring patient safety.
Language: English
Field: Medicine
Open Access status: gold
Estimated APC paid: $4,090
OpenAlex ID: W7226001971
FWCI:
Cites: 33
Cited by:
Related to: 10