S29 Radiologist-explainable AI concordance and confidence in explainable AI supported clinical imaging decisions in lung cancer early diagnosis
Year: 2026
Type: conference-abstract
Abstract: Background Explainable Artificial Intelligence (XAI) may support pulmonary nodule risk stratification, but its clinical value depends on how radiologists use AI outputs during interpretation. Although the impact of AI has mostly been evaluated performance metrics, interactions between clinicians and XAI remain poorly understood. We evaluated whether AI-predicted malignancy risk with morphology-based explanations influenced radiologist decisions and confidence. Methods Four consultant radiologists independently assessed 100 biopsy-confirmed pulmonary nodules from a multi-centre UK cohort using a counterbalanced design. Each radiologist reader reviewed one list with AI assistance and the other without AI assistance. Reader pairs were assigned opposite conditions so that each case was assessed under both conditions by different readers. XAI support quantified radiological characteristics, such as spiculation, nodule type, to assist radiologists. In the AI-assisted condition, radiologists were shown AI-predicted malignancy risk with explanations simultaneously. Readers recorded a binary malignancy decision and confidence score. Results Aggregate performance of readers was comparable with and without AI assistance (accuracy 0.78 vs 0.76, sensitivity 0.94 vs 0.93, specificity 0.68 vs 0.64, all p>0.05). Average radiologist confidence was higher for correct than incorrect calls in both conditions (3.69 vs 3.00, p < 0.01 with AI, 3.55 vs 2.57, p < 0.001 without AI). Specifically, confidence was highest when their decisions agreed with the correct AI output (3.85) but remained high when concurring with an incorrect AI (3.39). However, confidence was low when readers correctly disagree an incorrect AI (2.53) and lowest when disagreeing with a correct AI (1.82). Discussion AI assistance did not significantly improve aggregate diagnostic performance. In both conditions, radiologist confidence was higher for correct than incorrect cases suggesting that radiologists had some insight into which calls were reliable. However, confidence tracked concordance with the AI more than diagnostic correctness as readers remained confident when agreeing with an incorrect AI. Among discordant reads, confidence was relatively higher when readers correctly overrode an incorrect AI than when they disagreed with a correct one. These data have important implications for radiologist trust in AI decision-support and whether this varies according to case difficulty. The role of AI in ‘more challenging’ cases requires further study.
Source: Thorax
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W7225160719
