ConsensusXAI: A framework to examine class-wise agreement in medical imaging
Year: 2025
Type: conference-paper
Abstract: Explainable AI (XAI) is essential for trust and transparency in deep learning, especially in medical imaging. Existing local explanation methods provide per-instance insights but fail to show whether similar explanations hold across samples of the same class. This limits global interpretability and demands time-consuming manual review by clinicians to trust models in practice. We introduce the Consensus Alignment Score (CAS), a novel metric that quantifies consistency of explanations at the class level. We also present ConsensusXAI, an open-source, model- and method-agnostic framework that evaluates explanation agreement quantitatively (via CAS) and qualitatively (through consensus heatmaps) per class. Unlike prior benchmarks, ConsensusXAI uses a latent-space clustering approach, Latent Consensus, to identify dominant explanation patterns, exposing biases and inconsistencies towards certain classes. Evaluated across two different medical imaging modalities for both correct and incorrect predictions on two different backbones, our method consistently reveals meaningful class-level insights, outperforming traditional consensus method i.e. SSIM, and enabling faster, more confident clinical adoption of AI models.
Authors: Haider, Abbas; id_orcid 0000-0003-1802-3086, Wright, David, Hogg, Ruth, Wang, Hui, Peto, Tunde, Gault, Richard
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7115279766
FWCI:
Cites:
Cited by:
Related to: