Work

Trust Fusion: Explainable Multi-Modal AI for Intelligent Medical Diagnosis

Year: 2026
Type: article
Abstract: Medical diagnosis is an important part of healthcare because early and accurate diagnosis helps doctors provide better treatment and improves patient outcomes. Deep learning techniques have shown good performance in disease detection and medical image analysis. However, many existing models work like black boxes, making it difficult for doctors to understand how a diagnosis is made. This lack of transparency reduces confidence in using these systems in clinical practice. In addition, most diagnostic models use only one type of data and do not fully utilize the valuable information available from medical images and Electronic Health Records (EHRs). This paper presents MedXTrust, a trust-aware multi-modal explainable deep learning framework for medical diagnosis. The framework combines medical images with patient clinical records to improve both diagnostic accuracy and interpretability. A Vision Transformer (ViT) is used to extract features from medical images, while a Clinical Transformer processes structured EHR data. The extracted features are combined using a cross-modal attention mechanism to learn the relationship between imaging and clinical information. To improve transparency, the framework uses SHAP, Grad-CAM++, and counterfactual explanations to help doctors understand the reasons behind each prediction. A trust evaluation module is also included to measure explanation quality, stability, and overall system reliability. Experimental results on publicly available datasets show that the proposed framework provides accurate predictions along with clear and meaningful explanations, making it a reliable tool for supporting clinical decision-making.
Language: English
Open Access status: diamond
OpenAlex ID: W7217921529
FWCI:
Cites: 18
Cited by:
Related to: 10