A Cross-Modal Transformer with Explainable AI for Multimodal Breast Cancer Detection
Year: 2026
Type: conference-paper
Abstract: Breast cancer remains one of the most significant global health challenges, particularly in developing countries. In many regions, early and reliable detection is hindered by limited access to sophisticated medical technologies. Overdiagnosis is another critical concern, occurring when cancer is predicted in patients who do not actually have the disease, leading to unnecessary treatment and increased burden on healthcare systems. In this study, we propose a multimodal cross-modal transformer-based deep learning approach that integrates mammography images with structured clinical data to improve diagnostic reliability and reduce false-positive predictions. The proposed Cross-Modal Transformer enables dynamic interaction between visual (image) and tabular (clinical) features through a cross-attention mechanism. Additionally, Explainable Artificial Intelligence (XAI) techniques are incorporated to enhance model interpretability and clinical trust. Experimental results demonstrate that the proposed model performs better than all comparison methods, achieving an accuracy of 84% and an F1-score of 85%, and surpassing single-modal baselines such as ResNet-50 and the FT-Transformer.
Institutions: Jahangirnagar University, BRAC University
Language: English
Topic: AI in cancer detection
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204451692