Robust multimodal breast cancer diagnosis using DeiT and ANN fusion with minimal features on imbalanced clinical-pathological data
Year: 2025
Type: article
Abstract: Breast cancer remains a leading cause of mortality and requires early and accurate diagnosis to improve patient outcomes. Traditional diagnostic methods often rely on either clinical records or imaging, limiting their ability to capture complementary information. This study proposes a deep learning-based multimodal framework that integrates pathological images with structured clinical data to improve breast cancer classification. We evaluated CNNs and transformer-based architectures (ViT, DeiT) on 3,764 paired pathological images and structured clinical data from 185 patients. We applied transfer learning, class weighting, SMOTE, and augmentation to address challenges of limited data and class imbalance. The DeiT-based multimodal model achieved 98% accuracy, 98% F1-score, and 100% AUC-ROC, outperforming both CNN- and transformer-only baselines. This work contributes a data-efficient multimodal framework that achieves state-of-the-art performance with limited data. It explicitly addresses class imbalance to enhance generalisation, and incorporates feature selection to improve interpretability. Moreover, it demonstrates that a lightweight DeiT-based model can outperform both CNNs and other transformers under data-scarce conditions.
Authors: Tayfun Kok, Temitayo Matthew Fagbola
Institution: University of Hull
Language: English
Topic: AI in cancer detection
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W4415667254