Work

Multimodal deep learning frameworks for breast cancer detection using ultrasound, mammography, and clinical data

Year: 2026

Type: article

Abstract: Background Accurate differentiation between benign and malignant breast lesions remains challenging when relying on a single imaging modality. Multimodal deep learning offers the potential to integrate complementary diagnostic information from mammography, ultrasound, and clinical data to improve classification performance . Methods A prospective dataset of 92 biopsy-confirmed patients with paired mammography and ultrasound images was analyzed. Four multimodal deep learning models—CUF-MT, AMW-CNN, MM-ABMIL, and CNN-LSTM—were developed using five-fold cross-validation and evaluated on an internal test set. Performance was assessed using AUC, accuracy, sensitivity, specificity, precision, and MCC. Additional analyses included ablation studies, missing-modality experiments, and statistical comparison using McNemar’s test. Results Multimodal models consistently outperformed unimodal approaches. AMW-CNN achieved the highest cross-validation performance (AUC = 0.992), while CUF-MT demonstrated the best generalization on the internal test set (AUC = 0.91). McNemar’s test revealed statistically significant differences between models, with AMW-CNN outperforming others in most pairwise comparisons. Missing-modality analysis showed that ultrasound was the dominant contributor to classification performance, whereas mammography primarily improved prediction calibration. Age provided complementary information, supported by a large effect size (Cohen’s d = 2.94). Ablation studies confirmed the importance of adaptive fusion and cross-modal interactions. Conclusion Multimodal deep learning enhances breast lesion classification, with different architectures offering complementary strengths in performance and generalization. The proposed framework highlights the importance of modality-aware evaluation and robustness analysis, supporting the development of clinically reliable decision-support systems.

Source: Informatics in Medicine Unlocked

DOI: 10.1016/j.imu.2026.101761

Authors: Samaneh Rashedi, Amir Rashedi, Bentolhoda Otroshi Shahreza, Maryam Tabatabaeian, Iraj Abedi

Institutions: Isfahan University of Medical Sciences, Shiraz University of Technology, Arak University of Medical Sciences

Language: English

Topic: AI in cancer detection

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,470