Work

ITCNET: a visual and language pretraining model based on multimodal fusion in predicting the breast cancer subtypes on DCE-MRI

Year: 2026

Type: article

Abstract: OBJECTIVE: To develop a deep learning model for precise prediction of breast cancer molecular subtypes by integrating dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), clinical data, and radiology reports. METHODS: We analyzed a retrospective cohort of 505 patients with pathologically confirmed breast cancer. Several models were developed and evaluated: image-based models (Swin Transformer, DenseNet121, ResNet50), unimodal models based on clinical data and radiology reports (using BERT), and multimodal fusion models. The proposed ITCNET model integrates all three data modalities. RESULTS: The Swin Transformer was the best image-based model. The report-based BERT model surpassed other unimodal approaches. Among bimodal models, fusing reports with clinical data yielded the highest performance. Ultimately, the comprehensive ITCNET model achieved superior accuracy (0.9377) and AUC (0.9935). CONCLUSION: The ITCNET model demonstrates that multimodal fusion significantly enhances the prediction of breast cancer molecular subtypes, showcasing the potential of visual-language pretraining to advance precision oncology.

Source: BMC Medical Informatics and Decision Making

DOI: 10.1186/s12911-026-03513-1

Authors: Yang Lou, Haoming Zhuang

Institutions: Affiliated Hospital of Hebei University, Hebei University, Northeastern University

Language: English

Topic: Radiomics and Machine Learning in Medical Imaging

Subfield: Radiology, Nuclear Medicine and Imaging

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,990