MIFF-CLIP-HCC: Contrastive Language-Image Pretraining-enhanced multimodal information fusion framework for early recurrence prediction in postoperative hepatocellular carcinoma
Year: 2026
Type: article
Abstract: Background: Hepatocellular carcinoma (HCC) is one of the most common malignant tumors in China, and its prognosis is affected by many factors, such as tumor stage, liver function status, and patient health. To formulate treatment plans and evaluate prognosis, clinicians require accurate prediction of early postoperative recurrence of HCC. To address this clinical need, this study proposes a novel deep learning framework, termed MIFF-CLIP-HCC, to predict the early postoperative recurrence of HCC based on multimodal information fusion framework (MIFF) and Contrastive Language-Image Pretraining (CLIP). Methods: The MIFF-CLIP-HCC framework comprises two core steps. It first uses a fusion framework with a bidirectional attention mechanism to integrate multimodal information, including arterial phase images, venous phase images, and clinical data from patients with HCC. It then introduces image features extracted by the foundational CLIP model to deeply mine effective information from arterial and venous phase images, which increases the performance of the model. Results: Between 2022 and 2025, a total of 240 early-stage HCC patients from two centers were included. Of these, 180 were from The First Affiliated Hospital of Sun Yat-sen University (Center 1) (including 114 early recurrence cases and 66 non-early recurrence cases, with an 8:2 ratio for training and internal testing), and the remaining 60 were from Sun Yat-sen University Cancer Center (Center 2) (including 35 early recurrence cases and 25 non-early recurrence cases), designated as the external test cohort. The accuracy and robustness of the model were validated on internal and external test sets. The model achieved area under the receiver operating characteristic curve (AUC) values of 0.844 [95% confidence interval (CI): 0.726-0.963] on the internal test set and 0.759 (95% CI: 0.651-0.867) on the external test set, outperforming other fusion frameworks (best baseline AUC: 0.779 on internal and 0.690 on external). Conclusions: The results demonstrate that the proposed MIFF-CLIP-HCC framework, which employs dual optimization strategies of multimodal information fusion and CLIP, significantly improves the accuracy of early postoperative recurrence prediction for HCC, thus demonstrating considerable potential for clinical application.
Language: English
Subfield: Radiology, Nuclear Medicine and Imaging
Field: Medicine
Domain: Health Sciences
Open Access status: diamond
OpenAlex ID: W7223207822