Work

Multi-modal segment anything model (mmSAM) for tumor segmentation in multi-tracer oncologic PET/CT

Year: 2026

Type: article

Abstract: Lesion segmentation is important in targeted radionuclide therapy dosimetry. In this study, we proposed a novel 2D multi-modal Segment Anything Model (mmSAM) for lesion segmentation in whole-body multi-tracer PET/CT images. The AutoPET 2024 dataset, including 18 F-PSMA (369 subjects), 18 F-FDG (170 subjects) and 68 Ga-PSMA (170 subjects) PET/CT images and their corresponding tumor masks, was used in this study. The 18 F-PSMA PET/CT dataset was used as the primary dataset and was divided into 233: 36: 100 for training, validation and testing the mmSAM, using both PET and CT input. The transferability of the primary model was evaluated on 100 patients on other 2 datasets, without and with fine-tuning using 70 cases of 18 F-FDG and 68 Ga-PSMA respectively. Standard single modal 2D SAM with only PET input, 3D nnUNet with two-channel PET/CT input and the thresholding-based method were also implemented for comparison. Mean Dice, the 95th percentile Hausdorff distance (mean HD95), mean standardized uptake value (mean |SUV mean | error), metabolic tumor volume (mean |MTV| error), true positive rate (TPR), positive predictive value (PPV), and false discovery rate (FDR) were computed for all tumors. Statistical significance among different segmentation methods was evaluated using the Wilcoxon test. mmSAM achieved the best performance as compared to other segmentation methods for the primary 18 F-PSMA dataset (mean Dice/HD95/|SUV mean | error/|MTV| error/TPR/PPV/FDR = 0.76/1.92 mm/5.10%/14.60%/100%/96.51%/3.49%, all p < 0.05), without fine-tuning (mean Dice/HD95/|SUV mean | error/|MTV| error/TPR/PPV/FDR = 0.61/2.28 mm/ 15.83%/27.71%/100%/78.91%/21.09% for 18 F-FDG; 0.77/1.33 mm/5.55%/17.96%/100% /97.78%/2.22% for 68 Ga-PSMA, p < 0.05), as well as with fine-tuning (mean Dice/HD95/|SUV mean | error/|MTV| error/TPR/PPV/FDR = 0.65/1.81 mm/6.62%/13.93%/100%/ 80.50%/19.50% for 18 F-FDG; 0.81/1.15 mm/4.62%/14.30%/100%/99.30%/0.70% for 68 Ga-PSMA, p < 0.05) on the cross-tracer datasets. The proposed mmSAM is promising for lesion segmentation in multi-tracer oncologic PET/CT images. Fine-tuning significantly enhances segmentation accuracy for cross-tracer studies.

Source: EJNMMI Physics

DOI: 10.1186/s40658-026-00887-z

Authors: Zhijie Fang, Yibin Liu, Tao Tan, Feng Wang, Greta S. P. Mok

Institutions: University of Macau, Macao Polytechnic University, Nanjing First Hospital, Nanjing Medical University

Language: English

Topic: Medical Image Segmentation Techniques

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $2,490