Depthwise-dilated convolutional adapters for medical object tracking and segmentation using the segment anything model 2
Year: 2025
Type: article
Abstract: Recent advances in medical image segmentation have been driven by deep learning; however, most existing methods remain limited by modality-specific designs and exhibit poor adaptability to dynamic medical imaging scenarios. The Segment Anything Model 2 (SAM2) and its related variants, which introduce a streaming memory mechanism for real-time video segmentation, present new opportunities for prompt-based, generalizable solutions. Nevertheless, adapting these models to medical video scenarios typically requires large-scale datasets for retraining or transfer learning, leading to high computational costs and the risk of catastrophic forgetting. To address these challenges, we propose DD-SAM2, an efficient adaptation framework for SAM2 that incorporates a Depthwise-Dilated Adapter (DD-Adapter) to enhance multi-scale feature extraction with minimal parameter overhead. This design enables effective fine-tuning of SAM2 on medical videos with limited training data. Unlike existing adapter-based methods focused solely on static images, DD-SAM2 fully exploits SAM2's streaming memory for medical video objects tracking and segmentation. Comprehensive evaluations on TrackRad2025 (tumor segmentation) and EchoNet-Dynamic (left ventricle tracking) datasets demonstrate superior performance, achieving Dice scores of 0.93±0.04 and 0.97±0.01, respectively. To the best of our knowledge, this work provides an initial attempt at systematically exploring adapter-based fine-tuning strategies for SAM2 applied medical video segmentation and tracking. Code, datasets, and models will be made publicly available at https://github.com/apple1986/DD-SAM2.
Authors: Guoping Xu, Christopher Kabat, You Zhang
Institution: The University of Texas Southwestern Medical Center
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,000
Awards: G2836918769, High-Precision Lung Radiotherapy by Intra-treatment Dynamic Cone-beam CT imaging and Dosimetry-guided Plan Adaptation, Accurate 4D Liver Tumor Localization for Radiotherapy using Contrast-Agent-Free X-ray Imaging and Liver Biomechanical Modeling, G5428726694, G6066490611, High-Precision Proton Therapy for Liver Cancer: Developing an End-to-end Strategy with Real-time Liver Tumor Localization and On-the-fly Plan Delivery Adaptation
OpenAlex ID: W4415202774