HAT-SAM3: endoscopy-aware adaptation of a foundation segmentation model for generalizable polyp segmentation
Year: 2026
Type: article
Abstract: Automatic polyp segmentation remains difficult when models are evaluated outside their training distribution. Polyps vary substantially in scale, shape, color, boundary contrast, and surface appearance; colonoscopy further introduces specular reflection, blur, fluid, and illumination change. These factors make cross-dataset generalization a stricter problem than in-domain accuracy. In this paper, we present HAT-SAM3, an endoscopy-aware foundation model adaptation based on highlight-aware training (HAT) for generalizable polyp segmentation. Instead of designing another task-specific decoder, we adapt a visual foundation model, SAM 3, as a pretrained segmentation prior for cross-dataset polyp segmentation. To account for unreliable local appearance cues caused by specular saturation, we use a training-only boundary regularizer that stochastically adds saturated pixels in a narrow exterior band around annotated polyp masks to the foreground supervision, while leaving inference unchanged. In the primary five-dataset public benchmark, HAT-SAM3 achieves an average mDice of 0.884 and the best listed Dice on four of five datasets, with the most consistent improvements on out-of-domain test sets. In an additional Kvasir-to-external zero-shot evaluation, HAT-SAM3 improves mDice over the strongest reported baseline by 0.062–0.131 across three external benchmarks. Together, these evaluations indicate that endoscopy-aware adaptation of a foundation segmentation prior can improve cross-dataset polyp segmentation. The code is publicly available at https://github.com/HaoLi12345/polyp .
Source: The Visual Computer
Authors: Hao Li, Anum Masood
Institutions: Vanderbilt University, Queen Mary University of London
Language: English
Subfield: Oncology
Field: Medicine
Domain: Health Sciences
Open Access status: hybrid
Estimated APC paid: $3,190
OpenAlex ID: W7166525014