Segmentation of Image Observations for Articulated Endoscope Control
Year: 2025
Type: article
Abstract: We present a method to reduce the sim-to-real gap for down-stream reinforcement-learning approaches that are trained in simulation. Our pipeline converts endoscopic videos into simplified segmentation masks without any model retraining. Using the zero-shot capability of the Segment Anything Model (SAM), a lightweight HSV filter and SORT tracking, our method produces consistent semantic masks for a pickand- place task with an articulated endoscope. We quantitatively show the importance of each of these components. Unlike typical sim-to-real approaches that render synthetic data more realistic, we make real footage "look synthetic". Experiments on a physical setup demonstrate promising segmentation masks even under partial occlusion, motion blur and lighting variations.
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7124683097
FWCI:
Cites:
Cited by:
Related to: