Year: 2025
Type: conference-paper
Abstract: Segmented light field images can serve as a powerful representation in many computer vision tasks exploiting geometry and appearance of objects, such as object pose tracking. For those images, segmentation presents an additional objective of recognizing the same segment through all the views. Segment Anything Model 2 (SAM 2) allows for producing semantically meaningful segments for monocular images and videos. Using the video SAM 2 on less general 4D functions such as light fields is ineffective. In this work, we present a novel segmentation method that adapts SAM 2 to the light field domain without retraining or modifying the model. By utilizing epipo-lar constraints, our method produces high quality and view-consistent masks, outperforming the SAM 2 video tracking baseline and working 7 times faster, moving towards a real-time segmentation speed. We achieve this by exploiting the epipolar geometry cues to propagate the masks between the views, probing the SAM 2 latent space to estimate their occlusion, and further prompting SAM 2 for their refinement. The code and additional materials are available at https://roboticimaging.org/Projects/LFSAM/.
Authors: Nikolai Goncharov, Donald G. Dansereau
Institution: The University of Sydney
Language: English
Topic: Advanced Vision and Imaging
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4409917070