Alligat0R: Pre-Training through Covisibility Segmentation for Relative Camera Pose Regression
Year: 2025
Type: conference-paper
Abstract: Pre-training techniques have greatly advanced computer vision, with CroCo's cross-view completion approach yielding impressive results in tasks like 3D reconstruction and pose regression. However, cross-view completion is ill-posed in non-covisible regions, limiting its effectiveness. We introduce Alligat0R, a novel pre-training approach that replaces cross-view learning with a covisibility segmentation task. Our method predicts whether each pixel in one image is covisible in the second image, occluded, or outside the field of view, making the pre-training effective in both covisible and non-covisible regions, and provides interpretable predictions. To support this, we present Cub3, a large-scale dataset with 5M image pairs and dense covisibility annotations derived from the nuScenes and ScanNet datasets. Cub3 includes diverse scenarios with varying degrees of overlap. The experiments show that our novel pre-training method Alligat0R significantly outperforms CroCo in relative pose regression. Code is available at https://github.com/thibautloiseau/alligat0r.
DOI: 10.52202/085713-0463
Language: English
Subfield: Aerospace Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4417413543
FWCI:
Cites:
Cited by:
Related to: