Trinocular vision with deep learning for object twist estimation: a benchmark approach and dataset
Year: 2025
Type: article
Abstract: Accurate estimation of a rigid object’s 6D pose and twist is a fundamental challenge for enabling autonomous systems operation and interaction with the environment. Monocular vision can mitigate the drift inherent in inertial methods and enables the estimation of non-cooperative target twist. However, the estimation accuracy of monocular vision is compromised by inherent depth ambiguity, a limitation that multi-camera systems can overcome. The lack of multi-view datasets with high-precision annotations hinders the development of robust perception algorithms. To address this, we present the first trinocular pose and twist estimation dataset for non-cooperative targets, comprising images of aircraft (7,824 for training, 4,710 for testing) and satellites (7683 for training, 4380 for testing), all manually annotated with sub-pixel-level keypoints and pose labels derived from optimization. We develop a neural network that predicts semantic keypoints for robust pose estimation. Combined with a multi-view optimization framework and twist estimation, our system achieves a mean angular velocity error of \(0.1^{\circ }\) /s and a mean linear velocity error of 0.3mm/s. Our open-source dataset and method provide a critical benchmark for future research in aerospace missions. We have open-sourced the dataset at the following https://www.kaggle.com/datasets/mingshiwuwjh/trinocular-pose-and-twist-estimation-dataset.
Source: Artificial Intelligence Review
Authors: Jinghao Wang, KaiKai Cui, Teng Teng, Zi Wang, Boming Ren, Qifeng Yu
Language: English
Subfield: Aerospace Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,190
Awards: G1718745046, 空间非合作目标位姿单目测量及不确定度评定方法研究
OpenAlex ID: W4417121671