Work

Automatic Virtual‐to‐real Calibration and Dynamic Registration of Deformable Tissue for Endoscopic Submucosal Dissection

Year: 2025

Type: article

Abstract: During endoscopic submucosal dissection, precise and intuitive sensing of target tissues enhances surgical accuracy. Augmented reality (AR) technology currently offers a solution to provide intuitive guidance. To enhance the AR user experience, an automated method for calibrating and dynamically registering deformable tissues is proposed. First, an automatic calibration method is proposed to help register the target tissue from the virtual to the real world. The calibration method is based on a 6D pose estimator, which is built on the feature‐matching network, SuperGlue and the depth estimation network, Metric3D. Subsequently, a dynamic registration method is proposed to track the deformation of the target tissue in real‐time. Moreover, a piece of cloth is utilized for four automatic calibration trials, resulting in a mean absolute error (MAE) of calibration accuracy at 3.79 ± 0.64 mm. The dynamic registration accuracy is also assessed by varying the deformation of the target, yielding an MAE of 6.03 ± 0.96 mm. Finally, an ex vivo experiment involving a piece of small intestine is conducted to validate the effectiveness of the proposed system, with an MAE of 3.11 ± 0.56 mm for AR calibration and 3.20 ± 1.96 mm for dynamic registration error.

Source: Advanced Intelligent Systems

DOI: 10.1002/aisy.202400845

Authors: Yupeng Wang, Huxin Gao, An Wang, Hongliang Ren

Institution: Chinese University of Hong Kong

Language: English

Topic: Augmented Reality Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

SDG: Good health and well-being

Open Access status: gold

Estimated APC paid: $3,970