Year: 2025
Type: letter
Abstract: This paper presents a correspondence-free, function-based sim-to-real learning method for controlling deformable freeform surfaces. Unlike traditional sim-to-real transfer methods that strongly rely on marker points with full correspondences, our approach simultaneously learns a deformation function space and a confidence map – both parameterized by a neural network – to map simulated shapes to their real-world counterparts. As a result, the sim-to-real learning can be conducted by input from either a 3D scanner as point clouds (without correspondences) or a motion capture system as marker points (tolerating missed markers). The resultant sim-to-real transfer can be seamlessly integrated into a neural network-based computational pipeline for inverse kinematics and shape control. We demonstrate the versatility and adaptability of our method on two vision devices and across four pneumatically actuated soft robots: a deformable membrane, a robotic mannequin, and two soft manipulators.
Source: IEEE Transactions on Robotics
Authors: Yingjun Tian, Guoxin Fang, Renbo Su, Aoran Lyu, Neelotpal Dutta, Weiming Wang, Simeon Gill, Andrew Weightman, Charlie C.L. Wang
Institutions: University of Manchester, Chinese University of Hong Kong
Language: English
Subfield: Biomedical Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7079465600