Manipulating Elasto-Plastic Objects With 3D Occupancy and Learning-Based Predictive Control
Year: 2025
Type: preprint
Abstract: Manipulating elasto-plastic objects remains a significant challenge due to severe self-occlusion, difficulties of representation, and complicated dynamics. This work proposes a novel framework for elasto-plastic object manipulation with a quasi-static assumption for motions, leveraging 3D occupancy to represent such objects, a learned dynamics model trained with 3D occupancy, and a learning-based predictive control algorithm to address these challenges effectively. We build a novel data collection platform to collect full spatial information and propose a pipeline for generating a 3D occupancy dataset. To infer the 3D occupancy during manipulation, an occupancy prediction network is trained with multiple RGB images supervised by the generated dataset. We design a deep neural network empowered by a 3D convolution neural network (CNN) and a graph neural network (GNN) to predict the complex deformation with the inferred 3D occupancy results. A learning-based predictive control algorithm is introduced to plan the robot actions, incorporating a novel shape-based action initialization module specifically designed to improve the planner efficiency. The proposed framework in this paper can successfully shape the elasto-plastic objects into a given goal shape and has been verified in various experiments both in simulation and the real world.
Source: arXiv (Cornell University)
Authors: Zhang, Zhen, Chu, Xiangyu, Tang, Yunxi, Zhao, Lulu, Huang, Jing, Jiang, Zhongliang, Au, K. W. Samuel
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Funder: Chinese University of Hong Kong
Award: G1584325729
OpenAlex ID: W6979253549
Cites:
Cited by:
Related to: