RoboTransfer: Controllable Geometry-Consistent Video Diffusion for Manipulation Policy Transfer
Year: 2025
Type: preprint
Abstract: The goal of general-purpose robotics is to create agents that can seamlessly adapt to and operate in diverse, unstructured human environments. Imitation learning has become a key paradigm for robotic manipulation, yet collecting large-scale and diverse demonstrations is prohibitively expensive. Simulators provide a cost-effective alternative, but the sim-to-real gap remains a major obstacle to scalability. We present RoboTransfer, a diffusion-based video generation framework for synthesizing robotic data. By leveraging cross-view feature interactions and globally consistent 3D geometry, RoboTransfer ensures multi-view geometric consistency while enabling fine-grained control over scene elements, such as background editing and object replacement. Extensive experiments demonstrate that RoboTransfer produces videos with superior geometric consistency and visual fidelity. Furthermore, policies trained on this synthetic data exhibit enhanced generalization to novel, unseen scenarios. Project page: https://horizonrobotics.github.io/robot_lab/robotransfer.
Source: arXiv (Cornell University)
Authors: Liu, Liu, Wang, Xiaofeng, Zhao, Guosheng, Li, Keyu, Qin, Wenkang, Zhu, Jiagang, Qiu, Jiaxiong, Zhu, Zheng, Huang, Guan, Su, Zhizhong
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6891834103
Cites:
Cited by:
Related to: