Year: 2024
Type: conference-paper
Abstract: Key-points serve as critical visual features for robotic manipulation, yet their potential for guiding robots to handle objects and generalize across tasks remains underexplored. In this work, we propose a key-point based servoing framework that leverages a pre-trained Vision Transformer (ViT) to extract semantically rich key-points from images. Unlike prior works limited to known objects, a generic lightweight neural network is trained to directly map 2D key-point displacements to 3D Cartesian-space servo commands, bypassing explicit perspective geometry modeling. This network learns a unified 2D-to-3D correspondence that generalizes across diverse tasks and scenarios. Experiments in simulation and real-world setups demonstrate that our method achieves higher precision than optimization-based pose estimation and better generalization across unseen tasks. By bridging key-points with task-agnostic servo command prediction, this work advances feature-based manipulation research and offers practical solutions for logistics and service robots handling unstructured objects.
Authors: Chen Zhang, Zhuoli Xie, Genliang Chen
Institution: Shanghai Jiao Tong University
Language: English
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4412524639