Work

Point Cloud Based Visual Planning and Servoing for Autonomous Vine Pruning

Year: 2026

Type: article

Abstract: This paper addresses vision based autonomous vine pruning with robots. The complex structure of vines makes visual servoing difficult due to challenges in feature extraction and 3D pose estimation. A novel approach interweaving visual servoing and vision based planning is proposed. An online planner features a Nonlinear Model Predictive Control to compute on-the-fly suitable 3D waypoints. These are navigated by a Position Based Visual Servoing integrating Iterative Closest Point based point-cloud alignment. Qualitative and quantitative evaluations are conducted on live experiments where a Franka Emika manipulator embedding a eye-in-hand stereo camera performs a sequence of pruning actions on seven increasingly complex vine stocks, in spite of unexpected motion of the vine and/or the robot's base. Outdoor experiments are also carried out in strong wind conditions.

Source: IEEE Robotics and Automation Letters

DOI: 10.1109/lra.2026.3664530

Authors: Fadi Gebrayel, Thomas O’Brien, Martin Mujica, Patrick Danès

Institutions: Centre National de la Recherche Scientifique, Université Toulouse III - Paul Sabatier, Laboratoire d'Analyse et d'Architecture des Systèmes, Université de Toulouse, University of Newcastle Australia

Language: English

Topic: Robot Manipulation and Learning

Subfield: Control and Systems Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W4414626301

FWCI:

Cites:

Cited by:

Related to: