Recognition, Stabilization, and Command-Mapping Algorithms for Contactless Hand-Gesture Control of an ABB Industrial Robot
Year: 2026
Type: article
Abstract: Although contactless hand-gesture interfaces can support intuitive human–robot interaction, transient classification errors may trigger unintended commands when classifier outputs are mapped directly to robot actions. This study presents a processing framework for contactless control of an ABB IRB 120 industrial robot. The framework combines RGB acquisition, geometric normalization of 21 MediaPipe hand landmarks, population-based sample filtering, multilayer perceptron classification, temporal transition stabilization, and a two-hand command grammar. Thirteen gesture classes were represented by 63-dimensional landmark vectors, and 19,250 samples were retained after processing. The classifier achieved an approximate mean per-class recall of 99.46% on the held-out test subset. Four temporal processing configurations were compared using 30 repetitions per interface action: raw output, majority voting, three-consecutive-label confirmation, and the proposed cellular-automaton (CA)-based stabilizer. The latter two methods improved command reliability relative to the raw output and produced comparable results. The CA stabilizer processed labels sampled every 100 ms and introduced a validation delay of approximately 300 ms. Robot-program control achieved an 86.7% success rate across 90 trials, while Cartesian direction control achieved a 97.2% success rate across 180 trials. The mean displacement overshoot was 2.42 mm, or 24.2% of the nominal 10 mm command. The framework provides a practical perception-to-control interface while complementing, rather than replacing, certified robot safety functions.
Source: Sensors
DOI: 10.3390/s26217073
Authors: Bartosz Bieńkowski, Wojciech Kaczmarek
Institution: Military University of Technology in Warsaw
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $3,137
OpenAlex ID: W7225633657