Year: 2025
Type: conference-paper
Abstract: Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation to detect shoulder and wrist keypoints, and uses linear extrapolation to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating accuracy, the gesturing method is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over method limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: https://github.com/NMKsas/gesture_pointer.git
Authors: Noora Sassali, Roel Pieters
Institutions: Tampere University, University of Tampere
Language: English
Subfield: Human-Computer Interaction
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4415822039