Year: 2025
Type: conference-paper
Abstract: This paper presents the development of a 32-degree-of-freedom (DoF) humanoid robotic system designed for autonomous piano performance. The system integrates a vision-based music sheet reader with a YOLOv8 neural network for real-time detection and classification of musical symbols, achieving a mean average precision (mAP) of 96% at IoU 0.5. A heuristic-based synchronization and motion planning module computes optimal finger trajectories and hand placements, enabling expressive and temporally accurate performances. The robotic hardware comprises two anthropomorphic hands mounted on linear rails, each with independently actuated fingers capable of vertical, horizontal, and rotational movements. Experimental validation demonstrates the system’s ability to execute complex musical passages with precision and synchronization. Limitations related to dynamic expressiveness and symbol generalization are discussed, along with proposed enhancements for future iterations. The results highlight the potential of AI-driven robotic systems in musical applications and contribute to the broader field of intelligent robotic performance.
Institution: Transylvania University of Brașov
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4416129879