Year: 2026
Type: article
Abstract: With the development of computer vision and real-time interaction technologies, interactive music creation based on body gesture recognition has gradually become an important direction in contemporary electronic music practice. As an open-source, lightweight computer vision framework, MediaPipe offers new technical pathways and practical possibilities for interactive music creation due to its features such as rich solution and parameter offerings, high hardware compatibility, flexibility, and extensive support from the community. This study focuses on the application of MediaPipe technology in interactive music creation. It first outlines the core technical characteristics of MediaPipe and, in comparison with currently popular gesture recognition technologies, analyzes its applicable scenarios and features in creative practice. On this basis, three interactive music works based on MediaPipe are selected as case studies. These works are analyzed from the perspectives of sound material, interaction design, and technical implementation, exploring the functional roles and expressive modes of body gesture recognition in different creative contexts. Through the analysis of these works, the study summarizes several creative characteristics of MediaPipe-based interactive music, including the body as a medium for sound generation and control, the body as a tool for visual expression, and the body as a structural and narrative element within the composition. The findings suggest that MediaPipe not only lowers the technical threshold of interactive music creation, but also encourages a transformation of the body from a functional control interface into an aesthetic and structural component of musical works. By combining technical analysis with creative practice, this study aims to provide a practical reference for computer vision–based interactive music creation and to offer methodological insights for future artistic research and development in this field.
Source: Advances in Humanities Research
Authors: Nianzhi Xue, Xinyu Wang
Institution: Zhejiang Conservatory of Music
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7204444864
FWCI:
Cites:
Cited by:
Related to: