A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives
Year: 2025
Type: preprint
Abstract: Multi-modal music generation, using multiple modalities like text, images, and video alongside musical scores and audio as guidance, is an emerging research area with broad applications. This paper reviews this field, categorizing music generation systems from the perspective of modalities. The review covers modality representation, multi-modal data alignment, and their utilization to guide music generation. Current datasets and evaluation methods are also discussed. Key challenges in this area include effective multi-modal integration, large-scale comprehensive datasets, and systematic evaluation methods. Finally, an outlook on future research directions is provided, focusing on creativity, efficiency, multi-modal alignment, and evaluation.
Source: arXiv (Cornell University)
Authors: Li, Shuyu, Ji, Shulei, Wang, Zihao, Wu, Songruoyao, Yu, Jiaxing, Zhang, Kejun
Institution: Zhejiang University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6929731675