Work

Discrete diffusion model with contrastive learning for music to natural and long dance generation

Year: 2025

Type: article

Abstract: With the deep integration of culture and technology, the digital research on cultural content like music and dance is constantly evolving. This paper focuses on the research of music and dance generation, aiming to boost cultural dissemination. The core challenge of this task is to generate natural dance sequences that align with the duration of the provided music. Therefore, we propose a discrete diffusion model with contrastive learning. First, the dance VQ-VAE model is introduced and pre-trained to learn the mapping relationship between dance data and discrete token sequences. Second, the Music-conditional Contrast Learning loss is designed to enhance the training of the discrete diffusion model, enabling it to predict discrete token sequences conditioned on musical features. Subsequently, the discrete token sequences are decoded into dance sequences with the dance VQ-VAE decoder. Finally, the temporal consistency between multiple sequences is enhanced by implementing time constraints to generate long dance sequences.

Source: npj Heritage Science

DOI: 10.1038/s40494-025-01668-0

Authors: Huaxin Wang, Yujian Jiang, Xiangzhong Zhou, Wei Jiang

Institution: Communication University of China

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Natural Science Foundation of China, Ministry of Science and Technology of the People's Republic of China, Fundamental Research Funds for the Central Universities

Awards: G1224959027, G1446587438