MLUG: Bootstrapping Language-Motion Pre-Training for Unified Motion-Language Understanding and Generation
Year: 2024
Type: article
Abstract: In the realm of computer vision and animation, the generation of human motion from textual descriptions represents a frontier of significant challenge and potential. This paper introduces MLUG, a groundbreaking framework poised to transform motion synthesis by harnessing the power of vision-language pre-training techniques. MLUG addresses the nuanced challenge of creating semantically rich, physically plausible, and emotionally expressive human motions through a novel integration of a unimodal encoder with motion-text contrastive loss, a motion-grounded text encoder, a motion-grounded motion decoder, and a motion length predictor. These components work in concert to align textual descriptions with dynamic motion sequences, offering an innovative solution to the limitations of existing models in open-vocabulary motion generation and emotional expressiveness. Through extensive evaluations, MLUG demonstrates unparalleled effectiveness in generating realistic and diverse motions from a broad spectrum of textual inputs, setting a new benchmark in the field.
Source: Sensors
DOI: 10.3390/s24227354
Authors: Hongliang Luo, Wei Xi, Daniel Tang
Institution: Xi'an Jiaotong University
Language: English
Topic: Human Motion and Animation
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $2,955
OpenAlex ID: W4404527497