Year: 2025
Type: preprint
Abstract: Recent advances in generative modeling and tokenization have driven significant progress in text-to-motion generation, leading to enhanced quality and realism in generated motions. However, effectively leveraging textual information for conditional motion generation remains an open challenge. We observe that current approaches, primarily relying on fixed-length text embeddings (e.g., CLIP) for global semantic injection, struggle to capture the composite nature of human motion, resulting in suboptimal motion quality and controllability. To address this limitation, we propose the Composite Aware Semantic Injection Mechanism (CASIM), comprising a composite-aware semantic encoder and a text-motion aligner that learns the dynamic correspondence between text and motion tokens. Notably, CASIM is model and representation-agnostic, readily integrating with both autoregressive and diffusion-based methods. Experiments on HumanML3D and KIT benchmarks demonstrate that CASIM consistently improves motion quality, text-motion alignment, and retrieval scores across state-of-the-art methods. Qualitative analyses further highlight the superiority of our composite-aware approach over fixed-length semantic injection, enabling precise motion control from text prompts and stronger generalization to unseen text inputs.
Source: arXiv (Cornell University)
Authors: Chang, Che-Jui, Liu, Qingze Tony, Zhou, Honglu, Pavlovic, Vladimir, Kapadia, Mubbasir
Institution: Rutgers, The State University of New Jersey
Language: English
Topic: Human Motion and Animation
Subfield: Control and Systems Engineering
Field: Engineering
Domain: Physical Sciences
Open Access status: green
Awards: NSF Engineering Research Center for Smart Streetscapes (CS3), CHS: Medium: Data Driven Biomechanically Accurate Modeling of Human Gait on Unconstrained Terrain, EAGER: SAI: Cognitive Models of Human Social Wayfinding for the Redesign of Public Spaces, G7157213963, Collaborative Research: RI: Medium: Learning Joint Crowd-Space Embeddings for Cross-Modal Crowd Behavior Prediction, The Role of Agency and Interactivity in Embodied Conversational Agents to Support Teacher Training in High Poverty Schools, Collaborative Research: RI: Medium: Learning Joint Crowd-Space Embeddings for Cross-Modal Crowd Behavior Prediction
OpenAlex ID: W4407185619