Year: 2018
Type: conference-paper
Abstract: The growing use of virtual humans demands generating increasingly realistic behavior for them while minimizing cost and time. Gestures are a key ingredient for realistic and engaging virtual agents and consequently automatized gesture generation has been a popular area of research. So far, good gesture generation has relied on explicit formulation of if-then rules and probabilistic modelling of annotated features. Machine learning approaches have yielded only marginal success, indicating a high complexity of the speech-to-motion learning task. In this work, we explore the use of transfer learning using previous motion modelling research to improve learning outcomes for gesture generation from speech. We use a recurrent network with an encoder-decoder structure that takes in prosodic speech features and generates a short sequence of gesture motion. We pre-train the network with a motion modelling task. We recorded a large multimodal database of conversational speech for the purpose of this work.
Authors: Ylva Ferstl, Rachel McDonnell
Institution: Trinity College Dublin
Language: English
Topic: Human Motion and Animation
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Funder: Science Foundation Ireland
Award: G2693890202
OpenAlex ID: W2901285216
