Year: 2019
Type: dissertation
Abstract: This thesis investigates a number of new approaches for visual speech \nsynthesis using data-driven methods to implement a talking face. \nThe main contributions in this thesis are the following. The accuracy \nof shared Gaussian process latent variable model (SGPLVM) \nbuilt using the active appearance model (AAM) and relative spectral \ntransform-perceptual linear prediction (RASTAPLP) features is improved \nby employing a more accurate AAM. This is the first study \nto report that using a more accurate AAM improves the accuracy of \nSGPLVM. Objective evaluation via reconstruction error is performed \nto compare the proposed approach against previously existing methods. \nIn addition, it is shown experimentally that the accuracy of AAM \ncan be improved by using a larger number of landmarks and/or larger \nnumber of samples in the training data. \nThe second research contribution is a new method for visual speech \nsynthesis utilising a fully Bayesian method namely the manifold relevance \ndetermination (MRD) for modelling dynamical systems through \nprobabilistic non-linear dimensionality reduction. This is the first time \nMRD was used in the context of generating talking faces from the \ninput speech signal. The expressive power of this model is in the ability \nto consider non-linear mappings between audio and visual features \nwithin a Bayesian approach. An efficient latent space has been learnt \niii \nAbstract iv \nusing a fully Bayesian latent representation relying on conditional nonlinear \nindependence framework. In the SGPLVM the structure of the \nlatent space cannot be automatically estimated because of using a maximum \nlikelihood formulation. In contrast to SGPLVM the Bayesian approaches \nallow the automatic determination of the dimensionality of the \nlatent spaces. The proposed method compares favourably against several \nother state-of-the-art methods for visual speech generation, which \nis shown in quantitative and qualitative evaluation on two different \ndatasets. \nFinally, the possibility of incremental learning of AAM for inclusion \nin the proposed MRD approach for visual speech generation is \ninvestigated. The quantitative results demonstrate that using MRD in \nconjunction with incremental AAMs produces only slightly less accurate \nresults than using batch methods. These results support a way of \ntraining this kind of models on computers with limited resources, for \nexample in mobile computing. \nOverall, this thesis proposes several improvements to the current \nstate-of-the-art in generating talking faces from speech signal leading \nto perceptually more convincing results.
Author: Samia Albasri
Language: English
Topic: Speech and Audio Processing
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W2973149904