Video Speech Synthesis a Flexible Integrated System for Analysing and Synthesising Speech Production in the Visual Domain
Year: 2024
Type: conference-paper
Abstract: Earlier papers (l,Z) have suggested the potential of a reproducible, controllable and well-specified visual stimulus for analytical investigations of speech perception by normal hearing subjects in the bimodal, audio-visual domain.The output of such a 'video speech synthesiser' would be an animated real time graphical display of the essential facial topography during Speech utterance production.This would be synchronised with the output of an audio speech synthesiser to generate the audio-visual stimulus.The kinds of experiments which may be undertaken depend upon the sophistication of the video synthesiser.For example, discrimination experiments to explore the nature of categorical per-:eption in certain bimodal phonetic spaces such as the /ma,ba,pa/ continuum,(3) may be carried out using a simple terminal-analogue version of the video speth synthesiser.More complex experiments using purely visual and coherent or conflicting audio-visual stimuli (h) require a more detailed control of the vishal articulatory movements and timings.A sufficiently well-specified video synthesiser might ultimately be used to help rehabilitate the hearing-impaired by providing a device in which small but significant variations could be con trolled and exaggerated during a period of perceptual training.The use of a computer offers the most flexible approach to the implementation of a video speech synthesiser.Implementation of a simple terminal-analogue model of facial articulation has been described previously (I), in which the
DOI: 10.25144/23280
Author: NM BROOKE
Institution: Lancaster University
Language: English
Subfield: Experimental and Cognitive Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: bronze
OpenAlex ID: W4400950387