Work

Deep Learning Models for Ukrainian Text to Speech Synthesis

Year: 2023
Type: conference-paper
Abstract: The developed technology performs text to speech generation for the Ukrainian language. Implemented technology at first performs mel-spectrogram prediction using character sequence as an input and then based on composed acoustic features, the final audio signal is reconstructed. As a part of research, an overview and comparison of existing text to speech technologies for the Ukrainian language was conducted. For both stages deep learning architecture models were used – Tacotron2 for mel-spectrogram generation and Parallel WaveGan for voice audio generation. The research also contains a comparison with other architectures and vocoders, such as FastSpeech2 and WaveNet. Also, a grapheme-to-phoneme translation was improved in order to get better pronunciation quality for Ukrainian language. Experiments contain the usage of pre-trained models on English and other languages, in order to leverage transfer learning techniques. Research also contains a review and analysis of existing open-source datasets for the Ukrainian language. A mean opinion score metric was proposed and used in order to evaluate the final solution, with a statistically significant variety of generated testing samples and participants, based on defined criteria of the metric grade. Experiments show dependency on the training vocabulary of the dataset and further development implies augmentation of the dataset with different topics. The best configuration for all deep learning models was found based on training and testing results are also shown in the research, with hardware requirements needed to reproduce the training. Experiments also show the satisfying quality of generated voice, trained on a specially collected and processed dataset of the single male voice. Testing experiments 20 people participated, while each person evaluated about 100 generated sound samples of length from 40 to 60 seconds score of 4.02 of mean opinion score metric was achieved.
Language: English
Open Access status: bronze
OpenAlex ID: W7166320384
FWCI: 0.111
Cites: 15
Cited by: 2
Related to: 10