Work

Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations

Year: 2020

Type: preprint

Abstract: We present a novel approach to any-to-one (A2O) voice conversion (VC) in a sequence-to-sequence (seq2seq) framework. A2O VC aims to convert any speaker, including those unseen during training, to a fixed target speaker. We utilize vq-wav2vec (VQW2V), a discretized self-supervised speech representation that was learned from massive unlabeled data, which is assumed to be speaker-independent and well corresponds to underlying linguistic contents. Given a training dataset of the target speaker, we extract VQW2V and acoustic features to estimate a seq2seq mapping function from the former to the latter. With the help of a pretraining method and a newly designed postprocessing technique, our model can be generalized to only 5 min of data, even outperforming the same model trained with parallel data.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2010.12231

Authors: Huang, Wen-Chin, Wu, Yi-Chiao, Hayashi, Tomoki, Toda, Tomoki

Institution: Nagoya University

Language: English

Topic: Speech Recognition and Synthesis

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: Japan Society for the Promotion of Science, Core Research for Evolutional Science and Technology

Awards: Next generation speech translation research, G6718509927, G7789893098

OpenAlex ID: W3093848788

Cites: 26

Cited by:

Related to: 10