Error Reduction Network for DBLSTM-based Voice Conversion

Abstract

So far, many of the deep learning approaches for voice conversion produce good quality speech by using a large amount of training data. This paper presents a Deep Bidirectional Long Short-Term Memory (DBLSTM) based voice conversion framework that can work with a limited amount of training data. We propose to implement a DBLSTM based average model that is trained with data from many speakers. Then, we propose to perform adaptation with a limited amount of target data. Last but not least, we propose an error reduction network that can improve the voice conversion quality even further. The proposed framework is motivated by three observations. Firstly, DBLSTM can achieve a remarkable voice conversion by considering the long-term dependencies of the speech utterance. Secondly, DBLSTM based average model can be easily adapted with a small amount of data, to achieve a speech that sounds closer to the target. Thirdly, an error reduction network can be trained with a small amount of training data, and can improve the conversion quality effectively. The experiments show that the proposed voice conversion framework is flexible to work with limited training data and outperforms the traditional frameworks in both objective and subjective evaluations.

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning

“…Liu et al propose to use PPG for emotional voice conversion [227]. Zhang et al also show that the average model framework can benefit from a small amount of parallel training data using an error reduction network [228]. 4) Disentangling speaker from linguistic content: In the context of voice conversion, speech can be considered as a composition of speaker voice identity and linguistic content.…”

Pretraining Techniques for Sequence-to-Sequence Voice Conversion

“…In a recent VC review paper [48], it was shown that a sufficient amount of efforts has been dedicated to transferring knowledge from ASR and TTS to improving various aspects of VC, regardless of using a seq2seq model or not. The PPGbased methods [49]- [53] and the Parratron system described in Section II-A facilitated nonparallel, any-to-one VC by utilizing ASR and TTS modules, respectively. Another line of work show that training an integrated system capable of performing either TTS or VC can boost individual performances [29], [54].…”

An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning

“…Liu et al proposes to use PPG for emotional voice conversion [214]. Zhang et al also shows that the average model framework can benefit from a small amount of parallel training data using an error reduction network [215].…”