Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments

Abstract

Voice Conversion (VC) is a technique that aims to transform the non-linguistic information of a source utterance to change the perceived identity of the speaker. While there is a rich literature on VC, most proposed methods are trained and evaluated on clean speech recordings. However, many acoustic environments are noisy and reverberant, severely restricting the applicability of popular VC methods to such scenarios. To address this limitation, we propose Voicy, a new VC framework particularly tailored for noisy speech. Our method, which is inspired by the de-noising auto-encoders framework, is comprised of four encoders (speaker, content, phonetic and acoustic-ASR) and one decoder. Importantly, Voicy is capable of performing non-parallel zero-shot VC, an important requirement for any VC system that needs to work on speakers not seen during training. We have validated our approach using a noisy reverberant version of the LibriSpeech dataset. Experimental results show that Voicy outperforms other tested VC techniques in terms of naturalness and target speaker similarity in noisy reverberant environments.

Learning Noise-independent Speech Representation for High-quality Voice Conversion for Noisy Target Speakers

“…Besides, an intuitive way is to leverage a speech enhancement module to remove noise before training [19], but it will inevitably affect the quality of generated speech because the extra speech distortion after speech enhancement will propagate to the acoustic model as well as the vocoder [20]. Motivated by denoising auto-encoder [21], denoising training strategy is also applied in several robust VC systems [22,19]. It has been reported that the denoising method leads to worse naturalness than the speech enhancement method, while the speech enhancement-based method has lower speaker similarity scores than the denoising approach [19].…”