The Huya Multi-Speaker and Multi-Style Speech Synthesis System for M2voc Challenge 2020

Abstract

Text-to-speech systems now can generate speech that is hard to distinguish from human speech. In this paper, we propose the Huya multi-speaker and multi-style speech synthesis system which is based on DurIAN and HiFi-GAN to generate high-fidelity speech even under low-resource condition. We use the fine-grained linguistic representation which leverages the similarity in pronunciation between different languages and promotes the speech quality of code-switch speech synthesis. Our TTS system uses the HiFi-GAN as the neural vocoder which has higher synthesis stability for unseen speakers and can generate higher quality speech with noisy training data than WaveRNN in the challenge tasks. The model is trained on the datasets released by the organizer as well as CMU-ARCTIC, AIShell-1 and THCHS-30 as the external datasets and the results were evaluated by the organizer. We participated in all four tracks and three of them entered high score lists. The evaluation results show that our system outperforms the majority of all participating teams.

The Multi-speaker Multi-style Voice Cloning Challenge 2021

“…For the joint optimization solution, the straightforward method is speaker look-up table, in which each speaker code is encoded in a trainable embedding table, jointly trained with acoustic model via backpropagation. T06 and T24 [39] have used this solution in their systems.…”