WG-WaveNet: Real-Time High-Fidelity Speech Synthesis without GPU

Abstract

In this paper, we propose WG-WaveNet, a fast, lightweight, and high-quality waveform generation model. WG-WaveNet is composed of a compact flow-based model and a post-filter. The two components are jointly trained by maximizing the likelihood of the training data and optimizing loss functions on the frequency domains. As we design a flow-based model that is heavily compressed, the proposed model requires much less computational resources compared to other waveform generation models during both training and inference time; even though the model is highly compressed, the post-filter maintains the quality of generated waveform. Our PyTorch implementation can be trained using less than 8 GB GPU memory and generates audio samples at a rate of more than 5000 kHz on an NVIDIA 1080Ti GPU. Furthermore, even if synthesizing on a CPU, we show that the proposed method is capable of generating 44.1 kHz speech waveform 1.2 times faster than real-time. Experiments also show that the quality of generated audio is comparable to those of other methods. Audio samples are publicly available online.

Universal Adaptor: Converting Mel-Spectrograms Between Different Configurations for Speech Synthesis

“…Neural text-to-speech (TTS) has achieved remarkable audio qualities recently [1,2]. Most recent TTS systems comprise two cascaded separated modules: synthesizer [1][2][3][4] and vocoder [5][6][7][8][9][10][11][12][13]. In the first module, the synthesizer takes text as input and outputs an audio mid-representation.…”

Empirical Evaluation of Deep Learning Model Compression Techniques on the WaveNet Vocoder

“…As a result, TTS research has focused on finding alternative vocoder architectures such as Parallel-WaveNet (Oord et al 2018), WaveRNN (Kalchbrenner et al 2018), Clar-iNet (Ping, Peng, and Chen 2018) and WaveGlow (Prenger, Valle, and Catanzaro 2019) that achieve higher performance when deployed on existing hardware. There is a degree of ambiguity as to the highest quality vocoder as audio quality evaluation is subjective but all authors agree that WaveNet produces at least as good if not higher quality audio than the more recent approaches (Kim et al 2018;Oord et al 2018;Prenger, Valle, and Catanzaro 2019;Tian et al 2020;Hsu and Lee 2020).…”