Parallel Wavegan: A Fast Waveform Generation Model Based on Generative Adversarial Networks with Multi-Resolution Spectrogram

Abstract

We propose Parallel WaveGAN, a distillation-free, fast, and smallfootprint waveform generation method using a generative adversarial network. In the proposed method, a non-autoregressive WaveNet is trained by jointly optimizing multi-resolution spectrogram and adversarial loss functions, which can effectively capture the time-frequency distribution of the realistic speech waveform. As our method does not require density distillation used in the conventional teacher-student framework, the entire model can be easily trained even with a small number of parameters. In particular, the proposed Parallel WaveGAN has only 1.44 M parameters and can generate 24 kHz speech waveform 28.68 times faster than real-time on a single GPU environment. Perceptual listening test results verify that our proposed method achieves 4.16 mean opinion score within a Transformer-based text-to-speech framework, which is comparative to the best distillation-based Parallel WaveNet system.

Investigation of training data size for real-time neural vocoders on CPUs

“…Because of the autoregressive architecture, the WaveNet vocoder has a problem with the slow inference speed and thus is limited in its application. To solve this problem, various neural vocoder models have been proposed, and some can synthesize speech waveforms in real-time, even in a restricted environment with mobile CPUs [5,6]. These models are mostly trained on large sets of training data of more than 10 hours.…”

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning

“…More recently, Parallel WaveGAN [109] has also been proposed to generate high-quality voice using a generative adversarial network. Parallel WaveGAN is a distillationfree and fast waveform generation method, where a nonautoregressive WaveNet is trained by jointly optimizing multi-resolution spectrogram and adversarial loss functions.…”

Improving LPCNET-Based Text-to-Speech with Linear Prediction-Structured Mixture Density Network

“…where λ is the hyper-parameter that controls the balance between L nll and L pl . As the STFT loss function is able to effectively capture the time-frequency distribution of the realistic speech waveform, especially for the harmonic components [12,13], the entire training process becomes more efficient.…”