Waveform Generation for Text-to-speech Synthesis Using Pitch-synchronous Multi-scale Generative Adversarial Networks

Abstract

The state-of-the-art in text-to-speech synthesis has recently improved considerably due to novel neural waveform generation methods, such as WaveNet. However, these methods suffer from their slow sequential inference process, while their parallel versions are difficult to train and even more expensive computationally. Meanwhile, generative adversarial networks (GANs) have achieved impressive results in image generation and are making their way into audio applications; parallel inference is among their lucrative properties. By adopting recent advances in GAN training techniques, this investigation studies waveform generation for TTS in two domains (speech signal and glottal excitation). Listening test results show that while direct waveform generation with GAN is still far behind WaveNet, a GAN-based glottal excitation model can achieve quality and voice similarity on par with a WaveNet vocoder.

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

“…Our previous works [18,19] show that the capacity of AR vocoder is highly related to the length of the receptive field, and we argue that the No-AR vocoder has a similar tendency. Specifically, the receptive field length of PWG_30 is 6139 (2 0 +...+2 9 =1023 with three cycles and two sides plus one) and that of PWG_20 is 4093.…”

“…1, the conventional parametric vocoders generate speech samples in an AR manner such as LPC vocoders [47], [48] and mel-generalized cepstrum (MGC) vocoders [49], [50] or in a non-AR manner such as STRAIGHT [4] and WORLD [5]. Motivated by the development of deep NNs, NNbased excitation generation models with the AR mechanism such as LPCNet [11] and the non-AR mechanism such as GlotGAN [17], [18] and GELP [19] have been proposed to improve the generated speech quality. Moreover, the authors of [31] and [32] also proposed a neural source-filter (NSF) network to model the source-filter generative framework with an advanced neural filter.…”

“…Among the models with speech knowledge, GlotGAN [17], [18] achieved early success in generating glottal excitation signals, but it suffered severe speech quality degradation when directly applied to raw speech waveform generation. GELP [19] has been proposed to improve the glottal generator by using short-time Fourier transform (STFT)-based regression loss and the adversarial loss of the final generated waveforms.…”

GELP: GAN-Excited Linear Prediction for Speech Synthesis from Mel-Spectrogram

“…The current study adapts the loss functions from our previous work [22]. However, now we can define the losses directly in speech domain, since the LP synthesis filter and overlap-add are integrated to the computation graph.…”

“…In this paper, we combine a MFCC-based envelope model [21] with recent GAN training insights [22]. Furthermore, we use neural net architectures that operate directly on raw audio, and integrate a parallel inference capable LP synthesis filter into the computation graph.…”