A New Glottal Neural Vocoder for Speech Synthesis

Abstract

Direct modeling of waveform generation for speech synthesis, e.g. WaveNet, has made significant progress on improving the naturalness and clarity of TTS. Such deep neural network-based models can generate highly realistic speech but at high computational and memory costs. We propose here a novel neural glottal vocoder which tends to bridge the gap between the traditional parametric vocoder and end-to-end speech sample generation. In the analysis, speech signals are decomposed into corresponding glottal source signals and vocal tract filters by the glottal inverse filtering. Glottal pulses are parameterized into energy, DCT coefficients (shape) and phase. The phase trajectory of successive glottal pulses is rendered with a trainable weighting matrix to keep a smooth pitch synchronous phase trajectory. We design a hybrid, i.e., both feed-forward and recurrent, neural network to reconstruct the glottal waveform including the optimized weighting matrix. Speech is then synthesized by filtering the generated glottal waveform with the vocal tract filter. The new neural glottal vocoder can generate high-quality speech with efficient computations. Subjective tests show that it gets an MOS score of 4.12 and 75% preference over the conventional glottal vocoder with a perceived quality comparable to WaveNet and natural recording in analysis-by-synthesis.

Waveform Generation for Text-to-speech Synthesis Using Pitch-synchronous Multi-scale Generative Adversarial Networks

“…We use REAPER for GCI and pitch estimation [31]. Similar phase-locked representations have been successfully applied not only in our previous work ( [30,24,25]), but also in [23,32]. Furthermore, using GCIs to center waveforms in a window can be seen analogous to using facial landmarks to center images, as done in the highly successful CelebA-HQ dataset [10].…”

“…Early approaches used a point-wise least squares loss in time-domain [20,21], and while this captures the gross waveshape well, the produced output is essentially a conditional average and lacks in stochastic high frequency contents (due to the averaging). The missing stochastic component can be recreated using signal processing techniques for aperiodicity modification, resulting in high quality synthetic speech [22,23], but this involves making signal model assumptions that may not hold generally. Further efforts have been made to model the stochastic part directly using GANs [24] or WaveNet ("GlotNet") [21].…”

GlotNet—A Raw Waveform Model for the Glottal Excitation in Statistical Parametric Speech Synthesis

“…For the separate frame-rate conditioning model, the parameter count is similar to the WaveNets, but the FLOPS count is significantly lower due to the operation at 200Hz rate (as opposed to 16kHz). For comparison, [19] reported their WaveNet running at 209G FLOPS, although the paper lacks information on the exact model configuration and how the FLOPS count was estimated. For further comparison, our estimation method gives 1008G FLOPS for the model configuration proposed in [16] (30 dilation layers, 256 residual channels, 2048 skip channels).…”

“…Additionally, acoustic conditioning enables the same models to be used in other (non-TTS) waveform generation tasks, such as voice conversion [18]. Despite their recent success, WaveNet models suffer from their need for substantial amounts of training data and large model sizes, making them expensive to train and use [19]. Furthermore, the autoregressive nature of WaveNets makes inference inherently slow.…”

“…The first TTS studies [28], [29] utilizing the DNN-based glottal excitation generation used a GIF technique proposed in [30]. More recently, improved TTS quality has been obtained by using a more accurate GIF method [31], jointly trained waveform and noise models [32], or alternative waveform representations [19]. All these previous studies are based on a fixed length pitchsynchronous waveform representation that enables the use of simple, fully connected networks for fast inference.…”