Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis

Abstract

Neural waveform models have demonstrated better performance than conventional vocoders for statistical parametric speech synthesis. One of the best models, called WaveNet, uses an autoregressive (AR) approach to model the distribution of waveform sampling points, but it has to generate a waveform in a time-consuming sequential manner. Some new models that use inverse-autoregressive flow (IAF) can generate a whole waveform in a one-shot manner but require either a larger amount of training time or a complicated model architecture plus a blend of training criteria.As an alternative to AR and IAF-based frameworks, we propose a neural source-filter (NSF) waveform modeling framework that is straightforward to train and fast to generate waveforms. This framework requires three components to generate waveforms: a source module that generates a sine-based signal as excitation, a non-AR dilated-convolution-based filter module that transforms the excitation into a waveform, and a conditional module that pre-processes the input acoustic features for the source and filter modules. This framework minimizes spectralamplitude distances for model training, which can be efficiently implemented using short-time Fourier transform routines. As an initial NSF study, we designed three NSF models under the proposed framework and compared them with WaveNet using our deep learning toolkit. It was demonstrated that the NSF models generated waveforms at least 100 times faster than our WaveNet-vocoder, and the quality of the synthetic speech from the best NSF model was comparable to that from WaveNet on a single-speaker Japanese speech corpus.

Quasi-Periodic Parallel WaveGAN: A Non-Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

“…Motivated by the development of deep NNs, NNbased excitation generation models with the AR mechanism such as LPCNet [11] and the non-AR mechanism such as GlotGAN [17], [18] and GELP [19] have been proposed to improve the generated speech quality. Moreover, the authors of [31] and [32] also proposed a neural source-filter (NSF) network to model the source-filter generative framework with an advanced neural filter.…”

“…For instance, without explicitly modeling the excitation signals as conventional source-filter models, it is difficult for WN to generate speech with accurate pitches outside the fundamental frequency (F 0 ) range of training data when conditioned on the scaled F 0 feature [33], [34]. However, using carefully designed mixed periodic and aperiodic inputs and source-filter-like architectures, the authors of [30]- [32] proposed different NN-based models attaining pitch control- lability. In our previous works [33], [34], we also proposed a quasi-periodic WN (QPNet), which has a conventionalvocoding-like framework while using a unified network without the requirement of specific mixed inputs.…”

“…To address this problem, we extend the AR PDCNN of QPNet to a non-AR PDCNN and apply the quasi-periodic (QP) structure to parallel WaveGAN (PWG) [25], which is a compact non-AR model with a WN-like network architecture consisting of stacked DCNN layers. The proposed QPPWG speech generation model [46] attains pitch controllability using a simple pitch-dependent architecture without the requirement of specific mixed periodic and aperiodic inputs as in [30]- [32]. Although QPPWG greatly improved the pitch controllability of PWG, the behind mechanisms of the QP structure in the non-AR model, the characteristic of each component, and the effective architectures are remained to be explored.…”

Measuring context dependency in birdsong using artificial neural networks

“…the Mel-frequency cepstral coefficients, MFCCs, are used in human speech analysis; Chung et al, 2016; Chorowski et al, 2019; Tjandra et al, 2019), and/or adopt more anatomically/bio-acoustically realistic articulatory systems for the decoder module (cf. Wang et al, 2020, implemented the source-filter model of vocalization based on an artificial neural network). Such Embodied VAEs would allow constructive investigation of vocal learning beyond mere acoustic analysis.…”

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning

“…We note that NSF is straightforward to train and fast to generate waveform. It is reported 100 times faster than WaveNet vocoder, and yet achieving comparable voice quality on a large speech corpus [108].…”