An autoregressive recurrent mixture density network for parametric speech synthesis

Wang, Xin; Takaki, Shinji; Yamagishi, Junichi

doi:10.1109/icassp.2017.7953087

Cited by 43 publications

(32 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, the ideas presented here still apply while the acoustic features are used to control generative neural models. The shortcomings in the acoustic model can be addressed in the future with the use of more advanced models, such as mixture density network LSTMs [8].…”

Section: Discussionmentioning

confidence: 99%

“…In conjunction, the two major issues in SPSS, over-smoothness of the generated acoustic parameters, and "buzzy" synthetic sound quality have been attributed to the acoustic model and vocoder, respectively. Recent efforts have improved the acoustic model performance resulting in more natural synthetic parameter trajectories using, for example, autoregressive mixture density networks [8], or generative adversarial network-based post-filtering [9]. However, the performance of these systems is still upper-bounded by the analysis-synthesis quality of the vocoder.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reducing Mismatch in Training of DNN-Based Glottal Excitation Models in a Statistical Parametric Text-to-Speech System

Juvela¹,

Bollepalli²,

Yamagishi³

et al. 2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

(2017). Reducing mismatch in training of DNN-based glottal excitation models in a statistical parametric text-to-speech system. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (Vol. 2017-August, pp. 1368-1372 AbstractNeural network-based models that generate glottal excitation waveforms from acoustic features have been found to give improved quality in statistical parametric speech synthesis. Until now, however, these models have been trained separately from the acoustic model. This creates mismatch between training and synthesis, as the synthesized acoustic features used for the excitation model input differ from the original inputs, with which the model was trained on. Furthermore, due to the errors in predicting the vocal tract filter, the original excitation waveforms do not provide perfect reconstruction of the speech waveform even if predicted without error. To address these issues and to make the excitation model more robust against errors in acoustic modeling, this paper proposes two modifications to the excitation model training scheme. First, the excitation model is trained in a connected manner, with inputs generated by the acoustic model. Second, the target glottal waveforms are re-estimated by performing glottal inverse filtering with the predicted vocal tract filters. The results show that both of these modifications improve performance measured in MSE and MFCC distortion, and slightly improve the subjective quality of the synthetic speech.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Reducing Mismatch in Training of DNN-Based Glottal Excitation Models in a Statistical Parametric Text-to-Speech System

Juvela¹,

Bollepalli²,

Yamagishi³

et al. 2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the acoustic models that predict acoustic features from the linguistic features, we used shallow and deep neural AR models [9], [5] to generate the MGCs and F0, respectively. The recipes for training these acoustic models were the same as those in another of our previous study [54].…”

Section: B Model Configurationsmentioning

confidence: 99%

Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis

Wang

Takaki

Yamagishi

2020

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Neural waveform models have demonstrated better performance than conventional vocoders for statistical parametric speech synthesis. One of the best models, called WaveNet, uses an autoregressive (AR) approach to model the distribution of waveform sampling points, but it has to generate a waveform in a time-consuming sequential manner. Some new models that use inverse-autoregressive flow (IAF) can generate a whole waveform in a one-shot manner but require either a larger amount of training time or a complicated model architecture plus a blend of training criteria.As an alternative to AR and IAF-based frameworks, we propose a neural source-filter (NSF) waveform modeling framework that is straightforward to train and fast to generate waveforms. This framework requires three components to generate waveforms: a source module that generates a sine-based signal as excitation, a non-AR dilated-convolution-based filter module that transforms the excitation into a waveform, and a conditional module that pre-processes the input acoustic features for the source and filter modules. This framework minimizes spectralamplitude distances for model training, which can be efficiently implemented using short-time Fourier transform routines. As an initial NSF study, we designed three NSF models under the proposed framework and compared them with WaveNet using our deep learning toolkit. It was demonstrated that the NSF models generated waveforms at least 100 times faster than our WaveNet-vocoder, and the quality of the synthetic speech from the best NSF model was comparable to that from WaveNet on a single-speaker Japanese speech corpus.

show abstract

“…The phase-based weighting matrix is introduced to reconstruct the glottal waveform through weighting the CW component, as shown in equation (4). The equation (3) shows that the weighting matrix function F (•) is a complicated non-linear function of the phase vector Φ(n). Thus, we use two fully connected layers followed by different nonlinear activations to simulate the phase-based weighting function F (•).…”

Section: Hybrid Neural Networkmentioning

confidence: 99%

“…The quality of SPSS system is mainly affected by three factors: vocoder, acoustic model accuracy and over-smoothing [1]. Recently, deep neural networks, especially the sequential neural network [2,3], has improved the model accuracy and alleviate the over-smoothing issue. Despite those improvements, the synthetic speech quality is still limited by the vocoder, which causes the gap between SPSS and unit concatenation approaches.…”

Section: Introductionmentioning

confidence: 99%

A New Glottal Neural Vocoder for Speech Synthesis

2018

View full text Add to dashboard Cite

Direct modeling of waveform generation for speech synthesis, e.g. WaveNet, has made significant progress on improving the naturalness and clarity of TTS. Such deep neural network-based models can generate highly realistic speech but at high computational and memory costs. We propose here a novel neural glottal vocoder which tends to bridge the gap between the traditional parametric vocoder and end-to-end speech sample generation. In the analysis, speech signals are decomposed into corresponding glottal source signals and vocal tract filters by the glottal inverse filtering. Glottal pulses are parameterized into energy, DCT coefficients (shape) and phase. The phase trajectory of successive glottal pulses is rendered with a trainable weighting matrix to keep a smooth pitch synchronous phase trajectory. We design a hybrid, i.e., both feed-forward and recurrent, neural network to reconstruct the glottal waveform including the optimized weighting matrix. Speech is then synthesized by filtering the generated glottal waveform with the vocal tract filter. The new neural glottal vocoder can generate high-quality speech with efficient computations. Subjective tests show that it gets an MOS score of 4.12 and 75% preference over the conventional glottal vocoder with a perceived quality comparable to WaveNet and natural recording in analysis-by-synthesis.

show abstract

An autoregressive recurrent mixture density network for parametric speech synthesis

Cited by 43 publications

References 18 publications

Reducing Mismatch in Training of DNN-Based Glottal Excitation Models in a Statistical Parametric Text-to-Speech System

Reducing Mismatch in Training of DNN-Based Glottal Excitation Models in a Statistical Parametric Text-to-Speech System

Neural Source-Filter Waveform Models for Statistical Parametric Speech Synthesis

A New Glottal Neural Vocoder for Speech Synthesis

Contact Info

Product

Resources

About