Investigating Very Deep Highway Networks for Parametric Speech Synthesis

Abstract

The depth of the neural network is a vital factor that affects its performance. Recently a new architecture called highway network was proposed. This network facilitates the training process of a very deep neural network by using gate units to control a information highway over the conventional hidden layer. For the speech synthesis task, we investigate the performance of highway networks with up to 40 hidden layers. The results suggest that a highway network with 14 non-linear transformation layers is the best choice on our speech corpus and this highway network achieves better performance than a feed-forward network with 14 hidden layers. On the basis of these results, we further investigate a multi-stream highway network where separate highway networks are used to predict different kinds of acoustic features such as the spectral and F0 features. Results of the experiments suggest that the multi-stream highway network can achieve better objective results than the single network that predicts all the acoustic features. Analysis on the output of highway gate units also supports the assumption for the multi-stream network that different hidden representation may be necessary to predict spectral and F0 features.

A Vector Quantized Variational Autoencoder (VQ-VAE) Autoregressive Neural $F_0$ Model for Statistical Parametric Speech Synthesis

“…Recently, researchers have used NNs as alternatives to HMMs to jointly model the F 0 and other spectral features [22], [23], [24]. Some researchers use NNs exclusively for F 0 modeling [25], [12], which may be reasonable because it was recently found that NNs may prioritize the spectral features over the F 0 [26]. In fact, many NN-based F 0 models have been proposed before the advent of SPSS-based TTS systems [27], [28], [29].…”

Voice Conversion Using Input-to-Output Highway Networks

“…Highway networks [8], [9] are weighted skip-connections between layers, and they often connect hidden layers. Given that the input and output are often in the same domain (e.g., cepstrum) in VC, we propose a VC using highway networks connected from the input to output as follows:…”

Improving Post-Filtering of Artificial Speech Using Pre-Trained LSTM Neural Networks

“…It is well known that the generated sequences of parameters from the HMMs are temporally smoothed, producing perceptual differences between synthetic and natural speech. There have been several attempts to improve the quality of synthesized speech, based on Deep Learning approaches: The first main approach is to substitute the HMM for deep neural networks (DNN) [7] [8] [9] [10], learning the map between linguistic specification directly to speech parameters. The second approach is to apply post-filters for the parameters generated by the HMMs [11] [12] [13].…”