Loss Function Considering Temporal Sequence for Feed-Forward Neural Network–Fundamental Frequency Case

Abstract

This paper describes a novel loss function for training feedforward neural networks (FFNNs), which can generate smooth speech parameter sequences without post-processing. In statistical parametric speech synthesis based on deep neural networks (DNNs), maximum likelihood parameter generation (MLPG) or recurrent neural networks (RNNs) are generally used to generate smooth speech parameter sequences. However, because the MLPG process requires utterance-level processing, it is not suitable for speech synthesis requiring low latency. Furthermore, networks such as long short-term memory RNNs (LSTM-RNNs) have high computational costs. As RNNs are not recommended in limited computational resource situations, we look at employing FFNNs as an alternative. One limitation of FFNNs is that they train to ignore relationships between speech parameters in adjacent frames. To overcome this limitation and generate smooth speech parameter sequences from FFNNs alone, we propose a novel loss function that uses long-and short-term features from speech parameters. We evaluated the proposed loss function with a focus on the fundamental frequency (F0) at found that, using the proposed loss function, an FFNN-only approach can generate F0 contours that are perceptually equal to or better in terms of naturalness than those generated by MLPG or LSTM-RNNs.

Loss Function Considering Multiple Attributes of a Temporal Sequence for Feed-Forward Neural Networks

“…In this paper, we propose a new training method that enables FFNNs to acquire parameters captured with a natural temporal structure for these sequences by Copyright c 2020 The Institute of Electronics, Information and Communication Engineers backpropagating the errors of multiple attributes from the temporal sequence through the loss function. As a preliminary study, the effectiveness of the proposed loss function in modeling the F0 sequence was investigated [11]. In this paper, the effectiveness of the proposed loss function in modeling the spectral feature sequence in addition to the F0 sequence is discussed.…”