A Regression Model of Recurrent Deep Neural Networks for Noise Robust Estimation of the Fundamental Frequency Contour of Speech

Abstract

The fundamental frequency (F 0) contour of speech is a key aspect to represent speech prosody that finds use in speech and spoken language analysis such as voice conversion and speech synthesis as well as speaker and language identification. This work proposes new methods to estimate the F 0 contour of speech using deep neural networks (DNNs) and recurrent neural networks (RNNs). They are trained using supervised learning with the ground truth of F 0 contours. The latest prior research addresses this problem first as a frame-byframe-classification problem followed by sequence tracking using deep neural network hidden Markov model (DNN-HMM) hybrid architecture. This study, however, tackles the problem as a regression problem instead, in order to obtain F 0 contours with higher frequency resolution from clean and noisy speech. Experiments using PTDB-TUG corpus contaminated with additive noise (NOISEX-92) show the proposed method improves gross pitch error (GPE) by more than 25 % at signal-tonoise ratios (SNRs) between-10 dB and +10 dB as compared with one of the most noise-robust F 0 trackers, PEFAC. Furthermore, the performance on fine pitch error (FPE) is improved by approximately 20 % against a state-of-the-art DNN-HMMbased approach.

Waveform to Single Sinusoid Regression to Estimate the F0 Contour from Noisy Speech Using Recurrent Deep Neural Networks

“…This work is an extension of our recent preliminary study [22]. In that study, we have successfully employed an RNN regression model, which maps spectral sequence directly onto F 0 values, to tackle the disadvantage in existing classification approaches mentioned above.…”

Statistical Regression Models for Noise Robust F0 Estimation Using Recurrent Deep Neural Networks

“…This work is an extension of our two recent, preliminary studies [15], [24], in which we introduced two types of RNN regression models for F0 estimation. Our first approach [15] uses the spectral magnitude to predict the (scalar) F0 value directly, while our second model [24] first maps the raw waveform input into a sinusoid (vector) oscillating with F0. F0 can easily be extracted from this sinusoid using standard signal processing operations (here, through autocorrelation).…”

“…PEFAC applies matched filters and autocorrelation in the log-frequency domain to achieve noise robustness. Nevertheless, the accuracy remains unsatisfactory in severe noise conditions, such as signal-to-noise ratios (SNRs) less than or equal to 0 dB [13], [15].…”

“…The classification approach to F0 tracking, however, is not without problems. Because of the quantized presentation, the F0 contour becomes less natural [15], [24]. There is a tradeoff between the tracking accuracy and naturalness of F0: while a larger number of quantization levels leads to a closer approximation to a continuous F0 contour, classification errors (i.e., prediction of the wrong F0 state) are also more likely to occur.…”

“…Nonetheless, as a supervised, datadriven approach, we do require a training corpus with groundtruth F0 values to train the statistical models. The key features of the selected F0 tracking approaches are summarized in Table I. A summary of the novel contributions with respect to our preliminary studies [15], [24] is as follows:…”