Speech dereverberation for enhancement and recognition using dynamic features constrained deep neural networks and feature adaptation

Abstract

This paper investigates deep neural networks (DNN) based on nonlinear feature mapping and statistical linear feature adaptation approaches for reducing reverberation in speech signals. In the nonlinear feature mapping approach, DNN is trained from parallel clean/distorted speech corpus to map reverberant and noisy speech coefficients (such as log magnitude spectrum) to the underlying clean speech coefficients. The constraint imposed by dynamic features (i.e., the time derivatives of the speech coefficients) are used to enhance the smoothness of predicted coefficient trajectories in two ways. One is to obtain the enhanced speech coefficients with a least square estimation from the coefficients and dynamic features predicted by DNN. The other is to incorporate the constraint of dynamic features directly into the DNN training process using a sequential cost function. In the linear feature adaptation approach, a sparse linear transform, called cross transform, is used to transform multiple frames of speech coefficients to a new feature space. The transform is estimated to maximize the likelihood of the transformed coefficients given a model of clean speech coefficients. Unlike the DNN approach, no parallel corpus is used and no assumption on distortion types is made. The two approaches are evaluated on the REVERB Challenge 2014 tasks. Both speech enhancement and automatic speech recognition (ASR) results show that the DNN-based mappings significantly reduce the reverberation in speech and improve both speech quality and ASR performance. For the speech enhancement task, the proposed dynamic feature constraint help to improve cepstral distance, frequency-weighted segmental signal-to-noise ratio (SNR), and log likelihood ratio metrics while moderately degrades the speech-to-reverberation modulation energy ratio. In addition, the cross transform feature adaptation improves the ASR performance significantly for clean-condition trained acoustic models.

Speech Dereverberation Based on Integrated Deep and Ensemble Learning Algorithm

“…In previous studies, traditional fully connected DNNs were used to perform dereverberation [21][22][23]. More recently, the highway strat- egy has been popularly used and shown to provide improved performance [29].…”

“…Channel-inversion methods belong to the third category, which considers the reverberation as a convolution of the original sound with the room impulse response (RIR) and thereby performs an inverse filtering to deconvolve the captured signal [15][16][17][18][19][20]. Even though the above three categories of approaches have been shown to provide satisfactory performance, they usually require an accurate estimation of time-varied RIR, which may not always be accessible in practice [21].…”

“…Recently, deep neural network (DNN) models, which show strong regression capabilities, have been used to address the speech dereverberation issue [21,22]. The main concept here is to use a DNN model to characterize the non-linear spectral mapping from reverberant to anechoic speech in the training stage.…”

Amplitude and phase dereverberation of monocomponent signals

“…These methods process the reverberant signal by estimating the magnitude spectrogram of the late reverberation and remove it from the magnitude spectrogram of the input signal. Suppression methods are based on a wide variety of approaches, such as a stochastic model of the RIR [5], a linear prediction model of the speech [6], a non-negative matrix factorization approach [7], or more recently on deep neural networks [8].…”

“…This is the main drawback of these methods, because using this corrupted phase reintroduces reverberation and distortion in the signal, as shown in [8]. In the source separation literature, the idea of modeling the phase has recently been proposed [9] because a similar problem occurs (the phase of the mixture is generally used to synthesize the source signals), but not for dereverberation.…”