Exemplar-based speech enhancement for deep neural network based automatic speech recognition

Abstract

Deep neural network (DNN) based acoustic modelling has been successfully used for a variety of automatic speech recognition (ASR) tasks, thanks to its ability to learn higher-level information using multiple hidden layers. This paper investigates the recently proposed exemplar-based speech enhancement technique using coupled dictionaries as a pre-processing stage for DNN-based systems. In this setting, the noisy speech is decomposed as a weighted sum of atoms in an input dictionary containing exemplars sampled from a domain of choice, and the resulting weights are applied to a coupled output dictionary containing exemplars sampled in the shorttime Fourier transform (STFT) domain to directly obtain the speech and noise estimates for speech enhancement. In this work, settings using input dictionary of exemplars sampled from the STFT, Melintegrated magnitude STFT and modulation envelope spectra are evaluated. Experiments performed on the AURORA-4 database revealed that these pre-processing stages can improve the performance of the DNN-HMM-based ASR systems with both clean and multicondition training.

Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition

“…Recently, deep-learning-based SE approaches have received increased attention and it has been confirmed that they yield better performances than traditional methods in many tasks [10,11,12]. Because of the deep structure, the deep-learning-based models can effectively characterize the complex transformation of noisy speech to clean speech, or they can precisely estimate a mask to filter out noise components from the noisy speech.…”

Noise Robust Automatic Speech Recognition Based on Spectro-Temporal Techniques

“…While Huang et al applied a simple CNN on the FBANK features, others applied either a sophisticated feature extraction method [12,136] or a refined CNN architecture [32]. We see that the ARMA feature set outperforms both these approaches by a large margin.…”

“…In the latter comparison, it is also noteworthy that while Martinez et al reported a training time of 4 days for their DNNs on an NVIDIA Tesla K20C GPU, our neural net training times were below 12 hours on a GPU of equal performance (GeForce GTX 770) [1]. Lastly, the inclusion of Delta-like coefficients meant that the proposed framework could also attain lower WERs than those reported by Baby et al using a sophisticated speech enhancement technique [12].…”

“…Once again, this might be due to the noise robustness of ARMA features. [57] 18.5% DNN with DNN speech enhancement of FBANK [102] 17.5% DNN with Spectral masking [136] 22.8% CNN with PNS features plus Gabor Filter Kernels [32] 22.9% DNN with Exemplar Based Enhancement [12] 26.8% CNN with FBANK features [90] 28.9%…”

“…What is more is that with both proposed modifications, the improvement in recognition scores is significant (at p < 0.00005). [32] 16.6% Seltzer et al [206] 12.4% Martinez et al [145] 12.3% Baby et al [12] 11.9% Current work 11.6% Narayanan and Wang [171] 11.1% Rennie et al [192] 10.3%…”