Speech feature denoising and dereverberation via deep autoencoders for noisy reverberant speech recognition

Feng, Xue; Zhang, Yaodong; Glass, James

doi:10.1109/icassp.2014.6853900

Cited by 198 publications

(96 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Furthermore, results show additional improvement when fMLLR is applied in combination with the mapping approach. Note that in [55], a denoising autoencoder is used for mapping, and ASR improvement is reported when recognizing noisy speech. One of the reasons for this is that they initialize the deep neural network by performing pre-training using an efficient algorithm [55].…”

Section: Discussionmentioning

confidence: 99%

“…Note that in [55], a denoising autoencoder is used for mapping, and ASR improvement is reported when recognizing noisy speech. One of the reasons for this is that they initialize the deep neural network by performing pre-training using an efficient algorithm [55]. In our preliminary experiments, direct feature mapping (i.e, SDM MFCC to IHM MFCC) does not yield any improvement when recognizing distant speech if we did not initialize the neural network.…”

Section: Discussionmentioning

confidence: 99%

“…The mapping is performed between features extracted from noisy and clean speech signals to obtain an optimal set of parameters through the error backpropagation algorithm [52,53,54]. The goal is to obtain clean or enhanced speech from the noisy input via a nonlinear transformation using neural networks such as a deep denoising autoencoder [55] or a multilayer perceptron (MLP) [56]. A recent study [56] considers a multi-speaker scenario where the nonlinear mapping is performed with the objective of improving overlapped speech recognition using beamformed audio from a microphone array.…”

Section: Feature Mapping Techniques Using Dnnmentioning

confidence: 99%

See 2 more Smart Citations

Feature mapping using far-field microphones for distant speech recognition

Himawan

Motlíček

Sridharan

2016

Speech Communication

View full text Add to dashboard Cite

Acoustic modeling based on deep architectures has recently gained remarkable success, with substantial improvement of speech recognition accuracy in several automatic speech recognition (ASR) tasks. For distant speech recognition, the multi-channel deep neural network based approaches rely on the powerful modeling capability of deep neural network (DNN) to learn suitable representation of distant speech directly from its multi-channel source. In this model-based combination of multiple microphones, features from each channel are concatenated and used together as an input to DNN. This allows integrating the multi-channel audio for acoustic modeling without any pre-processing steps. Despite powerful modeling capabilities of DNN, an environmental mismatch due to noise and reverberation may result in severe performance degradation when features are simply fed to a DNN without a feature enhancement step. In this paper, we introduce the nonlinear bottleneck feature mapping approach using DNN, to transform the noisy and reverberant features to its clean version. The bottleneck features trained on clean signal are used as a teacher signal because they contain relevant information to phoneme classification, and the mapping is performed with the objective of suppressing noise and reverberation. The individual and combined impacts of beamforming and speaker adaptation techniques along with the feature mapping are examined for distant large vocabulary speech recognition, using a single and multiple far-field microphones. As an alternative to beamforming, experiments with concatenating multiple channel features are conducted. The experimental results on the AMI meeting corpus show that the feature mapping, used in combination with beamforming and speaker adaptation yields a distant speech recognition performance below 50% word error rate (WER), using DNN for acoustic modeling.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Feature Mapping Techniques Using Dnnmentioning

confidence: 99%

See 1 more Smart Citation

Feature mapping using far-field microphones for distant speech recognition

Himawan

Motlíček

Sridharan

2016

Speech Communication

View full text Add to dashboard Cite

show abstract

“…However, advances in deep learning research have led to recent breakthroughs in unsupervised audio feature extraction methods and exceptional recognition performance improvements [13,17,32]. Advances in novel machine learning algorithms, improved availability of computational resources, and the development of large databases have led to self-organization of robust audio features by efficient training of large-scale DNNs with large-scale datasets.…”

Section: Audio Feature Extraction Mechanismsmentioning

confidence: 99%

Audio-visual speech recognition using deep learning

et al. 2014

View full text Add to dashboard Cite

Audio-visual speech recognition (AVSR) system is thought to be one of the most promising solutions for reliable speech recognition, particularly when the audio is corrupted by noise. However, cautious selection of sensory features is crucial for attaining high recognition performance. In the machine-learning community, deep learning approaches have recently attracted increasing attention because deep neural networks can effectively extract robust latent features that enable various recognition algorithms to demonstrate revolutionary generalization capabilities under diverse application conditions. This study introduces a connectionist-hidden Markov model (HMM) system for noise-robust AVSR. First, a deep denoising autoencoder is utilized for acquiring noise-robust audio features. By preparing the training data for the network with pairs of consecutive multiple steps of deteriorated audio features and the corresponding clean features, the network is trained to output denoised audio features from the corresponding features deteriorated by noise. Second, a convolutional neural network (CNN) is utilized to extract visual features from raw mouth area images. By preparing the training data for the CNN as pairs of raw images and the corresponding phoneme label outputs, the network is trained to predict phoneme labels from the corresponding mouth area input images. Finally, a multi-stream HMM (MSHMM) is applied for integrating the acquired audio and visual HMMs independently trained with the respective features. By comparing the cases when normal and denoised mel-frequency cepstral coefficients (MFCCs) are utilized as audio features to the HMM, our unimodal isolated word recognition results demonstrate that approximately 65 % word recognition rate gain is attained with denoised MFCCs under 10 dB signalto-noise-ratio (SNR) for the audio signal input. Moreover, our multimodal isolated word recognition results utilizing MSHMM with denoised MFCCs and acquired visual features demonstrate that an additional word recognition rate gain is attained for the SNR conditions below 10 dB.

show abstract

“…In [24], some novel methods for taking advantage of reverberant speech training in modern DNN-based hidden Markov model (DNN-HMM) systems were proposed. Based on multi-condition training, feature-level dereverberation by deep autoencoders (DAEs) has been investigated in [25,26]. In these works, DAEs were trained using reverberant speech features as input and clean speech features as learning targets.…”

Section: Introductionmentioning

confidence: 99%

Joint training of DNNs by incorporating an explicit dereverberation structure for distant speech recognition

Gao

et al. 2016

EURASIP J. Adv. Signal Process.

View full text Add to dashboard Cite

We explore joint training strategies of DNNs for simultaneous dereverberation and acoustic modeling to improve the performance of distant speech recognition. There are two key contributions. First, a new DNN structure incorporating both dereverberated and original reverberant features is shown to effectively improve recognition accuracy over the conventional one using only dereverberated features as the input. Second, in most of the simulated reverberant environments for training data collection and DNN-based dereverberation, the resource data and learning targets are high-quality clean speech. With our joint training strategy, we can relax this constraint by using large-scale diversified real close-talking data as the targets which are easy to be collected via many speech-enabled applications from mobile internet users, and find the scenario even more effective. Our experiments on a Mandarin speech recognition task with 2000-h training data show that the proposed framework achieves relative word error rate reductions of 9.7 and 8.6 % over the multi-condition training systems for the cases of single-channel and multi-channel with beamforming, respectively. Furthermore, significant gains are consistently observed over the pre-processing approach using simply DNN-based dereverberation.

show abstract

Speech feature denoising and dereverberation via deep autoencoders for noisy reverberant speech recognition

Cited by 198 publications

References 14 publications

Feature mapping using far-field microphones for distant speech recognition

Feature mapping using far-field microphones for distant speech recognition

Audio-visual speech recognition using deep learning

Joint training of DNNs by incorporating an explicit dereverberation structure for distant speech recognition

Contact Info

Product

Resources

About