Fusion Strategies for Robust Speech Recognition and Keyword Spotting for Channel- and Noise-Degraded Speech

Abstract

Recognizing speech under high levels of channel and/or noise degradation is challenging. Current state-of-the-art automatic speech recognition systems are sensitive to changing acoustic conditions, which can cause significant performance degradation. Noise-robust acoustic features can improve speech recognition performance under varying background conditions, where it is usually observed that robust modeling techniques and multiple system fusion can help to improve the performance even further. This work investigates a wide array of robust acoustic features that have been previously used to successfully improve speech recognition robustness. We use these features to train individual acoustic models, and we analyze their individual performance. We investigate and report results for simple feature combination, feature-map combination at the output of convolutional layers, and fusion of deep neural nets at the senone posterior level. We report results for speech recognition on a large-vocabulary, noiseand channel-degraded Levantine Arabic speech corpus distributed through the Defense Advance Research Projects Agency (DARPA) Robust Automatic Speech Transcription (RATS) program. In addition, we report keyword spotting results to demonstrate the effect of robust features and multiple levels of information fusion.

Investigation and analysis of hyper and hypo neuron pruning to selectively update neurons during unsupervised adaptation

“…We observed that time-frequency convolution (using TFCNN [11,17,18,19,20,21]) performed better than 1-D frequency convolution, and hence we have focused on the TFCNN acoustic models for our experiments presented in this paper. The TFCNN architecture is same as in [11,22], where two parallel convolutional layers are used at the input, one performing convolution across time, and the other across frequency on the input filterbank features. The TFCNNs had 75 filters to perform time convolution and 200 filters to perform frequency convolution.…”

An In-Vehicle Keyword Spotting System with Multi-Source Fusion for Vehicle Applications

“…However, the performance is not very sensitive to w s and w max . Therefore, the multisource fusion algorithm is proposed to help the system choose better sensitivity value [17]. How the vehicle information is processed to execute the algorithm is explained as follow.…”