Improving Noise Robustness of Automatic Speech Recognition via Parallel Data and Teacher-student Learning

Abstract

For real-world speech recognition applications, noise robustness is still a challenge. In this work, we adopt the teacherstudent (T/S) learning technique using a parallel clean and noisy corpus for improving automatic speech recognition (ASR) performance under multimedia noise. On top of that, we apply a logits selection method which only preserves the k highest values to prevent wrong emphasis of knowledge from the teacher and to reduce bandwidth needed for transferring data. We incorporate up to 8000 hours of untranscribed data for training and present our results on sequence trained models apart from cross entropy trained ones. The best sequence trained student model yields relative word error rate (WER) reductions of approximately 10.1%, 28.7% and 19.6% on our clean, simulated noisy and real test sets respectively comparing to a sequence trained teacher.Index Termsautomatic speech recognition, noise robustness, teacher-student training, domain adaptation * Ladislav Mosner performed the work while he was a research intern at Amazon.

Fully Learnable Front-End for Multi-Channel Acoustic Modeling Using Semi-Supervised Learning

“…The gains we observe using T/S training are in the same range as the results reported in [15], and our experiments demonstrate that we can distill information from a single channel teacher model to a multi-channel student model and learn the front-end components in a data-driven manner. The second observation we make is the WER improvements with pre-training improve performance even after T/S training.…”

“…We also soften the senone logits output by the teacher using temperature T . For all our experiments, we use T = 2 since that was found to be the optimal parameter in [15].…”

“…In Li et al [14], the authors improve speech recognition performance of a distant microphone by applying T/S training to utterances recorded simultaneously using a close-talking distant microphones. In a similar vein, Mosner et al [15] apply T/S to improve noise robustness by creating a parallel corpus by adding multimedia interference to clean utterances. T/S strategy has also been used for improving the overall ASR performance of the student model by leveraging significantly larger amount of untranscribed or unlabelled speech data.…”

Conditional Teacher-student Learning

“…Teacher-student (T/S) learning [1,2] has been widely applied to a variety of deep learning tasks in speech, language and image processing including model compression [1,2], domain adaptation [3,4,5], small-footprint natural machine translation (NMT) [6], low-resource NMT [7], far-field automatic speech recognition (ASR) [8,9], lowresource language ASR [10] and neural network pre-training [11]. T/S learning falls in the category of transfer learning, where the network of interest, as a student, is trained by mimicking the behavior of a well-trained network, as a teacher, in the presence of the same or stereo training samples.…”

Realizing Petabyte Scale Acoustic Modeling

“…The AM is based on the standard HMM/deep learning hybrid, and we summarize details relevant to this paper in Section II-B. Other aspects of this system have been described elsewhere ( [25], [26], [27], [28]). The LM [29] estimates the a priori probability that the speaker will utter a sequence of words.…”