Efficient Training of Neural Transducer for Speech Recognition

Abstract

As one of the most popular sequence-to-sequence modeling approaches for speech recognition, the RNN-Transducer has achieved evolving performance with more and more sophisticated neural network models of growing size and increasing training epochs. While strong computation resources seem to be the prerequisite of training superior models, we try to overcome it by carefully designing a more efficient training pipeline. In this work, we propose an efficient 3-stage progressive training pipeline to build highly-performing neural transducer models from scratch with very limited computation resources in a reasonable short time period. The effectiveness of each stage is experimentally verified on both Librispeech and Switchboard corpora. The proposed pipeline is able to train transducer models approaching state-of-the-art performance with a single GPU in just 2-3 weeks. Our best conformer transducer achieves 4.1% WER on Librispeech test-other with only 35 epochs of training.

End-to-End Speech Recognition: A Survey

“…As shown in Figure 10, on Librispeech, the current best-published classical hybrid systems range around 2.3% (test-clean) and 4.9% (test-other) word error rate [323], [222], while there already are a number of E2E systems providing similar performance [224], [205], [320], [206], with some E2E systems clearly outperforming former state-of-the-art results with word error rates down to 1.8% (test-clean) and 3.7% (test-other) [324] with similar results reported in [45], [97]. Merging insights from classical HMM-based and monotonic RNN-T provided similarly well results with a limited training budget [124]. Finally, when trained on Switchboard 300h, the current best result, obtained with an E2E system [180] is 5.4% compared to 6.6% word error rate for the best hybrid system result [325] on the HUB5'00 Switchboard test set, in Figure 9 IX.…”

“…Dedicated training schedules have been developed to guide the optimization process and as part of that reach proper alignment behavior explicitly or implicitly [52], [124], [147]. Many approaches exist for learning rate control [161], [162]: NewBob [163], [164] and enhancements [162]; global versus parameter-wise learning rate control (exponential decay, power decay, etc.)…”

End-to-End Speech Recognition: A Survey

“…Merging insights from classical HMM-based and monotonic RNN-T provided similarly well results with a limited training budget [94]. Finally, when trained on Switchboard 300h, the current best result, obtained with an E2E system [151] is 5.4% compared to 6.6% word error rate for the best hybrid system result [306] on the HUB5'00 Switchboard test set.…”

“…( 4) by a single term corresponding to the alignment sequence generated. Recently, a similar procedure has been introduced in [94] also employing E2E models, only. In this work, CTC is used to initialize the training and to generate an initial alignment, followed by intermediate Viterbi-style RNN-T training and final fullsum fine tuning.…”

“…Dedicated training schedules have been developed to guide the optimization process and as part of that reach proper alignment behavior explicitly or implicitly [118], [45], [94].…”