Have Best of Both Worlds: Two-Pass Hybrid and E2E Cascading Framework for Speech Recognition

Guoli, Ye,; Mazalov, Vadim; Li, Jinyu; Gong, Yifan

doi:10.1109/icassp43922.2022.9747144

Cited by 6 publications

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the streaming is built on a causal encoder where at the current step the model observes only the historical information and has no access the future information. To further improve the recognition accuracy of the neural transducer model, researchers proposed two-pass models [11], deliberation models [12][13][14][15], and spell correction models [16].…”

Section: Introductionmentioning

confidence: 99%

Acoustic-aware Non-autoregressive Spell Correction with Mask Sample Decoding

Fan¹,

Ye²,

Gaur³

et al. 2022

Preprint

View full text Add to dashboard Cite

Masked language model (MLM) has been widely used for understanding tasks, e.g. BERT. Recently, MLM has also been used for generation tasks. The most popular one in speech is using Mask-CTC for non-autoregressive speech recognition. In this paper, we take one step further, and explore the possibility of using MLM as a non-autoregressive spell correction (SC) model for transformertransducer (TT), denoted as MLM-SC. Our initial experiments show that MLM-SC provides no improvements on Librispeech data. The problem might be the choice of modeling units (word pieces) and the inaccuracy of the TT confidence scores for English data. To solve the problem, we propose a mask sample decoding (MS-decode) method where the masked tokens can have the choice of being masked or not to compensate for the inaccuracy. As a result, we reduce the WER of a streaming TT from 7.6% to 6.5% on the Librispeech testother data and the CER from 7.3% to 6.1% on the Aishell test data, respectively.

show abstract