Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition

Gao, Zhi‐Fu; Zhang, Shiliang; McLoughlin, Ian; Yan, Zhijie

doi:10.21437/interspeech.2022-9996

Cited by 29 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, its performance on smaller datasets is inferior to that of RNNT. Paraformer, a non-autoregressive model, enhances contextual modeling via the GLM [ 44 ] sampler but underperforms compared to autoregressive models in this experiment. Paraformer (U2) also employs non-autoregressive decoding and adds CTC loss for joint optimization, improving the encoder’s feature extraction capability.…”

Section: Experiments and Analysismentioning

confidence: 98%

Nonlinear Regularization Decoding Method for Speech Recognition

Zhang,

Wang,

et al. 2024

Sensors

View full text Add to dashboard Cite

Existing end-to-end speech recognition methods typically employ hybrid decoders based on CTC and Transformer. However, the issue of error accumulation in these hybrid decoders hinders further improvements in accuracy. Additionally, most existing models are built upon Transformer architecture, which tends to be complex and unfriendly to small datasets. Hence, we propose a Nonlinear Regularization Decoding Method for Speech Recognition. Firstly, we introduce the nonlinear Transformer decoder, breaking away from traditional left-to-right or right-to-left decoding orders and enabling associations between any characters, mitigating the limitations of Transformer architectures on small datasets. Secondly, we propose a novel regularization attention module to optimize the attention score matrix, reducing the impact of early errors on later outputs. Finally, we introduce the tiny model to address the challenge of overly large model parameters. The experimental results indicate that our model demonstrates good performance. Compared to the baseline, our model achieves recognition improvements of 0.12%, 0.54%, 0.51%, and 1.2% on the Aishell1, Primewords, Free ST Chinese Corpus, and Common Voice 16.1 datasets of Uyghur, respectively.

show abstract

Section: Experiments and Analysismentioning

confidence: 98%

Nonlinear Regularization Decoding Method for Speech Recognition

Zhang,

Wang,

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Speech Synthesis Due to the significant difference in the pronunciation synthesis of different languages, we use dif-ferent models for different languages. For Chinese, Gao et al (Gao et al 2022) proposed a single-round nonautoregressive model Paraformer. The predictor module is used to predict the number of target characters in the speech, and the sampler transforms the acoustic feature vector and the target character vector into a feature vector containing semantic information.…”

Section: Speech Recognitionmentioning

confidence: 99%

A Simple Survey of Pre-trained Language Models

Zhu¹

2022

Preprint

View full text Add to dashboard Cite

Pre-trained Language Models (PTLM) have remarkable and successful performance in solving lots of NLP tasks nowadays. And previous researchers have created many SOTA models and these models are included in many long surveys(Qiu et al., 2020). So, we would like to conduct a simple and short survey on this topic to help researchers understand the sketch of PTLM more quickly and comprehensively. In this short survey, we would provide a simple but comprehensive review of techniques, benchmarks, and methodologies in PTLM. And we would also introduce the applications evaluation of PTLM in this simple survey.

show abstract

“…Non-autoregressive speech processing is first used in [18]. After that, many more non-autoregressive methods are proposed [19][20][21][22][23][24][25]. Among the methods, there are two that are appropriate to achieve non-autoregressive spell correction.…”

Section: Introductionmentioning

confidence: 99%

Acoustic-aware Non-autoregressive Spell Correction with Mask Sample Decoding

Fan¹,

Ye²,

Gaur³

et al. 2022

Preprint

View full text Add to dashboard Cite

Masked language model (MLM) has been widely used for understanding tasks, e.g. BERT. Recently, MLM has also been used for generation tasks. The most popular one in speech is using Mask-CTC for non-autoregressive speech recognition. In this paper, we take one step further, and explore the possibility of using MLM as a non-autoregressive spell correction (SC) model for transformertransducer (TT), denoted as MLM-SC. Our initial experiments show that MLM-SC provides no improvements on Librispeech data. The problem might be the choice of modeling units (word pieces) and the inaccuracy of the TT confidence scores for English data. To solve the problem, we propose a mask sample decoding (MS-decode) method where the masked tokens can have the choice of being masked or not to compensate for the inaccuracy. As a result, we reduce the WER of a streaming TT from 7.6% to 6.5% on the Librispeech testother data and the CER from 7.3% to 6.1% on the Aishell test data, respectively.

show abstract

Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition

Cited by 29 publications

References 28 publications

Nonlinear Regularization Decoding Method for Speech Recognition

Nonlinear Regularization Decoding Method for Speech Recognition

A Simple Survey of Pre-trained Language Models

Acoustic-aware Non-autoregressive Spell Correction with Mask Sample Decoding

Contact Info

Product

Resources

About