Exploiting Semi-Supervised Training Through a Dropout Regularization in End-to-End Speech Recognition

Abstract

In this paper, we explore various approaches for semisupervised learning in an end-to-end automatic speech recognition (ASR) framework. The first step in our approach involves training a seed model on the limited amount of labelled data. Additional unlabelled speech data is employed through a data-selection mechanism to obtain the best hypothesized output, further used to retrain the seed model. However, uncertainties of the model may not be well captured with a single hypothesis. As opposed to this technique, we apply a dropout mechanism to capture the uncertainty by obtaining multiple hypothesized text transcripts of an speech recording. We assume that the diversity of automatically generated transcripts for an utterance will implicitly increase the reliability of the model. Finally, the data-selection process is also applied on these hypothesized transcripts to reduce the uncertainty. Experiments on freely-available TEDLIUM corpus and proprietary Adobe's internal dataset show that the proposed approach significantly reduces ASR errors, compared to the baseline model.

Multi-Spectral Widefield Microscopy of the Beating Heart Through Post-Acquisition Synchronization and Unmixing

“…Experiments on the Fisher English shows that the proposed approach can further improve the WER over the regular semisupervised training framework. While this chapter primarily focused on LF-MMI training, it is clear that the idea can be further extended to other frameworks such as end-to-end based semi-supervised training [Dey et al, 2019]. Had the time allowed, it would have made sense to investigate the proposed semi-supervised training approach in cross-lingual adaptation scenarios.…”

Spiker-Converter: A Semi-Supervised Framework for Low-Resource Speech Recognition with Stable Adversarial Training

“…Another way of using unlabeled speech is to generate pseudo-labels for them through a seed ASR model. Further sample selecting and filtering are applied to get informative ones [15,16].…”

Semi-Supervised Speech Recognition Via Graph-Based Temporal Classification

“…In [18], an N -best list of ASR hypotheses is used by summing over the weighted losses of multiple pseudo-labels for a single speech utterance, where weights are estimated from scores of a strong language model (LM). In [19], multiple pseudo-labels are generated for each unlabeled speech utterance using different dropout settings, which are used for selftraining with the purpose of capturing ASR uncertainties.…”