Meta Learning for End-to-End Low-Resource Speech Recognition

DARTS-ASR: Differentiable Architecture Search for Multilingual Speech Recognition and Adaptation

“…Following the previous works [22,23], we used a CNN-BiLSTM-Head structure as the multilingual ASR model, as shown in Figure 2(a), and adopted Connectionist Temporal Classification (CTC) [26] loss as the objective function. The baseline model architecture followed the previous work [23], where the CNN module was a 6-layer VGG block as shown in Figure 2(b), and the BiLSTM module was a 3-layer bidirectional LSTM network with 360 cells in each direction. We experimented with the channel number of convolutions in VGG as 128 or 512, and the results of these two settings in the following subsection were named as VGG-Small and VGG-Large.…”

“…Following the previous works [17,19,22,23], we conducted experiments on the multilingual dataset, IARPA BA-BEL [24]. The experiment results show that our approach out-arXiv:2005.07029v1 [eess.AS] 13 May 2020 performed the baseline fixed-topology architecture by 10.2% and 10.0% relative reduction on character error rates (CER) under monolingual and multilingual ASR settings respectively.…”

“…In order to examine the generalizability of DARTS-ASR, we apply our approach not only on many languages to perform monolingual ASR, but also on a multilingual ASR setting, where the architecture and parameter weights are pre-trained on some source languages, and then adapted on the target language. It has recently been shown that multilingual ASR [15,16,17,18,19,20,21,22,23] can improve ASR performance on many low-resource languages. In the above previous works, the initial parameters or shared encoder learned from many source languages are used to build a better acoustic model for the target language.…”

Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation

“…It is known that multilingual training or pre-training with re- lated languages improves low-resource end-to-end ASR significantly [16,17,18,19]. Meta learning methods [38] have recently been introduced to improve the efficiency of multilingual pre-training. Besides cross-lingual transfer learning, leveraging auxiliary data is another approach to improve lowresource ASR, for example, incorporating (synthetic) text translation data as additional inputs [21,20,22] or co-training with weakly supervised data [39] or text-to-speech (TTS) data [40].…”