Improving the Performance of Transformer Based Low Resource Speech Recognition for Indian Languages

Shetty, Vishwas M.; J, Metilda Sagaya Mary N.; Umesh, S.

doi:10.1109/icassp40776.2020.9053808

Cited by 28 publications

(15 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Their extensive experimentation with classical SOTA techniques shows that transformer based models perform better than i.e., Mono-DNN, Mono-LSTM, SHL-MDNN, SHL-MLSTM, and SHL-MLSTM-residual models. (Shetty and NJ, 2020) also use transformer based models to show performance gains when using language ID. They conduct experiments in three Indian languages i.e., Gujarati, Tamil, and Telugu.…”

Section: Models That Use Only Labeled Datamentioning

confidence: 99%

A Survey of Multilingual Models for Automatic Speech Recognition

Yadav¹,

Sitaram²

2022

Preprint

View full text Add to dashboard Cite

Although Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, the majority of the world's languages do not have usable systems due to the lack of large speech datasets to train these models. Cross-lingual transfer is an attractive solution to this problem, because low-resource languages can potentially benefit from higher-resource languages either through transfer learning, or being jointly trained in the same multilingual model. The problem of cross-lingual transfer has been well studied in ASR, however, recent advances in Self Supervised Learning are opening up avenues for unlabeled speech data to be used in multilingual ASR models, which can pave the way for improved performance on low-resource languages. In this paper, we survey the state of the art in multilingual ASR models that are built with cross-lingual transfer in mind. We present best practices for building multilingual models from research across diverse languages and techniques, discuss open questions and provide recommendations for future work.

show abstract

Section: Models That Use Only Labeled Datamentioning

confidence: 99%

A Survey of Multilingual Models for Automatic Speech Recognition

Yadav¹,

Sitaram²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A few multi-lingual approaches are recently proposed in the E2E framework [23,24,25]. Authors in [23] propose the multilingual RNN-T model with language specific adapters and datasampling to handle data imbalance.…”

Section: Relation To Prior Workmentioning

confidence: 99%

“…Audio-to-byte E2E system is proposed in [24] where bytes are used as target units instead of grapheme or word piece units, as bytes are suitable to scale to multiple languages. A transformer based multi-lingual E2E model, along with methods to incorporate language information is proposed in [25]. Although multi-lingual methods are attractive to address the problem of low-resource languages, the transfer learning methods, besides being simple and effective, have the benefit of not needing the high-resource language data, but only the models trained on them.…”

Section: Relation To Prior Workmentioning

confidence: 99%

Transfer Learning Approaches for Streaming End-to-End Speech Recognition System

Joshi

Zhao

Mehta³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

Transfer learning (TL) is widely used in conventional hybrid automatic speech recognition (ASR) system, to transfer the knowledge from source to target language. TL can be applied to end-to-end (E2E) ASR system such as recurrent neural network transducer (RNN-T) models, by initializing the encoder and/or prediction network of the target language with the pre-trained models from source language. In the hybrid ASR system, transfer learning is typically done by initializing the target language acoustic model (AM) with source language AM. Several transfer learning strategies exist in the case of the RNN-T framework, depending upon the choice of the initialization model for encoder and prediction networks. This paper presents a comparative study of four different TL methods for RNN-T framework. We show 10% − 17% relative word error rate reduction with different TL methods over randomly initialized RNN-T model. We also study the impact of TL with varying amount of training data ranging from 50 hours to 1000 hours and show the efficacy of TL for languages with a very small amount of training data.

show abstract

“…In this context, many works are based on E2E frameworks to explore different ASR scenarios and improve the performance accordingly. For example, code-switch [6][7][8][9] and low-resource task [10][11][12] etc.…”

Section: Introductionmentioning

confidence: 99%

Leveraging Phone Mask Training for Phonetic-Reduction-Robust E2E Uyghur Speech Recognition

Ma,

Hu,

Kang

et al. 2022

Preprint

View full text Add to dashboard Cite

In Uyghur speech, consonant and vowel reduction are often encountered, especially in spontaneous speech with high speech rate, which will cause a degradation of speech recognition performance. To solve this problem, we propose an effective phone mask training method for Conformer-based Uyghur end-to-end (E2E) speech recognition. The idea is to randomly mask off a certain percentage features of phones during model training, which simulates the above verbal phenomena and facilitates E2E model to learn more contextual information. According to experiments, the above issues can be greatly alleviated. In addition, deep investigations are carried out into different units in masking, which shows the effectiveness of our proposed masking unit. We also further study the masking method and optimize filling strategy of phone mask. Finally, compared with Conformer-based E2E baseline without mask training, our model demonstrates about 5.51% relative Word Error Rate (WER) reduction on reading speech and 12.92% on spontaneous speech, respectively. The above approach has also been verified on test-set of open-source data THUYG-20, which shows 20% relative improvements.

show abstract

Improving the Performance of Transformer Based Low Resource Speech Recognition for Indian Languages

Cited by 28 publications

References 14 publications

A Survey of Multilingual Models for Automatic Speech Recognition

A Survey of Multilingual Models for Automatic Speech Recognition

Transfer Learning Approaches for Streaming End-to-End Speech Recognition System

Leveraging Phone Mask Training for Phonetic-Reduction-Robust E2E Uyghur Speech Recognition

Contact Info

Product

Resources

About