Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model

Kannan, Anjuli; Datta, Arindrima; Sainath, Tara N.; Weinstein, Eugene A.; Ramabhadran, Bhuvana; Wu, Yangjie; Bapna, Ankur; Chen, Zhifeng; Lee, Seungji

doi:10.48550/arxiv.1909.05330

Cited by 20 publications

(32 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Datta et al [21] unified different writing systems through a many-to-one transliteration transducer. Recently, large-scale multilingual ASR systems have been investigated [3], [4], [6], [10], [22]. Pratap et al [3] proposed jointly training on 16,000 hours speech data of 51 languages with up to 1 billion parameters.…”

Section: A Multilingual and Cross-lingual Speech Recognitionmentioning

confidence: 99%

“…Some researchers have proposed to apply the Adapters to the E2E ASR tasks. In [10], Kannan et al proposes to use the adapters to handle the data imbalance problem for large-scale multilingual ASR. After obtaining the model trained on the union of data from all languages, they trained the languagedependent adapters on each of the languages again so that the multilingual backbone shares information across languages while the adapters could allow for per-language specialization.…”

Section: B Adaptersmentioning

confidence: 99%

“…Due to the limited training data in low-resource languages, direct re-training makes the model easily overfit. These problems make the transfer-based methods inefficient [9], [10]. Recently, the adapter module was proposed for parameter-efficient fine-tuning in multilingual or cross-lingual settings [9]- [11], which can mitigate overfitting.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Exploiting Adapters for Cross-Lingual Low-Resource Speech Recognition

Hou

Zhu

Wang

et al. 2022

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Cross-lingual speech adaptation aims to solve the problem of leveraging multiple rich-resource languages to build models for a low-resource target language. Since the low-resource language has limited training data, speech recognition models can easily overfit. In this paper, we propose to use adapters to investigate the performance of multiple adapters for parameterefficient cross-lingual speech adaptation. Based on our previous MetaAdapter that implicitly leverages adapters, we propose a novel algorithms called SimAdapter for explicitly learning knowledge from adapters. Our algorithm leverages adapters which can be easily integrated into the Transformer structure. MetaAdapter leverages meta-learning to transfer the general knowledge from training data to the test language. SimAdapter aims to learn the similarities between the source and target languages during fine-tuning using the adapters. We conduct extensive experiments on five-low-resource languages in Common Voice dataset. Results demonstrate that our MetaAdapter and SimAdapter methods can reduce WER by 2.98% and 2.55% with only 2.5% and 15.5% of trainable parameters compared to the strong full-model fine-tuning baseline. Moreover, we also show that these two novel algorithms can be integrated for better performance with up to 3.55% relative WER reduction.

show abstract

Section: A Multilingual and Cross-lingual Speech Recognitionmentioning

confidence: 99%

Section: B Adaptersmentioning

confidence: 99%

See 1 more Smart Citation

Exploiting Adapters for Cross-Lingual Low-Resource Speech Recognition

Hou

Zhu

Wang

et al. 2022

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…In the world of speech recognition, training a single recognizer for multiple languages is not a thematic stranger [3] from Hidden Markov Model (HMM) based models [17,18], hybrid models [19] to end-to-end neural based models with CTC [20,21] or sequence-to-sequence models [22,5,23,24,25,26], with the last approach being inspired by the success of multilingual machine translation [1,2]. The literature especially mentions the merits of disclosing the language identity (when the utterance is supposed to belong to a single language) to the model, whose architecture is designed to incorporate the language information.…”

Section: Related Work and Comparisonmentioning

confidence: 99%

Efficient Weight factorization for Multilingual Speech Recognition

Pham¹,

Nguyen²,

Stueker³

et al. 2021

Preprint

View full text Add to dashboard Cite

End-to-end multilingual speech recognition involves using a single model training on a compositional speech corpus including many languages, resulting in a single neural network to handle transcribing different languages. Due to the fact that each language in the training data has different characteristics, the shared network may struggle to optimize for all various languages simultaneously. In this paper we propose a novel multilingual architecture that targets the core operation in neural networks: linear transformation functions. The key idea of the method is to assign fast weight matrices for each language by decomposing each weight matrix into a shared component and a language dependent component. The latter is then factorized into vectors using rank-1 assumptions to reduce the number of parameters per language. This efficient factorization scheme is proved to be effective in two multilingual settings with 7 and 27 languages, reducing the word error rates by 26% and 27% rel. for two popular architectures LSTM and Transformer, respectively.

show abstract

“…Training a single ASR model to support multiple languages is promising and challenging [20,21,22,23]. Through shared learning of model parameters across languages [24,25,26], multilingual ASR models can perform better than monolingual models, particularly for those languages with less data.…”

Section: Multilingual Speech Recognitionmentioning

confidence: 99%

A Configurable Multilingual Model is All You Need to Recognize All Languages

Zhang¹,

Li²,

Sun³

et al. 2021

Preprint

View full text Add to dashboard Cite

Multilingual automatic speech recognition (ASR) models have shown great promise in recent years because of the simplified model training and deployment process. Conventional methods either train a universal multilingual model without taking any language information or with a 1-hot language ID (LID) vector to guide the recognition of the target language. In practice, the user can be prompted to pre-select several languages he/she can speak. The multilingual model without LID cannot well utilize the language information set by the user while the multilingual model with LID can only handle one pre-selected language. In this paper, we propose a novel configurable multilingual model (CMM) which is trained only once but can be configured as different models based on users' choices by extracting language-specific modules together with a universal model from the trained CMM. Particularly, a single CMM can be deployed to any user scenario where the users can pre-select any combination of languages. Trained with 75K hours of transcribed anonymized Microsoft multilingual data and evaluated with 10-language test sets, the proposed CMM improves from the universal multilingual model by 26.0%, 16.9%, and 10.4% relative word error reduction when the user selects 1, 2, or 3 languages, respectively. CMM also performs significantly better on code-switching test sets.

show abstract

Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Model

Cited by 20 publications

References 0 publications

Exploiting Adapters for Cross-Lingual Low-Resource Speech Recognition

Exploiting Adapters for Cross-Lingual Low-Resource Speech Recognition

Efficient Weight factorization for Multilingual Speech Recognition

A Configurable Multilingual Model is All You Need to Recognize All Languages

Contact Info

Product

Resources

About