Dual Language Models for Code Switched Speech Recognition

Garg, Saurabh; Parekh, Tanmay; Jyothi, Preethi

doi:10.21437/interspeech.2018-1343

Cited by 20 publications

(14 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Baheti et al (2017) find that fine-tuning with CS data after pretraining on monolingual data works best. Finally, another line of works suggests using a dual language model, where two monolingual LMs are combined by a probabilistic model (Garg et al, 2017(Garg et al, , 2018.…”

Section: Related Workmentioning

confidence: 99%

Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training

Gonen

Goldberg

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Our implementation is based on the Carmel FST toolkit. 1 We create an FST for converting a sentence into a sequence of phonemes, and its inverse FST. The words to phoneme mapping is based on pronunciation dictionaries, according to the language tag of each word in the sentence.We use The CMU Pronouncing Dictionary 2 for English and a dictionary from CMUSphinx 3 for Spanish. As the phoneme inventories in the two datasets do not match, we map the Spanish phonemes to the CMU dict inventory using a manually constructed mapping. 4 To favor frequent words over infrequent ones, we add unigram probabilities to the edges of the transducer (taken from googlebooks unigrams 5 ). We filter some words that produce noise (for example, single letter words that are too frequent). When creating a monolingual sentence, we use an FST with the words of that language only. As many phoneme sequences in Spanish do not produce English alternatives (and vice versa) we allow minor changes in the phoneme sequences between the languages. Specifically, we create a small list of similar phonemes (such as "B" and "V"), 6 and generate an FST that for each phoneme allows changing it to one of its alternatives or 1 https://www.isi.edu/licensed-sw/carmel/ 2

show abstract

Section: Related Workmentioning

confidence: 99%

Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training

Gonen

Goldberg

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…Recent work [6] attempts to train a CS language model using fine-tuning. Similar work [7] integrates two monolingual language models (LMs) by introducing a special "switch" token in both languages when training the LM, and further incorporating this within automatic speech recognition (ASR). Other works synthesize additional CS text using the modeled distribution from the data [8,9].…”

Section: Introductionmentioning

confidence: 99%

Training Code-Switching Language Model with Monolingual Data

Chuang

Sung

Lee

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

A lack of code-switching data complicates the training of code-switching (CS) language models. We propose an approach to train such CS language models on monolingual data only. By constraining and normalizing the output projection matrix in RNN-based language models, we bring embeddings of different languages closer to each other. Numerical and visualization results show that the proposed approaches remarkably improve the performance of CS language models trained on monolingual data. The proposed approaches are comparable or even better than training CS language models with artificially generated CS data. We additionally use unsupervised bilingual word translation to analyze whether semantically equivalent words in different languages are mapped together.

show abstract

“…Equivalence Constraint and Functional Head Constraint are used to build a better CS language model [6,7,8], and CS models with syntactic and semantic features are built to exploit more information [9,10]. Because of a large amount of monolingual data, monolingual language models for host and guest languages are learned separately, and then combined with a probabilistic model for switching between the two [11].…”

Section: Introductionmentioning

confidence: 99%

Code-Switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation

Chang¹,

Chuang²,

Lee³

2019

Interspeech 2019

View full text Add to dashboard Cite

Code-switching is about dealing with alternative languages in speech or text. It is partially speaker-dependent and domainrelated, so completely explaining the phenomenon by linguistic rules is challenging. Compared to most monolingual tasks, insufficient data is an issue for code-switching. To mitigate the issue without expensive human annotation, we proposed an unsupervised method for code-switching data augmentation. By utilizing a generative adversarial network, we can generate intra-sentential code-switching sentences from monolingual sentences. We applied the proposed method on two corpora, and the result shows that the generated code-switching sentences improve the performance of code-switching language models.

show abstract

Dual Language Models for Code Switched Speech Recognition

Cited by 20 publications

References 15 publications

Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training

Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training

Training Code-Switching Language Model with Monolingual Data

Code-Switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation

Contact Info

Product

Resources

About