Recurrent neural network language modeling for code switching conversational speech

Adel, Heike; Vu, Ngoc Thang; Kraus, Franziska; Schlippe, Tim; Li, Haizhou; Schultz, Tanja

doi:10.1109/icassp.2013.6639306

Cited by 75 publications

(63 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the well-established research line in linguistics, implications of CS and other kinds of language switches for speechto-text systems have recently received some research interest, resulting in some robust acoustic modeling [1][2][3][4][5] and language modeling [6][7][8] approaches for CS speech. Language identification (LID) is a relevant task for the automatic speech recognition (ASR) of CS speech [9][10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Code-switching detection using multilingual DNNS

Yılmaz

Heuvel

Leeuwen

2016

2016 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Automatic speech recognition (ASR) of code-switching speech requires careful handling of unexpected language switches that may occur in a single utterance. In this paper, we investigate the feasibility of using multilingually trained deep neural networks (DNN) for the ASR of Frisian speech containing code-switches to Dutch with the aim of building a robust recognizer that can handle this phenomenon. For this purpose, we train several multilingual DNN models on Frisian and two closely related languages, namely English and Dutch, to compare the impact of single-step and two-step multilingual DNN training on the recognition and code-switching detection performance. We apply bilingual DNN retraining on both target languages by varying the amount of training data belonging to the higher-resourced target language (Dutch). The recognition results show that the multilingual DNN training scheme with an initial multilingual training step followed by bilingual retraining provides recognition performance comparable to an oracle baseline recognizer that can employ language-specific acoustic models. We further show that we can detect code-switches at the word level with an equal error rate of around 17% excluding the deletions due to ASR errors.

show abstract

Section: Introductionmentioning

confidence: 99%

Code-switching detection using multilingual DNNS

Yılmaz

Heuvel

Leeuwen

2016

2016 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

show abstract

“…Especially, we design a novel feature by applying recurrent neural network language model (RNNLM) to OCR confusion networks (c-nets). Since its first notable application in speech recognition [9], RNNLM has gained increased attention by its success in a variety of tasks, such as speech recognition, machine translation, OCR, and keyword spotting [9][10][11][12][13]. To the best of our knowledge, our work is the first application of RNNLM in OCR error detection and shows it effectively improves error detection rate.…”

Section: Introductionmentioning

confidence: 94%

“…;/ $%/0+.5 0+ compute the LM scores. For this reason, works using RNNLM often applied it to n-best rescoring because it is practically difficult to efficiently utilize wide context (more than 3-gram) on lattices [9][10][11]13]. Works that applied RNNLM to lattices usually made approximations to compute the LM scores [12, [27][28][29].…”

Section: A Recurrent Neural Network Language Modelmentioning

confidence: 99%

Confusion Network Based Recurrent Neural Network Language Modeling for Chinese OCR Error Detection

Chen¹,

Wu²,

Cao

et al. 2014

2014 22nd International Conference on Pattern Recognition

View full text Add to dashboard Cite

This paper presents a new framework for OCR error detection, which uses a conditional random field model to combine rich features from multiple sources, including confusion networks (c-nets), lexical local context and recurrent neural network language model (RNNLM) 1 . We propose a novel, efficient method for computing character-level c-net based RNNLM scores by using dynamic programming and c-net partial unfolding. Our experiments show that our error detection model has consistent observable improvements over a high baseline employed by our current OCR demo system, as measured by average precision and detection error trade-off curve on two test sets of Chinese image documents. Both linguistic and recognition features contribute to the high performance, with the former especially informative. In addition, we show that the new feature we proposed, the c-net RNNLM feature, plays a remarkable beneficial role in improving error detection rate. These results suggest that applications on top of image text recognition can benefit substantially from a hybrid strategy that combines techniques from optical character recognition and natural language processing.

show abstract

“…Impact of CS and other kinds of language switches on the speech-to-text systems have recently received research interest, resulting in several robust acoustic modeling [2][3][4][5][6][7][8] and language modeling [9][10][11] approaches for CS speech. Language identification (LID) is a relevant task for the automatic speech recognition (ASR) of CS speech [12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

Exploiting Untranscribed Broadcast Data for Improved Code-Switching Detection

2017

View full text Add to dashboard Cite

We have recently presented an automatic speech recognition (ASR) system operating on Frisian-Dutch code-switched speech. This type of speech requires careful handling of unexpected language switches that may occur in a single utterance. In this paper, we extend this work by using some raw broadcast data to improve multilingually trained deep neural networks (DNN) that have been trained on 11.5 hours of manually annotated bilingual speech. For this purpose, we apply the initial ASR to the untranscribed broadcast data and automatically create transcriptions based on the recognizer output using different language models for rescoring. Then, we train new acoustic models on the combined data, i.e., the manually and automatically transcribed bilingual broadcast data, and investigate the automatic transcription quality based on the recognition accuracies on a separate set of development and test data. Finally, we report code-switching detection performance elaborating on the correlation between the ASR and the code-switching detection performance.

show abstract

Recurrent neural network language modeling for code switching conversational speech

Cited by 75 publications

References 11 publications

Code-switching detection using multilingual DNNS

Code-switching detection using multilingual DNNS

Confusion Network Based Recurrent Neural Network Language Modeling for Chinese OCR Error Detection

Exploiting Untranscribed Broadcast Data for Improved Code-Switching Detection

Contact Info

Product

Resources

About