Improvements to N-gram Language Model Using Text Generated from Neural Language Model

Suzuki, Masayuki; Itoh, Nobuyasu; Nagano, Tohru; Kurata, Gakuto; Thomas, Samuel

doi:10.1109/icassp.2019.8683481

Cited by 22 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A language model is used to perform the decoding process on the network output for the speech-to-text conversion process. The decoding process is performed by calculating the probability of appearance of each word based on the exact word order [19]. This probability is calculated from the word chunks based on word order in the N-grams model.…”

Section: Language Modelmentioning

confidence: 99%

Speech-to-Text Conversion in Indonesian Language Using a Deep Bidirectional Long Short-Term Memory Algorithm

Dwijayanti¹,

Tami²,

Suprapto³

2021

IJACSA

View full text Add to dashboard Cite

Now-a-days, speech is used also for communication between humans and computers, which requires conversion from speech to text. Nevertheless, few studies have been performed on speech-to-text conversion in Indonesian language, and most studies on speech-to-text conversion were limited to the conversion of speech datasets with incomplete sentences. In this study, speech-to-text conversion of complete sentences in Indonesian language is performed using the deep bidirectional long short-term memory (LSTM) algorithm. Spectrograms and Mel frequency cepstral coefficients (MFCCs) were utilized as features of a total of 5000 speech data spoken by ten subjects (five males and five females). The results showed that the deep bidirectional LSTM algorithm successfully converted speech to text in Indonesian. The accuracy achieved by the MFCC features was higher than that achieved with the spectrograms; the MFCC obtained the best accuracy with a word error rate value of 0.2745% while the spectrograms were 2.0784%. Thus, MFCCs are more suitable than spectrograms as feature for speech-to-text conversion in Indonesian. The results of this study will help in the implementation of communication tools in Indonesian and other languages.

show abstract

Section: Language Modelmentioning

confidence: 99%

Speech-to-Text Conversion in Indonesian Language Using a Deep Bidirectional Long Short-Term Memory Algorithm

Dwijayanti¹,

Tami²,

Suprapto³

2021

IJACSA

View full text Add to dashboard Cite

show abstract

“…Sometimes these are called approximative models as they try to capture the knowledge of the neural model through their augmented training corpus. Suzuki et al [8] uses a domain balanced mixture of the training corpora to train a shallow RNNLM for text generation, and improve speech recognition results for Japanese, Korean and English. Wang et al [9] report using general domain pre-trained Transformer [10] to augment text corpora used to train LMs.…”

Section: Introduction and Problem Statementmentioning

confidence: 99%

“…Subword unit based ASR has been demonstrated to improve WER for several morphologically rich languages [12,13]. Suzuki et al [8] use subword approach for data augmentation to enrich text corpora to train BNLM, but compose these subwords back into words to prepare the final LM, unlike our approach that retokenizes words into subword units in the final LM.…”

Section: Introduction and Problem Statementmentioning

confidence: 99%

Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR

Tarján¹,

Szaszák²,

Fegyó³

et al. 2020

Preprint

View full text Add to dashboard Cite

Recently Deep Transformer models have proven to be particularly powerful in language modeling tasks for ASR. Their high complexity, however, makes them very difficult to apply in the first (single) pass of an online system. Recent studies showed that a considerable part of the knowledge of neural network Language Models (LM) can be transferred to traditional n-grams by using neural text generation based data augmentation. In our paper, we pre-train a GPT-2 Transformer LM on a general text corpus and fine-tune it on our Hungarian conversational call center ASR task. We show that although data augmentation with Transformer-generated text works well for isolating languages, it causes a vocabulary explosion in a morphologically rich language. Therefore, we propose a new method called subword-based neural text augmentation, where we retokenize the generated text into statistically derived subwords. We show that this method can significantly reduce the WER while greatly reducing vocabulary size and memory requirements. Finally, we also show that subword-based neural text augmentation outperforms the word-based approach not only in terms of overall WER but also in recognition of OOV words.

show abstract

“…Although by splitting the decoding into two parts we can leverage knowledge of the NNLMs and demonstrate significant Word Error Rate Reduction (WERR), it also introduces considerable processing delay [4,5,12]. Therefore, techniques exploiting the capabilities of NNLMs in a single-pass decoding approach have received particular attention recently [9,13]. A possible technique is to augment the in-domain training data with a large text corpus generated by an NNLM [1,3].…”

Section: Introductionmentioning

confidence: 99%

“…In a related work, Suzuki et al [13] use a domain balanced mixture of the training corpora to train a shallow RNNLM for text generation and improve speech recognition results for Japanese, Korean and English tasks. For Korean subword-based language models are also utilized, but only for text generation, since in the language model of the ASR system subwords are merged.…”

Section: Introductionmentioning

confidence: 99%

On the Effectiveness of Neural Text Generation based Data Augmentation for Recognition of Morphologically Rich Speech

Tarján,

Szaszák,

Fegyó

et al. 2020

Preprint

View full text Add to dashboard Cite

Advanced neural network models have penetrated Automatic Speech Recognition (ASR) in recent years, however, in language modeling many systems still rely on traditional Back-off N-gram Language Models (BNLM) partly or entirely. The reason for this are the high cost and complexity of training and using neural language models, mostly possible by adding a second decoding pass (rescoring). In our recent work we have significantly improved the online performance of a conversational speech transcription system by transferring knowledge from a Recurrent Neural Network Language Model (RNNLM) to the single pass BNLM with text generation based data augmentation. In the present paper we analyze the amount of transferable knowledge and demonstrate that the neural augmented LM (RNN-BNLM) can help to capture almost 50% of the knowledge of the RNNLM yet by dropping the second decoding pass and making the system real-time capable. We also systematically compare word and subword LMs and show that subword-based neural text augmentation can be especially beneficial in under-resourced conditions. In addition, we show that using the RNN-BNLM in the first pass followed by a neural second pass, offline ASR results can be even significantly improved.

show abstract

Improvements to N-gram Language Model Using Text Generated from Neural Language Model

Cited by 22 publications

References 18 publications

Speech-to-Text Conversion in Indonesian Language Using a Deep Bidirectional Long Short-Term Memory Algorithm

Speech-to-Text Conversion in Indonesian Language Using a Deep Bidirectional Long Short-Term Memory Algorithm

Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR

On the Effectiveness of Neural Text Generation based Data Augmentation for Recognition of Morphologically Rich Speech

Contact Info

Product

Resources

About