Language model representations for beam-search decoding

Antoniol, Giuliano; Brugnara, Fabio; Cettolo, Mauro; Federico, Marcello

doi:10.1109/icassp.1995.479666

Cited by 33 publications

(29 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All selection criteria have been empirically evaluated on a 10,000-word Italian newspaper dictation task [1]. Trigram selection methods have been applied for generating LMs of dierent sizes.…”

Section: Methodsmentioning

confidence: 99%

“…The u t ility o f i n troducing trigrams in a LM can also be measured with t h e amount of information about t h e current word (wt) gained by a ugmenting t h e context from (wt 1 Equations (9) and (10) explain the i n dividual contribution to the global information of contexts of type y and xy, respectively. T h e following score functions can thus be considered:…”

Section: Mutual Informationmentioning

confidence: 99%

“…The same LMs were compared in terms of recognition accuracy by employing t h em in a speaker independent, continuous speech recognizer [1]. In this case, the size of the t esting material is not comparable with t h a t used to m easure perplexity.…”

Section: Recognition Testsmentioning

confidence: 99%

See 2 more Smart Citations

Techniques for approximating a trigram language model

Brugnara

Federico²

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

Self Cite

View full text Add to dashboard Cite

In this paper several methods are proposed for reducing the size of a trigram language model (LM), which is often the biggest data structure in a continuous speech recognizer, without aecting its performance. The common factor shared by t h e dierent a p proaches is to select only a subset of the a v ailable trigrams, trying t o i d entify those trigrams that mostly contribute t o t h e performance of the full trigram LM. The proposed selection criteria apply to trigram contexts, both o f l e n gth o n e o r t w o. These criteria rely on information theory concepts, the back-o probabilities estimated by t h e LM, or on a measure of the p h onetic/linguistic uncertainty relative t o a given context. Performance of the reduced trigrams LMs are compared both i n t erms of perplexity a n d recognition accuracy. Results show t h a t all the considered methods perform better than the n aive frequency shifting m ethod. In fact, a 50% size reduction is obtained on a shift-1 trigram LM, at t h e cost of a 5% increase in word error rate. Moreover, the r e d u ced LMs improve b y around 15% the w ord error rate of a bigram LM of the same size.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Mutual Informationmentioning

confidence: 99%

See 1 more Smart Citation

Techniques for approximating a trigram language model

Brugnara

Federico²

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, for continuous speech recognition systems using higher order language models the linguistic state cannot be determined locally and the word boundaries are uncertain. Several solutions based on creating copies of the PPT for each unique linguistic context solve this problem [8,9,10], however these approaches create redundant sub-tree computations, the number of which correspond to the number of active linguistic contexts. A computation is redundant when a sub-tree instance is dominated by another instance of that sub-tree.…”

Section: Re-entrant Vs Non Re-entrant Treesmentioning

confidence: 99%

“…To overcome this difficulty Antoniol et.al. [9] used successor trees where higher order language model probabilities are distributed over the appropriate tree. However successor trees do not eliminate the redundant computations revealed by subtree dominance.…”

Section: Using Lm Probabilities In a Pptmentioning

confidence: 99%

Evaluation of a language model using a clustered model backoff

Miller

Alleva²

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

View full text Add to dashboard Cite

In this paper, we describe and evaluate a language model using word classes automatically generated from a word clustering algorithm. Class based language models have been shown to be effective for rapid adaptation, training on small datasets, and reduced memory usage. In terms of model perplexity, prior work has shown diminished returns for class based language models constructed using very large training sets. This paper describes a method of using a class model as a backoff to a bigram model which produced significant benefits even when trained from a large text corpus. Tests results on the Whisper continuous speech recognition system show that for a given word error rate, the clustered bigram model uses 2/3 fewer parameters compared to a standard bigram model using unigram backoff.

show abstract

Compressing the factoring table and performing garbage collection on unusable word hypotheses in a continuous speech recognition system

Kokubo¹,

Hayashi²,

Yamamoto³

et al. 2005

Electron. Comm. Jpn. Pt. III

View full text Add to dashboard Cite

SUMMARYWe have investigated methods for reducing the number of word hypotheses registered in the word graph and the amount of memory used by the factoring tables for the tree-structured dictionary with the objective of reducing the memory requirements of a continuous speech recognition system. By assigning word hypotheses in the word graph attributes relating to the number of continuation hypotheses in which they are included, we are able to efficiently determine unusable word hypotheses during pruning and can perform garbage collection. This procedure allows us to reduce the amount of memory needed for generating word hypotheses from 127 MB to 6.9 MB. In addition, by approximating the bigram values held in the factoring tables with POS bigrams, we were able to reduce the memory consumption of the factoring tables from 56 MB to 19 MB with almost no impairment of recognition performance. As a result of these reductions in memory requirements, the memory consumption of the decoder has been reduced from 246 MB to 113 MB.

show abstract

Language model representations for beam-search decoding

Cited by 33 publications

References 7 publications

Techniques for approximating a trigram language model

Techniques for approximating a trigram language model

Evaluation of a language model using a clustered model backoff

Compressing the factoring table and performing garbage collection on unusable word hypotheses in a continuous speech recognition system

Contact Info

Product

Resources

About