Interpolation of n-gram and mutual-information based trigger pair language models for Mandarin speech recognition

Zhou, Guodong; Lua, K.T.

doi:10.1006/csla.1998.0118

Cited by 20 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The main idea of the TR-classifier is based on computing the average mutual information of each couple of words which belong to the vocabulary V i . Couples of words or "triggers" that are considered important for a topic identification task, are those which have the highest average mutual information (AM I) values [8,18]. Each topic is then endowed with a number of selected triggers M , calculated using training corpora of topic T i .…”

Section: An Overview On Tr-classifiermentioning

confidence: 99%

Efficiency of TR-Classifier versus TFIDF

Abbas¹,

Smaïli

Berkani

2010

2010 First International Conference on Integrated Intelligent Computing

View full text Add to dashboard Cite

In this paper, we present a method of topic identification based on computing triggers pairs: TR-classifier (TRiggers-based classifier). Indeed, it is used for the purpose to identify topics of texts. Hence, the first step to be realized is the construction of a vocabulary for each topic. Topic vocabularies are composed of words ranked according to their frequencies from the maximum to the minimum. We note that the size of each topic vocabulary is 400. For each word of the vocabulary, average mutual information (AMI) is calculated. The used triggers are selected according to the highest AMI values. In order to evaluate the TR-Classifier, we compared it to the well-known TFIDF.

show abstract

Section: An Overview On Tr-classifiermentioning

confidence: 99%

Efficiency of TR-Classifier versus TFIDF

Abbas¹,

Smaïli

Berkani

2010

2010 First International Conference on Integrated Intelligent Computing

View full text Add to dashboard Cite

show abstract

“…On the one hand, the Normalized Mutual Information (NMI), previously used for the estimation of parameters of acoustic models for speech recognition [1], or for the adaptation of trigger-based LMs [5]. On the other hand, a minimization of the global perplexity of a LM obtained as the interpolation of all the clusters considered.…”

Section: Introductionmentioning

confidence: 99%

Mutual Information and Perplexity Based Clustering of Dialogue Information for Dynamic Adaptation of Language Models

Lucas-Cuesta

Fernández-Martínez

Moreno

et al. 2012

Advances in Speech and Language Technologies for Iberian Languages

View full text Add to dashboard Cite

Abstract. We present two approaches to cluster dialogue-based information obtained by the speech understanding module and the dialogue manager of a spoken dialogue system. The purpose is to estimate a language model related to each cluster, and use them to dynamically modify the model of the speech recognizer at each dialogue turn. In the first approach we build the cluster tree using local decisions based on a Maximum Normalized Mutual Information criterion. In the second one we take global decisions, based on the optimization of the global perplexity of the combination of the cluster-related LMs. Our experiments show a relative reduction of the word error rate of 15.17%, which helps to improve the performance of the understanding and the dialogue manager modules.

show abstract

“…Language modeling is the attempt to characterize, capture and exploit the regularities and constraints in natural language. Among various language modeling approaches, ngram modeling has been widely used in many applications, such as speech recognition, machine translation (Katz 1987;Jelinek 1989;Gale and Church 1990;Brown et al 1992;Yang et al 1996;Bai et al 1998;Zhou et al 1999;Rosenfeld 2000;Gao et al 2002). Although ngram modeling is simple in nature and easy to use, it has obvious deficiencies.…”

Section: Introductionmentioning

confidence: 99%

“…Psychological experiments in Meyer et al (1975) indicated that the human's reaction to a highly associated word pair was stronger and faster than that to a poorly associated word pair. Such preference information is very useful for natural language processing (Church et al 1990;Hiddle et al 1993;Rosenfeld 1994;Zhou et al1998;Zhou et al 1999). Obviously, the preference relationships between words can expand from a short to long distance.…”

Section: Introductionmentioning

confidence: 99%

Modeling of long distance context dependency in Chinese

Zhou¹

2003

Proceedings of the Second SIGHAN Workshop on Chinese Language Processing -

View full text Add to dashboard Cite

Ngram modeling is simple in language modeling and has been widely used in many applications. However, it can only capture the short distance context dependency within an N-word window where the largest practical N for natural language is three. In the meantime, much of context dependency in natural language occurs beyond a three-word window. In order to incorporate this kind of long distance context dependency, this paper proposes a new MI-Ngram modeling approach. The MI-Ngram model consists of two components: an ngram model and an MI model. The ngram model captures the short distance context dependency within an N-word window while the MI model captures the long distance context dependency between the word pairs beyond the N-word window by using the concept of mutual information. It is found that MI-Ngram modeling has much better performance than ngram modeling. Evaluation on the XINHUA new corpus of 29 million words shows that inclusion of the best 1,600,000 word pairs decreases the perplexity of the MI-Trigram model by 20 percent compared with the trigram model. In the meanwhile, evaluation on Chinese word segmentation shows that about 35 percent of errors can be corrected by using the MI-Trigram model compared with the trigram model.

show abstract

Interpolation of n-gram and mutual-information based trigger pair language models for Mandarin speech recognition

Cited by 20 publications

References 14 publications

Efficiency of TR-Classifier versus TFIDF

Efficiency of TR-Classifier versus TFIDF

Mutual Information and Perplexity Based Clustering of Dialogue Information for Dynamic Adaptation of Language Models

Modeling of long distance context dependency in Chinese

Contact Info

Product

Resources

About