Topic n-gram count language model adaptation for speech recognition

Abstract

We introduce novel language model (LM) adaptation approaches using the latent Dirichlet allocation (LDA) model. Observed n-grams in the training set are assigned to topics using soft and hard clustering. In soft clustering, each n-gram is assigned to topics such that the total count of that n-gram for all topics is equal to the global count of that n-gram in the training set. Here, the normalized topic weights of the n-gram are multiplied by the global n-gram count to form the topic n-gram count for the respective topics. In hard clustering, each n-gram is assigned to a single topic with the maximum fraction of the global n-gram count for the corresponding topic. Here, the topic is selected using the maximum topic weight for the n-gram. The topic n-gram count LMs are created using the respective topic n-gram counts and adapted by using the topic weights of a development test set. We compute the average of the confidence measures: the probability of word given topic and the probability of topic given word. The average is taken over the words in the n-grams and the development test set to form the topic weights of the n-grams and the development test set respectively. Our approaches show better performance over some traditional approaches using the WSJ corpus.

What to Expect from Expected Kneser-Ney Smoothing

“…Also, notice that in this setup, expected WB achieved consistent improvements over fractional WB. A similar experiment was carried out with other training corpora and seed LMs, including LDA-based seeds like in [16] with analogous results.…”

Contributions to speech analytics based on speech recognition and topic identification

“…In this regard, there are different ways in which the interpolation weight can be selected: it can be set empirically by minimizing the perplexity in a development stage with data not seen during training [Clarkson, 1999, Tur and; it can also be estimated under some optimization algorithm, such as Expectation Maximization [Daumé et al, 2010] or Maximum A Posteriori (MAP) adaptation [Wang and Stolcke, 2007]; or it can be set dynamically depending on the current situation of the interaction (the topic of the speech, a specific speaker, etc.) [Haidar andO'Shaughnessy, 2012, Seymore andRosenfeld, 1997].…”

Sequence-based detection of sleeping cell failures in mobile networks

“…for natural language processing and text analysis applications such as speech recognition, parsing, spelling, etc. [7,51,9,33,41]. In addition, N-gram is applied for whole-genome protein sequences [26] and for computer virus detection [17,24].…”