Bayesian Learning of a Language Model from Continuous Speech

Abstract

SUMMARYWe propose a novel scheme to learn a language model (LM) for automatic speech recognition (ASR) directly from continuous speech. In the proposed method, we first generate phoneme lattices using an acoustic model with no linguistic constraints, then perform training over these phoneme lattices, simultaneously learning both lexical units and an LM. As a statistical framework for this learning problem, we use non-parametric Bayesian statistics, which make it possible to balance the learned model's complexity (such as the size of the learned vocabulary) and expressive power, and provide a principled learning algorithm through the use of Gibbs sampling. Implementation is performed using weighted finite state transducers (WFSTs), which allow for the simple handling of lattice input. Experimental results on natural, adult-directed speech demonstrate that LMs built using only continuous speech are able to significantly reduce ASR phoneme error rates. The proposed technique of joint Bayesian learning of lexical units and an LM over lattices is shown to significantly contribute to this improvement.

A summary of the 2012 JHU CLSP workshop on zero resource speech technologies and models of early language acquisition

“…The word segmentation part needs to allow for variability in the phonetic realization of words, but can also provide top-down pressure for sounds in similar contexts to be labeled as the same phone. Steps in this direction have been taken [16,31,32], but we are aware of no fully integrated system using speech data as input. It might also be possible to improve results on automatic tokenizations by identifying each token as, say, consonantal or vocalic.…”

Autonomous Learning of Representations

“…These algorithms, however, don't consider recognition errors in the phoneme sequences as we do. Word discovery on noisy phoneme lattices was also considered in [41], using similar methods. A comparison in [26] showed greatly improved F-scores of our proposed method compared to [41] for word n-grams greater than 1.…”

“…The second level is targeted at the discovery of the lexical units, the words, and learning their probabilities, the language model, from the phoneme sequences of the first level [55,41,25,26,39]. In speech recognition, the mapping of words to phoneme sequences is typically determined by a pronunication lexicon.…”

“…The maximization is carried out by Gibbs sampling, first sampling the word sequence from P (W|Y, G), calculated similar to (10), by keeping G constant in (13) and then the language model from P (G|W) [39] in an alternating and iterative fashion. The previous formulation can be extended to acoustic features X as input [41]. The maximization problem then becomes (Ŵ,Ĝ,Ŷ) = arg max…”

Learning a Lexicon and Translation Model from Phoneme Lattices

“…This is achieved using blocked Gibbs sampling, with each utterance constituting one block. To sample from WFSTs, we use forwardfiltering/backward-sampling (Scott, 2002;Neubig et al, 2012), creating forward probabilities using the forward algorithm for hidden Markov models before backward-sampling edges proportionally to the product of the forward probability and the edge weight. 3 3 No Metropolis-Hastings rejection step was used.…”