'Early recognition' of words in continuous speech

Abstract

In this paper, we present an automatic speech recognition (ASR) system based on the combination of an automatic phone recogniser and a computational model of human speech recognition -SpeM -that is capable of computing 'word activations' during the recognition process, in addition to doing normal speech recognition, a task in which conventional ASR architectures only provide output after the end of an utterance. We explain the notion of word activation and show that it can be used for 'early recognition', i.e. recognising a word before the end of the word is available.Our ASR system was tested on 992 continuous speech utterances, each containing at least one target word: a city name of at least two syllables. The results show that early recognition was obtained for 72.8% of the target words that were recognised correctly. Also, it is shown that word activation can be used as an effective confidence measure.

How Should a Speech Recognizer Work?

“…For each word and each path, SpeM computes an activation value: As long as a word is consistent with the acoustic input, its activation grows. Scharenborg et al (2003a) showed that this feature of SpeM allows the model to recognize words before their acoustic offset. Continuous and early recognition measures could be of considerable value in ASR systems, which often do not provide online recognition measures.…”

“…Then, the posterior probabilities for word W itself and for the best path on which W lies on the basis of the word's score (based on the acoustic score and penalties for insertions, deletions, and substitutions) are calculated. The details on how these probabilities are computed are given in Scharenborg et al (2003a). The key components of these computations, however, are as follows:…”

“…Using Bayes's rule to rank competitors is, for instance, also used by Jurafsky (1996) in his probabilistic model of lexical and syntactic access and disambiguation. The probability P(W|X) is the foundation on which we base the calculation of word activation (Scharenborg, ten Bosch, & Boves, 2003a).…”

Parallels between HSR and ASR: how ASR can contribute to HSR

“…(1) P( X ) in which P(W) is the prior probability of W, and P(X) denotes the prior probability of observing the signal X (for details, see [14]). Bayes' Rule and the probability P(W\X) play a central role in the mathematical framework on which statistical pattern matching techniques are built (i.e., most ASR implementations).…”

Acoustic Scores and Symbolic Mismatch Penalties in Phone Lattices

“…SpeM is a multi-pass decoder in which a phone recognizer in the first pass generates a phone lattice that is used in the subsequent lexical search module. SpeM has been used to successfully model a number of key results from psycho-linguistic experiments [8] [9]. In SpeM, mismatches between the phone sequences in the lattice and the phone representations (originating) from the lexicon are dealt with in a more flexible manner than in previous computational models of human auditory word recognition.…”