High-level approaches to confidence estimation in speech recognition

Abstract

Abstract-We describe some high-level approaches to estimating confidence scores for the words output by a speech recognizer. By "high-level" we mean that the proposed measures do not rely on decoder specific "side information" and so should find more general applicability than measures that have been developed for specific recognizers. Our main approach is to attempt to decouple the language modeling and acoustic modeling in the recognizer in order to generate independent information from these two sources that can then be used for estimation of confidence. We isolate these two information sources by using a phone recognizer working in parallel with the word recognizer. A set of techniques for estimating confidence measures using the phone recognizer output in conjunction with the word recognizer output is described. The most effective of these techniques is based on the construction of "metamodels," which generate alternative word hypotheses for an utterance. An alternative approach requires no other recognizers or extra information for confidence estimation and is based on the notion that a word that is semantically "distant" from the other decoded words in the utterance is likely to be incorrect. We describe a method for constructing "semantic similarities" between words and hence estimating a confidence. Results using the U.K. version of the Wall Street Journal are given for each technique.

Modelling Errors in Automatic Speech Recognition for Dysarthric Speakers

“…Instead, a hidden Markov model (HMM) is constructed for each of the phonemes in the phoneme inventory. We term these HMMs metamodels [40]. The function of a metamodel is best understood by comparison with a "standard" acoustic HMM: a standard acoustic HMM estimates Pr(O | p j ), where O is a subsequence of the complete sequence of observed acoustic vectors in the utterance, O, and p j is a postulated phoneme in P. A metamodel estimates Pr( P | p j ), where P is a subsequence of the complete sequence of observed (decoded) phonemes in the utterance P.…”

Recognition confidence scoring and its use in speech understanding systems

“…Because the raw acoustic scores are usually not particularly useful as confidence measures when used by themselves [1], methods for normalizing these scores are typically employed [3,8,13]. In this work all of the acoustic scores produced at the phonetic level are normalized against a catch-all model.…”

Speaker model adaptation based on confidence score

“…Confidence measures are used on various applications in speech recognition field. Experiments in large vocabulary continuous speech recognition, reported in [1] and [2], show that the use of confidence measure for constructing a word graph significantly increases the recognition performance. In [4] application of confidence measure in language identification task is explained.…”