High level knowledge sources in usable speech recognition systems

Abstract

The authors detail an integrated system which combines natural language processing with speech understanding in the context of a problem solving dialogue. The MINDS system uses a variety of pragmatic knowledge sources to dynamically generate expectations of what a user is likely to say.Understanding speech is a difficult problem. The ultimate goal of all speech recognition research is to create an intelligent assistant, who listens to what a user tells it and then carries out the instructions. An apparently simpler goal is the listening typewriter, a device which merely transcribes whatever it hears with only a few seconds delay. The listening typewriter seems simple, but in reality the process of transcription requires almost complete understanding as well. Today, we are still quite far from these ultimate goals. But progress is being made.One of the major problems in computer speech recognition and understanding is coping with large search spaces. The search space for speech recognition contains all the acoustic associated with words in the lexicon as well as all the legal word sequences. Today, the most widely used recognition systems are based on hidden Markov models (HMM) [Z]. In these systems, typically, each word is represented as a sequence of phonemes, and each phoneme is associated with a sequence of phonemes, and each phoneme is associated with a sequence of states. In general, the search space size increases as the size of the network of states increases. As search space size increases, speech recognition performance decreases. Knowledge can be used to constrain the exponential growth of a search space and hence increase processing speed and recognition accuracy [9, 171. Currently, the most common approach to constraining search space is to use a grammar. The grammars used for speech recognition constrain legal word sequences. Normally they are used in a strict left to right fashion and embody syntactic and semantic constraints on individual sentences. These constraints are represented in some form of probabilistic or semantic network which does not change from utterance to utterance [16-181. As we move toward habitable systems and spontaneous speech, the search space problem is greatly magnified. Habitable systems permit users to speak naturally. Grammars for naturally spoken sentences are significantly larger than the small grammars typically used by speech recognition systems. When one considers interjections, restarts and additional natural speech phenomena, the search space problem is further compounded. These problems point to the need for using knowledge sources beyond syntax and semantics to constrain the speech recognition process.There are many other knowledge sources besides syntax and semantics. Typically, these are clustered into the category of pragmatic knowledge. Pragmatic knowledge includes inferring and tracking plans, using context across clausal and sentence boundaries, determining local and global constraints on utterances and dealing with definite and pronominal reference. Work...

The role of voice input for human-machine communication.

“…The dialogue grammar approach models dialogue simply as a finite state transition network (59,60), in which state transitions occur on the basis of the type of communicative action that has taken place (e.g., a request). Such automata might be used to predict the next dialogue "states" that are likely, and thus could help speech recognizers by altering the probabilities of various lexical, syntactic, semantic, and pragmatic information (58,61). However, a number of drawbacks to the model are evident (62,63).…”

An evaluation of strategies for selectively verifying utterance meanings in spoken natural language dialog

“…The model of expectations used in the Circuit Fix-It Shop is similar to that of Young, Hauptmann, Ward, Smith and Werner (1989) in that the meanings of possible user responses are predicted based on the current dialog goal. The details of the system model can be found in Smith and Hipp (1994).…”

The Minds System: Using Context and Dialog to Enhance Speech Recognition

“…Using speech as an input medium for computer applications has resulted in two important findings. First, the grammars necessary to ensure some minimal coverage of a user's language have perplexities which are an order of magnitude larger than those used in today's high performing speech systems [18]. Second, the use of speech in problem solving tasks permits knowledge sources beyond the sentence level to be used to compensate for the extra search entailed by the increased perplexities.…”