Correcting recognition errors via discriminative utterance verification

Setlur, A.R.; Sukkar, R.A.; Jacob, J.

doi:10.1109/icslp.1996.607433

Cited by 25 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While many researchers have investigated the use of confidence scores, the primary focus has been on improving recognition accuracy (e.g. Setlur et al [1996]; Kemp and Schaaf [1997]; Mou and Zue [2000]; Hazen et al [2002]). Others discussed the use of confidence scores to support the detection of recognition errors, but these discussions typically focus on theoretical possibilities without reporting on the evaluation of implemented solutions (e.g.…”

Section: Introductionmentioning

confidence: 99%

Using confidence scores to improve hands-free speech based navigation in continuous dictation systems

Feng

Sears

2004

ACM Trans. Comput.-Hum. Interact.

View full text Add to dashboard Cite

Speech recognition systems have improved dramatically, but recent studies confirm that error correction activities still account for 66-75% of the users' time, and 50% of that time is spent just getting to the errors that need to be corrected. While researchers have suggested that confidence scores could prove useful during the error correction process, the focus is typically on error detection. More importantly, empirical studies have failed to confirm any measurable benefits when confidence scores are used in this way within dictation-oriented applications. In this article, we provide data that explains why confidence scores are unlikely to be useful for error detection. We propose a new navigation technique for use when speech-only interactions are strongly preferred and common, desktop-sized displays are available. The results of an empirical study that highlights the potential of this new technique are reported. An informal comparison between the current study and previous research suggests the new technique reduces time spent on navigation by 18%. Future research should include additional studies that compare the proposed technique to previous non-speech and speech-based navigation solutions.

show abstract

Section: Introductionmentioning

confidence: 99%

Using confidence scores to improve hands-free speech based navigation in continuous dictation systems

Feng

Sears

2004

ACM Trans. Comput.-Hum. Interact.

View full text Add to dashboard Cite

show abstract

“…It displays in particular the special case that only two hypotheses are considered, which reduces our approach to the standard "second-best" method using likelihood ratios between the firstand second-best path (cf. [32]). One observes that increasing the number of hypotheses considered results in a performance gain of the confidence measure and that, especially for obtaining low false-alarm rates our method clearly outperforms the "second-best" method.…”

Section: A Confidence Measures For Semantic Itemsmentioning

confidence: 93%

The thoughtful elephant: strategies for spoken dialog systems

Souvignier

Kellner

Rueber

et al. 2000

IEEE Trans. Speech Audio Process.

View full text Add to dashboard Cite

In this paper we present technology used in spoken dialog systems for applications of a wide range. They include tasks from the travel domain and automatic switchboards as well as large scale directory assistance. The overall goal in developing spoken dialog systems is to allow for a natural and flexible dialog flow similar to human-human interaction. This imposes the challenging task to recognize and interpret user input, where he/she is allowed to choose from an unrestricted vocabulary and an infinite set of possible formulations. We therefore put emphasis on strategies that make the system more robust while still maintaining a high level of naturalness and flexibility. In view of this paradigm, we found that two fundamental principles characterize many of the proposed methods: 1) to consider available sources of information as early as possible, and 2) to keep alternative hypotheses and delay the decision for a single option as long as possible.We describe how our system architecture caters to incorporating application specific knowledge, including, for example, database constraints, in the determination of the best sentence hypothesis for a user turn. On the next higher level, we use the dialog history to assess the plausibility of a sentence hypothesis by applying consistency checks with information items from previous user turns. In particular, we demonstrate how combination decisions over several turns can be exploited to boost the recognition performance of the system. The dialog manager can also use information on the dialog flow to dynamically modify and tune the system for the specific dialog situations. An important means to increase the "intelligence" of a spoken dialog system is to use confidence measures. We propose methods to obtain confidence measures for semantic items, whole sentences and even full N-best lists and give examples for the benefits obtained from their application. Experiences from field tests with our systems are summarized that have been found crucial for the system acceptance.Index Terms-Application specific knowledge, combined decisions, confidence measures, dialog history, natural language understanding, spoken dialog systems.

show abstract

“…Recognizer independent cues were also discovered by extracting word-(e.g., parsing mode) or utterance-level features (e.g., full/robust/no parse) from the output of syntactic or semantic parsers (Pao et al, 1998;Rayner et al, 1994) or from a maximum entropy based semantic-structured language model (Sarikaya et al, 2003). Cooccurrence analysis (Kaki et al, 1998;Sarma and Palmer, 2004;Voll et al, 2008) and alternative hypotheses (Mangu and Padmanabhan, 2001;Setlur et al, 1996;Zhou et al, 2006b) have also been applied in the context of recognition error detection and correction.…”

Section: Automatic Error Detectionmentioning

confidence: 98%

Third-party error detection support mechanisms for dictation speech recognition

Zhou

Shi

Sears

2010

Interacting with Computers

View full text Add to dashboard Cite

a b s t r a c tAlthough speech recognition has improved significantly in recent years, its adoption continues to be limited, in part, by the effort and frustration associated with correcting speech recognition errors. Error detection is a particularly challenging issue in third-party error correction where different individuals are responsible for the original dictation and correcting the resulting text. This research aims to address the difficulty experienced in third-party error detection by developing and evaluating a variety of support mechanisms. Drawing on a growing body of literature on human computer interaction and speech recognition, four support mechanisms were designed and evaluated, namely indexed audio, speech summarization, error prediction, and the presentation of alternative hypotheses. A user study assessed the impact of these support mechanisms on both performance and perceptions during error detection tasks. Performance measures included effectiveness and efficiency, and perception measures included confidence, perceived usefulness, and cognitive workload. The results provide strong support for the use of indexed audio in the context of third-party error detection. The results also confirm that consecutive error rate, or the percentage of recognition errors immediately adjacent to another error, has a negative impact on the effectiveness of third-party error detection. Other support mechanisms failed to improve either effectiveness or perceptions, but they did negate the negative impact as consecutive error rate increased. These findings have significant implications for speech recognition error detection research and the design of error detection support solutions.

show abstract

Correcting recognition errors via discriminative utterance verification

Cited by 25 publications

References 9 publications

Using confidence scores to improve hands-free speech based navigation in continuous dictation systems

Using confidence scores to improve hands-free speech based navigation in continuous dictation systems

The thoughtful elephant: strategies for spoken dialog systems

Third-party error detection support mechanisms for dictation speech recognition

Contact Info

Product

Resources

About