On the benefits of confidence visualization in speech recognition

Abstract

In a typical speech dictation interface, the recognizer's bestguess is displayed as normal, unannotated text. This ignores potentially useful information about the recognizer's confidence in its recognition hypothesis. Using a confidence measure (which itself may sometimes be inaccurate), we investigated providing visual feedback about low-confidence portions of the recognition using shaded, red underlining. An evaluation showed, compared to a baseline without underlining, underlining lowconfidence areas did not increase user's speed or accuracy in detecting errors. However, we found that when recognition errors were correctly underlined, they were discovered significantly more often than baseline. Conversely, when errors failed to be underlined, they were discovered less often. Our results indicate confidence visualization can be effective -but only if the confidence measure has high accuracy. Further, since our results show that users tend to trust confidence visualization, designers should be careful in its application if a high accuracy confidence measure is not available.

Qualitative investigation of the display of speech recognition results for communication with deaf people

“…It was difficult for them to assess whether the information was to be trusted. The same difficulties have been observed in [13], in an experiment in which hearing people dictated a text and then had to detect the errors made by the speech recognition.…”

Five Challenges for Intelligent Text Entry Methods

“…In conclusion, until confidence scores are more accurate they are unlikely to aid users. We have shown that if confidence scores are accurate enough they do indeed benefit users (Vertanen and Kristensson 2008). However, to reach such levels it may be necessary to incorporate external knowledge outside the speech recognizer.…”

None of a CHInd

“…Yet there has been speech technology and HCI collaboration since the Starner [13] and Shneiderman [12] reviews, for example [16,15]. Why is none of this more recent work referenced?…”