Interactive visualisation techniques for dynamic speech transcription, correction and training

Abstract

As performance gains in automatic speech recognition systems plateau, improvements to existing applications of speech recognition technology seem more likely to come from better user interface design than from further progress in core recognition components. Among all applications of speech recognition, the usability of systems for transcription of spontaneous speech is particularly sensitive to high word error rates. This paper presents a series of approaches to improving the usability of such applications. We propose new mechanisms for error correction, use of contextual information, and use of 3D visualisation techniques to improve user interaction with a recogniser and maximise the impact of user feedback. These proposals are illustrated through several prototypes which target tasks such as: off-line transcript editing, dynamic transcript editing, and real-time visualisation of recognition paths. An evaluation of our dynamic transcript editing system demonstrates the gains that can be made by adding the corrected words to the recogniser's dictionary and then propagating the user's corrections.

Interactive Transcription of Old Text Documents

“…The results showed that current errors are mostly produced by the language structure as well as the vocabulary of the old text documents presented. Finally, results showed that these tasks are perfectly suited for an interactive approach, as its automatic transcription is far from perfect, but within the range in which user interaction may be useful (Luz et al, 2008). It must be noted that, the presented results in this chapter are better than those previously published in each database paper .…”

“…Specifically, the objective of these tools is to facilitate the transcription to the user, when compared with the tedious manual transcription. Other more refined approaches in ASR locate errors and pass them to the user (Luz et al, 2008), further reducing the effort (Hakkani-Tür et al, 2006). CAT approaches are not new either in the transcription of old text documents.…”

“…The basic features of these approaches are the use of adequate interactive tools to navigate the image, together with an error detection tool, that highlights possibly incorrect words to the user . However, if errors occur frequently, it could be better to ignore the system output and complete the task manually (H. Nanjo and T.Kawahara, 2006;Luz et al, 2008). A bettter idea is to employ user interaction beyond the simple correction of the system output.…”

“…supervision of correctly recognised words does not cost the same that incorrectly recognised words. Furthermore, even though experiments with real users have sometimes shown that interactive approaches were meaningless (Luz et al, 2008), we are certain that this approach will be effective under certain assumptions. For instance, H. Nanjo and T.Kawahara (2006) show that a recognised transcription should have at most 25% of error for an interactive transcription approach to be applied.…”

“…In these applications, the system propose a transcription to the user, who revise or supervise the system output and corrects it if any errors were found. Similarly, this interactive approach has also been applied on top of other PR systems in very different fields, such as ASR (Luz et al, 2008) or machine translation (MT) (Barrachina et al, 2009), showing improvements over manual approaches. Figure 3.1 depicts a diagram showing the post-editing process for transcribing text documents.…”