Leveraging Error Correction in Voice-based Text Entry by Talk-and-Gaze

Abstract

We present the design and evaluation of Talk-and-Gaze (TaG), a method for selecting and correcting errors with voice and gaze. TaG uses eye gaze to overcome the inability of voiceonly systems to provide spatial information. The user's point of gaze is used to select an erroneous word either by dwelling on the word for 800 ms (D-TaG) or by uttering a "select" voice command (V-TaG). A user study with 12 participants compared D-TaG, V-TaG, and a voice-only method for selecting and correcting words. Corrections were performed more than 20% faster with D-TaG compared to the V-TaG or voice-only methods. As well, D-TaG was observed to require 24% less selection effort than V-TaG and 11% less selection effort than voice-only error correction. D-TaG was well received in a subjective assessment with 66% of users choosing it as their preferred choice for error correction in voice-based text entry.

Inclusive Multimodal Voice Interaction for Code Navigation

“…gaze, keyboards, switches, etc. [12,29,33]) is also an important research area that could provide insights on how additional modalities can complement code navigation (e.g. using gaze to perform selections when users experience challenges in terms of speech recognition accuracy).…”

“…Participants were initially presented with one of these snippets and then required to navigate to each of the errors embedded within the file. To eliminate code comprehension time impacting on task completion, participants were provided with the exact steps required for navigating to errors (thus utilizing the same approach used in previous related studies [33,34]). For instance, at the start of each task, participants were initially shown a screenshot of the "starting" position of the cursor, alongside a second screenshot highlighting the location of a syntax error and the required final cursor position.…”

“…Initial research has examined simple inline navigation approaches via speech (e.g. moving the cursor to a specific position on a line [29,32]) and page scrolling through utilising a grid visualisation [42], but there has been less work on other crucial features such as the use of a call graph to navigate and locate specific code blocks [17,22,33]. Multimodal voice-controlled methods developed to date are also yet to be empirically evaluated with developers who have physical impairments thus contributing to a current lack of understanding around the potential of alternative methods to support code navigation.…”