Tactile Graphics with a Voice

Abstract

We discuss the development of Tactile Graphics with a Voice (TGV), a system used to access label information in tactile graphics using QR codes. Blind students often rely on tactile graphics to access textbook images. Many textbook images have a large number of text labels that need to be made accessible. In order to do so, we propose TGV, which uses QR codes to replace the text, as an alternative to Braille. The codes are read with a smartphone application. We evaluated the system with a longitudinal study where 10 blind and low-vision participants completed tasks using three different modes on the smartphone application: (1) no guidance, (2) verbal guidance, and (3) finger-pointing guidance. Our results show that TGV is an effective way to access text in tactile graphics, especially for those blind users who are not fluent in Braille. We also found that preferences varied greatly across the modes, indicating that future work should support multiple modes. We expand upon the algorithms we used to implement the finger pointing, algorithms to automatically place QR codes on documents. We also discuss work we have started on creating a Google Glass version of the application.

Visually Augmented Audio-Tactile Graphics for Visually Impaired People

“…In contrast, there was a significant decrease in the errors in the rectangles with 6 cavities (z = −1.83, p = 0.034) and 8 cavities (z = −2.50, p = 0.006). Together with the results of the more complex graphs of Tests 4 and 5, this could be an indication that the cognitive load (e.g., Reference [1]) of the subjects can be reduced by the additional visual support. A further study could examine further in detail the extent to which more complex graphics can be mediated by supporting the residual vision of visually impaired people.…”

“…Baker et al [1] used manually attached visual markers (QR-Codes) encoding text in tactile graphics to assign verbal explanations to dedicated graphical elements. Standard smartphones were used to decode the marker the user was pointing at with his or her finger and to read aloud the corresponding explanation.…”

“…All of the participants were able to place the tactile map on the tablet's display and to use the capacitive code for detection of the map. When the users relocated the map, this was detected by the application and the correct angle was reported (1). All of the participants were successful, none of the users needed numerous trials, which resulted in registration timings considerably less than 10s.…”

Accessible Tutoring Platform Using Audio-Tactile Graphics Adapted for Visually Impaired People

“…Another attitude presents applying a method of visual tracking of activities performed by the user (movement of the fingers on a tactile picture) using cameras and dedicated image recognition software. The elements in the picture touched by the user have special labels containing QR codes, and they are identified using artificial intelligence (AI) methods for image processing [12][13][14][15]. It provides us with the opportunity to hear the audio description of the picture content.…”

Torino: A Tangible Programming Language Inclusive of Children with Visual Disabilities

“…For example, Petit et al focus on capturing images that are in school books through the design of specialized hardware that creates a refreshable tactile display (Petit, Dufresne, & Levesque, 2008). Baker et al (2016) focus on the challenge of the label information that surrounds tactile graphics such as charts, which can be become unwieldly in braille. They propose a solution that uses audio labels provided through QR codes.…”