Auditory browser for blind and visually impaired users

Abstract

This paper presents our work on the development of a multimodal auditory interface which permits blind users to work more easily and efficiently with GUI browsers. A macro-analysis phase, which can be either passive or active, informs on the global layout of HTML documents. A subsequent active micro-analysis phase allows to explore particular elements of the document. The interface is based on : (1) a mapping of the graphical HTML document into a 3D virtual sound space environment, where non-speech auditory cues differentiate HTML elements; (2) the transcription into sound not only of text, but also of images; (3) the use of a touch-sensitive screen to facilitate user interaction. Moreover, in order to validate the sonification model of the images, we have created an audio "memory game", that can be used as a pedagogical tool to help blind pupils learn spatial exploration cues.

Conceptual Design Model of Assistive Courseware for Low Vision (AC4LV) Learners

“…Low vision learners and other types of VI depend 100% on audio to explain everything that appear on screen [34], [35] [36], [37]. Without auditory explanation, the visual aspect means nothing to them.…”

Enabling the blind to see gestures

“…In therapeutic, games have been used in conjunction with virtual reality to help people with special needs developing new skills [169], like going to the mall [125], simulating street crossing [124] and learning how to browse the internet [229]. The strategy was also used in rehabilitation like learning how to use a wheelchair [125] or a cane [169].…”