Auditory substitution of vision: pattern recognition by the blind

Abstract

Pattern recognition in a computer environment was investigated in 6 early blind and 6 blindfolded sighted subjects using auditory substitution of vision. Subjects had to scan visual patterns displayed on a PC screen by moving the pen of a graphics tablet, which lead to corresponding displacements of the cursor on the screen. A small screen area centered on the pointer was then translated into sounds according to a visual-auditory transcription code. Subjects were trained to learn this code during 12 one-hour sessions. Performance of both groups signi®cantly increased with practice. This indicates that mental representations of visual patterns can be acquired through the auditory channel, even in the absence of visual experience. Moreover, blind subjects performed signi®cantly better than sighted subjects did. This could be interpreted as a result of partial compensation for their loss of vision. Pattern recognition in a computer environment is thus possible using a fairly natural vision-to-audition coding scheme.

Perception With Compensatory Devices: From Sensory Substitution to Sensorimotor Extension

“…Furthermore, in order to enhance the similarity with the human visual system, the receptor field of the PSVA has a higher resolution in the center of the picture. Studies that have been conducted with auditory devices have shown the possibility of object localization (Auvray et al, 2007a;Renier et al, 2005a) and form recognition (Arno et al, 1999;Arno et al, 2001 with the PSVA; Cronly- Dillon et al, 1999Dillon et al, , 2000Pollok, Schnitzler, Mierdorf, Stoerig, & Schnitzler, 2005 with the Voice). Interestingly, recent studies have also demonstrated the possibility of recreating visual illusions with the PSVA (Renier et al, 2005b).…”

“…Pollok and his colleagues (Pollok et al, 2005) reported that participants' performance in the recognition of two-dimensional natural objects with the Voice improved with practice, even if the practice occurred with three-dimensional objects. In addition, studies with the PSVA showed that early blind individuals are able to recognize simple two-dimensional visual patterns thanks to their auditory conversion, with better performance than that of blindfolded sighted participants (Arno et al, 2001). These results reveal that blind persons can use SSDs in order to localize and recognize objects on the basis of light alone, although their performance might not reach the same level as that of sighted persons using vision.…”

Mobile Video-to-Audio Transducer and Motion Detection for Sensory Substitution

“…Another system that started slightly later is called Prosthesis Substituting Vision with Audition (PSVA) (Capelle et al, 1998;Arno et al, 1999Arno et al, , 2001. The main differences with this project compared to the previous one are an on-line sonification without left-to-right scans, and frequency encoding depending on both horizontal and vertical positions of the pixels.…”

The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms

“…Although visually impaired people tend to perform better (Arno et al, 2001) and undergo functional changes to their brains (Kupers, Chebat, Madsen, Paulson, & Ptito, 2010;Ortiz et al, 2011) blindfolded sighted participants can complete sensory substitution tasks and are not necessarily qualitatively different despite being quantitatively worse. Finally, such devices may be useful in the sighted population itself by offering a dual-coding of vision; i.e.…”