This paper presents an experimental system for the conversion of images into sound patterns. The system was designed to provide auditory image representations within some of the known limitations of the human hearing system, possibly as a step towards the development of a vision substitution device for the blind. The application of an invertible (1-to-1) image-to-sound mapping ensures the preservation of visual information. The system implementation involves a pipelined special purpose computer connected to a standard television camera. The time-multiplexed sound representations, resulting from a real-time image-to-sound conversion, represent images up to a resolution of 64 x 64 pixels with 16 gray-tones per pixel. A novel design and the use of standard components have made for a low-cost portable prototype conversion system having a power dissipation suitable for battery operation. Computerized sampling of the system output and subsequent calculation of the approximate inverse (sound-to-image) mapping provided the first convincing experimental evidence for the preservation of visual information in the sound representations of complicated images. However, the actual resolution obtainable with human perception of these sound representations remains to be evaluated.

The lateral-occipital tactile-visual area (LOtv) is activated when objects are recognized by vision or touch. We report here that the LOtv is also activated in sighted and blind humans who recognize objects by extracting shape information from visual-to-auditory sensory substitution soundscapes. Recognizing objects by their typical sounds or learning to associate specific soundscapes with specific objects do not activate this region. This suggests that LOtv is driven by the presence of shape information.

No abstract

No abstract