The Topo-Speech sensory substitution system as a method of conveying spatial information to the blind and vision impaired

Abstract

Humans, like most animals, integrate sensory input in the brain from different sensory modalities. Yet humans are distinct in their ability to grasp symbolic input, which is interpreted into a cognitive mental representation of the world. This representation merges with external sensory input, providing modality integration of a different sort. This study evaluates the Topo-Speech algorithm in the blind and visually impaired. The system provides spatial information about the external world by applying sensory substitution alongside symbolic representations in a manner that corresponds with the unique way our brains acquire and process information. This is done by conveying spatial information, customarily acquired through vision, through the auditory channel, in a combination of sensory (auditory) features and symbolic language (named/spoken) features. The Topo-Speech sweeps the visual scene or image and represents objects’ identity by employing naming in a spoken word and simultaneously conveying the objects’ location by mapping the x-axis of the visual scene or image to the time it is announced and the y-axis by mapping the location to the pitch of the voice. This proof of concept study primarily explores the practical applicability of this approach in 22 visually impaired and blind individuals. The findings showed that individuals from both populations could effectively interpret and use the algorithm after a single training session. The blind showed an accuracy of 74.45%, while the visually impaired had an average accuracy of 72.74%. These results are comparable to those of the sighted, as shown in previous research, with all participants above chance level. As such, we demonstrate practically how aspects of spatial information can be transmitted through non-visual channels. To complement the findings, we weigh in on debates concerning models of spatial knowledge (the persistent, cumulative, or convergent models) and the capacity for spatial representation in the blind. We suggest the present study’s findings support the convergence model and the scenario that posits the blind are capable of some aspects of spatial representation as depicted by the algorithm comparable to those of the sighted. Finally, we present possible future developments, implementations, and use cases for the system as an aid for the blind and visually impaired.

Longitudinal stability of individual brain plasticity patterns in blindness

“…This may assist in determining the likelihood a blind individual would benefit from any of a large number of approaches available to them. Given the dizzying selection of sensory substitution techniques [93][94][95][96][97][98] and sensory aids (e.g. for reading: refreshable Braille displays, screen readers, and optical and electronic aids using touch, audition, and vision, respectively; in navigation even more options exist 99,100 ), narrowing these down to fit the individual would be valuable.…”

Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device

“…Most of these are based on the substitution of visual information through the auditory or tactile systems (Bach-y-Rita et al, 1969;Bach-y-Rita, 1983, 2004Ptito et al, 2005;Chebat et al, 2007) and the substitution of auditory information through the tactile system (Cieśla et al, 2019(Cieśla et al, , 2022, among others. Recent studies in our lab, both in the sighted (Netzer et al, 2021) and in the blind (Maimon et al, 2023), have also tested the ability to extend/augment visual-spatial perception (in both the front and the back) using auditory cues.…”

Improved Spatial Knowledge Acquisition through Sensory Augmentation

“…Similar techniques could be applied to sensory substitution, for instance, to compensate for sensory disability, disorder, or deficits [40,[118][119][120][121][122][123][124], or to other sensory augmentation devices to further enhance human perception by allowing humans to perceive information that they cannot naturally incorporate through their senses [27,[30][31][32][33][34][35]. Specifically, similar sensory augmentation techniques have been used to aid deaf and blind people during spatial navigation, showing that training with sensory augmentation devices can lead to improved navigational performance and reduced discomfort [46,121,125,126].…”