Visual-to-auditory sensory substitution alters language asymmetry in both sighted novices and experienced visually impaired users

Abstract

Visual-to-auditory sensory substitution devices (SSDs) provides improved access to the visual environment for the visually impaired by converting images into auditory information. Research is lacking on the mechanisms involved in processing data that is perceived through one sensory modality, but directly associated with a source in a different sensory modality. SSDs may be similar to reading, as an image (printed word) is converted into sound (when read aloud). Reading, and language more generally, are typically lateralised to the left cerebral hemisphere. Yet, unlike symbolic written language, SSDs convert images to sound based on visuospatial properties, with the right cerebral hemisphere potentially having a role in processing such visuospatial data. Here we investigated whether there is a hemispheric bias in the processing of visual-to-auditory sensory substitution information and whether that varies as a function of expertise and visual ability. We assessed the lateralisation of auditory processing with two tests: a standard dichotic listening test and a novel dichotic listening test created using the auditory information produced by an SSD, The vOICe. Although standard dichotic listening is lateralised to the left hemisphere, the auditory processing of images in SSDs is bilateral, possibly due to the increased influence of right hemisphere processing.

A Review of EEG and fMRI Measuring Aesthetic Processing in Visual User Experience Research

“…is summary of brain impairment conditions can inspire UX designers to invent more interactive installations or products to help visually impaired people understand this world. For example, current studies utilized visual-to-auditory sensory substitution devices to convert visual information to auditory information [113], and Mymemory mobile memory application helps people with traumatic brain injury [114]. Table 4 summarizes brain impairment conditions that influence aesthetic appreciation and UX and UI designs.…”