Congenitally blind adults can learn to identify face-shapes via auditory sensory substitution and successfully generalize some of the learned features

Abstract

Unlike sighted individuals, congenitally blind individuals have little to no experience with face shapes. Instead, they rely on non-shape cues, such as voices, to perform character identification. The extent to which face-shape perception can be learned in adulthood via a different sensory modality (i.e., not vision) remains poorly explored. We used a visual-to-auditory Sensory Substitution Device (SSD) that enables conversion of visual images to the auditory modality while preserving their visual characteristics. Expert SSD users were systematically taught to identify cartoon faces via audition. Following a tailored training program lasting ~ 12 h, congenitally blind participants successfully identified six trained faces with high accuracy. Furthermore, they effectively generalized their identification to the untrained, inverted orientation of the learned faces. Finally, after completing the extensive 12-h training program, participants learned six new faces within 2 additional hours of training, suggesting internalization of face-identification processes. Our results document for the first time that facial features can be processed through audition, even in the absence of visual experience across the lifespan. Overall, these findings have important implications for both non-visual object recognition and visual rehabilitation practices and prompt the study of the neural processes underlying auditory face perception in the absence of vision.

Face shape processing via visual-to-auditory sensory substitution activates regions within the face processing networks in the absence of visual experience

“…This study explored the properties of face-shape preference in the ventral visual stream of the congenitally blind, following tailored face training via the atypical auditory modality. Specifically, a group of SSD-expert congenitally blind adults learned to perceive cartooned faces in a ∼12 h unique training program aimed at conveying faces auditorily via soundscapes, in a shape, spatial layout, and color-preserving manner, using a visual-to-auditory SSD ( Arbel et al, 2022 ). Following training, the ventral visual stream in the blind brain showed preference for faces versus scrambled sounds in the fusiform gyrus as well as a parametrically modulated preference for trained SSD-faces over inverted, novel, and scrambled faces, also peaking in the fusiform gyrus.…”

“…We constructed a training program that included five sessions of 2 h each (participants received additional “refresher” sessions up to 12 h in total). For full details of the training program see Arbel et al, 2022 . In brief, participants were introduced to 6 cartoon faces which were adapted from the children’s game “guess who” and translated into soundscapes using EyeMusic (see Supplementary Figure 2A ).…”

“…Prior to the fMRI experiment, to ensure training effectiveness and the feasibility of the tasks inside the scanner, each participant completed three behavioral tasks [results are detailed in Arbel et al (2022) ]. Briefly, the first was a naming task in which participants named each of the six characters learned during training.…”

“…We constructed a training program that included five sessions of 2 h each (participants received additional "refresher" sessions up to 12 h in total). For full details of the training program see Arbel et al, 2022. In brief, participants were introduced to 6 cartoon faces which were adapted from the children's game "guess who" and translated into soundscapes using EyeMusic (see Supplementary Figure 2A). In the first stages of the training, participants learned to interpret only horizontal strips of the images (bottom, top, and middle), to gradually advance their skill so they could focus on perceiving small details embedded within the complex sounds (Supplementary Figure 1).…”

Faces in the Eye of the Mirrors. Faces without Mirror

“…There are very few reports about facial plastic surgery in the blind and none in born blind. In a 2007 editorial, Andrew Burd 35 found only one paper with direct reference to performing cosmetic surgery on a blind person (augmentation mammoplasty). In 2022, Salvet Ors published the case of a rhinoplasty in a blind man 36 .…”