Visual motion processing recruits regions selective for auditory motion in early deaf individuals

Abstract

In early deaf individuals, the auditory deprived temporal brain regions become engaged in visual processing. In our study we tested further the hypothesis that intrinsic functional specialization guides the expression of cross-modal responses in the deprived auditory cortex. We used functional MRI to characterize the brain response to horizontal, radial and stochastic visual motion in early deaf and hearing individuals matched for the use of oral or sign language. Visual motion showed enhanced response in the ‘deaf’ mid-lateral planum temporale, a region selective to auditory motion as demonstrated by a separate auditory motion localizer in hearing people. Moreover, multivariate pattern analysis revealed that this reorganized temporal region showed enhanced decoding of motion categories in the deaf group, while visual motion-selective region hMT+/V5 showed reduced decoding when compared to hearing people. Dynamic Causal Modelling revealed that the ‘deaf’ motion-selective temporal region shows a specific increase of its functional interactions with hMT+/V5 and is now part of a large-scale visual motion selective network. In addition, we observed preferential responses to radial, compared to horizontal, visual motion in the ‘deaf’ right superior temporal cortex region that also show preferential response to approaching/receding sounds in the hearing brain. Overall, our results suggest that the early experience of auditory deprivation interacts with intrinsic constraints and triggers a large-scale reallocation of computational load between auditory and visual brain regions that typically support the multisensory processing of motion information.HighlightsAuditory motion-sensitive regions respond to visual motion in the deafReorganized auditory cortex can discriminate between visual motion trajectoriesPart of the deaf auditory cortex shows preference for in-depth visual motionDeafness might lead to computational reallocation between auditory/visual regions.

Sensory experience modulates the reorganisation of temporal auditory regions for executive processing

“…Several studies of crossmodal plasticity propose a preservation of function in auditory areas, where these regions maintain their original computation but adapt to respond to a different sensory input (Lomber et al, 2010; Cardin et al, 2013; Bola et al, 2017; Benetti et al, 2017, 2021). Other studies have suggested that sensory-deprived auditory regions are involved in higher-order cognitive functions, suggesting a functional change (see for a review Cardin et al, 2020b).…”

Metamodal Coupling of Vibrotactile and Auditory Speech Processing Systems Through Matched Stimulus Representations

“…Our study extends these findings to show that metamodal engagement is possible in auditory cortex as well. To our knowledge, metamodal engagement of auditory cortex has been limited to posterior auditory association cortex (pSTS) and has only been found in congenitally deaf but not hearing individuals (Benetti et al, 2017, 2020; Bola et al, 2017; Twomey et al, 2017). Furthermore, these studies did not find evidence of metamodal engagement in neurotypical individuals.…”

“…A prime example of this process is reading, which is initially thought to recruit auditory speech processing pathways through grapheme-to-phoneme conversion (Pugh et al, 2001), and the same idea has given rise to promising therapeutic applications such as sensory substitution devices (SSDs, which, for instance enable processing of visual information in blind individuals by translating camera input to acoustic stimuli (Bach-y-Rita and Kercel, 2003; Meijer, 1992). Yet, other studies (Benetti et al, 2017, 2020; Bola et al, 2017; Fairhall et al, 2017; Mattioni et al, 2020; Pietrini et al, 2004; Twomey et al, 2017; Vetter et al, 2020) have failed to find or have found far less robust evidence of cross-modal engagement in neurotypical subjects, raising the critical question of the conditions under which a particular sensory area can be successfully recruited for “metamodal” processing.…”

“…For example, it has been argued (Reich et al, 2011; Striem-Amit et al, 2012) that metamodal engagement of the posterior fusiform cortex through Braille and written word stimuli occurs because stimuli in both modalities convey “shape” information. However, many studies (Benetti et al, 2017, 2020; Bola et al, 2017; Fairhall et al, 2017; Kanjlia et al, 2018; Kupers et al, 2006; Mattioni et al, 2020; Pietrini et al, 2004; Ptito et al, 2005; Twomey et al, 2017; Vetter et al, 2020) that ensure a cognitive congruence and shared task demands do not find evidence of cross-modal stimuli engaging the same task-relevant neural representations in neurotypical individuals. One possible explanation may be that neurotypical individuals, but not congenitally sensory-deprived individuals, have already learned representations in the targeted brain area that are optimized for the conventional sensory input to that region.…”

Somatosensory processing in deaf and deafblind individuals: How does the brain adapt as a function of sensory and linguistic experience? A critical review

“…This review will focus on studies on crossmodal plasticity, which concerns changes in areas that are typically associated with a modality that is not received and processed. Examples are higher activity in auditory areas in deaf compared to hearing individuals during visual stimulation (Finney et al, 2001;Fine et al, 2005;Benetti et al, 2021) and tactile stimulation (Karns et al, 2012;Zimmermann et al, 2021), or activity in visual areas of blind individuals who are listening to speech (Bedny et al, 2011). Benetti et al (2021) presented patterns of moving dots and observed significantly stronger activation in typical auditory areas in early deaf participants compared to hearing signers and non-signers.…”