Supramodal processing optimizes visual perceptual learning and plasticity

Abstract

Multisensory interactions are ubiquitous in cortex and it has been suggested that sensory cortices may be supramodal i.e. capable of functional selectivity irrespective of the sensory modality of inputs (Pascual-Leone and Hamilton, 2001; Renier et al., 2013; Ricciardi and Pietrini, 2011; Voss and Zatorre, 2012). Here, we asked whether learning to discriminate visual coherence could benefit from supramodal processing. To this end, three groups of participants were briefly trained to discriminate which of a red or green intermixed population of random-dot-kinematograms (RDKs) was most coherent in a visual display while being recorded with magnetoencephalography (MEG). During training, participants heard no sound (V), congruent acoustic textures (AV) or auditory noise (AVn); importantly, congruent acoustic textures shared the temporal statistics - i.e. coherence - of visual RDKs. After training, the AV group significantly outperformed participants trained in V and AVn although they were not aware of their progress. In pre- and post-training blocks, all participants were tested without sound and with the same set of RDKs. When contrasting MEG data collected in these experimental blocks, selective differences were observed in the dynamic pattern and the cortical loci responsive to visual RDKs. First and common to all three groups, vlPFC showed selectivity to the learned coherence levels whereas selectivity in visual motion area hMT+ was only seen for the AV group. Second and solely for the AV group, activity in multisensory cortices (mSTS, pSTS) correlated with post-training performances; additionally, the latencies of these effects suggested feedback from vlPFC to hMT+ possibly mediated by temporal cortices in AV and AVn groups. Altogether, we interpret our results in the context of the Reverse Hierarchy Theory of learning (Ahissar and Hochstein, 2004) in which supramodal processing optimizes visual perceptual learning by capitalizing on sensory-invariant representations - here, global coherence levels across sensory modalities.

Multisensory training can promote or impede visual perceptual learning of speech stimuli: visual-tactile vs. visual-auditory training

“…For example, sensory-guided plasticity using auditory or vibrotactile perception has been suggested as a possible approach to enhancing perceptual learning with a visual prosthesis (Merabet et al, 2005; Proulx et al, 2014). This suggestion is consistent with findings from psychophysical training experiments that show better visual perception after training with concordant acoustic patterns (Shams et al, 2011; van Wassenhove, 2013; Zilber et al, 2014). Such results encourage the view that multisensory stimuli are consistently useful in promoting unisensory perceptual learning.…”

“…The literature reports examples of auditory stimuli promoting visual perceptual learning with non-speech stimuli (Shams and Seitz, 2008; Shams et al, 2011; Zilber et al, 2014). But our recent study of prelingually deaf adults with late-acquired cochlear implants showed that visual speech impeded auditory perceptual learning (Bernstein et al, 2014), while the same AV training did not impede and even promoted to some extent the auditory perceptual learning of adults with normal hearing.…”

Interplay between functional connectivity and scale-free dynamics in intrinsic fMRI networks

“…However, we also outlined that these findings may results from attention effects, as originally observed in macaques [80]. To further investigate such issues and disentangle attention from operative effects in the recourse to high frequencies, future work will be devoted to the analysis of another existing MEG dataset [84] for which complementary eye tracker recordings will permit to probe attention through measurements of ocular saccades in cunjonction with behavioral performance.…”

Decoding perceptual thresholds from MEG/EEG

“…The time window from 100 to 600 ms was used (126 time-points). The ROIs were delineated on each participant (for more detail [13]) for both right and left hemispheres (Fig.3), except the frontalpole region which is a label from Freesurfer parcellation [14]. The dimensions of the data became: n = 196 at most, and p = 126 * 9000 ∼ 10 6 at most depending on the size of the label.…”

“…Similarly, mSTS showed a better discriminative power below the perceptual threshold. The regions of interest defined on basis of prior analysis [13] nicely show specificity for decoding. On the contrary, a control label (here, the frontal pole, black curve) barely reflects the perceptual threshold.…”