Neural Correlates of Interindividual Differences in Children's Audiovisual Speech Perception

Abstract

Children use information from both the auditory and visual modalities to aid in understanding speech. A dramatic illustration of this multisensory integration is the McGurk effect, an illusion in which an auditory syllable is perceived differently when it is paired with an incongruent mouth movement. However, there are significant interindividual differences in McGurk perception: some children never perceive the illusion, while others always do. Because converging evidence suggests that the posterior superior temporal sulcus (STS) is a critical site for multisensory integration, we hypothesized that activity within the STS would predict susceptibility to the McGurk effect. To test this idea, we used blood-oxygen level dependent functional magnetic resonance imaging (BOLD fMRI) in seventeen children aged 6 to 12 years to measure brain responses to three audiovisual stimulus categories: McGurk incongruent, non-McGurk incongruent and congruent syllables. Two separate analysis approaches, one using independent functional localizers and another using whole-brain voxel-based regression, showed differences in the left STS between perceivers and non-perceivers. The STS of McGurk perceivers responded significantly more than non-perceivers to McGurk syllables, but not to other stimuli, and perceivers’ hemodynamic responses in the STS were significantly prolonged. In addition to the STS, weaker differences between perceivers and non-perceivers were observed in the FFA and extrastriate visual cortex. These results suggest that the STS is an important source of interindividual variability in children’s audiovisual speech perception.

Audio–visual integration during speech perception in prelingually deafened Japanese children revealed by the McGurk effect

“…ba 83 13 ba 92 8 pa 83 0 pa 75 0 Visual da 8 29 ga 0 0 ka 0 4 ta 25 96 Fusion and others ta 0 46 da 8 50 ta 17 75 ha 0 4 pa 8 8 ta 0 42 a 0 17 ga 0 4 The McGurk effect in older children with CI may stem from a more mature audiovisual integration. Nath et al also observed audiovisual integration beginning around 6 years of age [15]. Although audiovisual integration involved in the McGurk effect did not appear in normal-hearing Japanese children probably due to language simplicity [20], older children with CI who hear artificial and insufficient speech sounds coded by CI might use more visual information to compensate for hearing difficulties.…”

“…The age of 6 years was chosen as the cutoff point because at this age the McGurk effect appears in native speakers of European languages [7,20] and audiovisual integration in STS also begins to function [15]. Children's perception of facial expressions also improves between 5 and 10 years of age [25,26].…”

“…Previous functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies suggest that left STS plays a critical role in audiovisual integration and the McGurk effect [13][14][15][16].…”

Fiber Pathways for Language in the Developing Brain: A Diffusion Tensor Imaging (DTI) Study

“…The first is the well-known SLF/AF pathway that anchors the dorsal stream and is thought to be involved in phonological processing, receptive and expressive language processing, and literacy. The second pathway is the newly described (Bahrick, Lickliter, & Flom, 2004;Blomert & Froyen, 2010;Erdener & Burnham, 2013;Gogate & Bahrick, 1998;Gogate, Bahrick, & Watson, 2000;Lachs, Pisoni, & Iler Kirk, 2001;Magnan & Ecalle, 2006;Magnan, Ecalle, Veuillet, & Collet, 2004;Nath, Fava, & Beauchamp, 2011;Ross et al, 2011;Sekiyama & Burnham, 2004). For example, amodal information, which refers to information shared across multiple sense modalities (i.e., temporal synchrony, rhythm, tempo), seems to have the effect of directing attention to what is important in the environment, and enhancing salient features of actions and objects Calvert, Campbell, & Brammer, 2000).…”

“…The first, the sensory-sensory (auditory-visual) pathway is anchored by the posterior superior temporal sulcus (STSp), and connects visual domains (especially MT/V5), primary auditory cortex, STSp, and inferior frontal regions (Arnal, Morillon, Kell, & Giraud, 2009;Calvert, 1997). Evidence in older children (Nath & Beauchamp, 2012;Nath et al, 2011) and adults (Arnal et al, 2009;Barrós-Loscertales et al, 2013;Calvert, 1997) and auditory stimuli (e.g., visual /da/, auditory /ba/), and as a consequence report hearing a syllable that is different from the original stimuli (e.g., ga).…”