Multisensory speech perception in autism spectrum disorder: From phoneme to whole‐word perception

Abstract

Lay Abstract In a noisy environment it can often be difficult to understand what a speaker is saying, but being able to see the speaker’s mouth makes it easier. In fact, the noisier the environment, the more it helps to both see and hear a speaker. This is due to a process known as multisensory integration, where information from two senses, in this case audition and vision, are combined. Autistic children often have difficulties with multisensory integration, which may add to difficulties with social communication. We tested how well autistic and typically developing children integrated auditory speech and visual speech, both at the level of whole words and also at the level of individual speech sounds, or phonemes. At both the whole-word and the phoneme level, autistic children were less able to correctly identify speech sounds, regardless of how they were presented (audio-only, visual-only, audiovisual). Autistic children showed equal benefit in phoneme perception from seeing the mouth movements accompanying the auditory speech. In noisy environments however, these equal gains in phoneme perception did not extend to whole-word recognition, where autistic individuals showed reduced benefit from seeing the mouth movements while hearing auditory speech. Such reductions in speech perception abilities, including a reduced benefit from the presence of visual speech cues in noisy environments, may contribute to social communication difficulties in autistic children. Scientific Abstract Speech perception in noisy environments is boosted when a listener can see the speaker’s mouth and integrate the auditory and visual speech information. Autistic children have a diminished capacity to integrate sensory information across modalities, which contributes to core symptoms of autism, such as impairments in social communication. We investigated the abilities of autistic and typically-developing (TD) children to integrate auditory and visual speech stimuli in various levels of noise. Measurements of both whole-word and phoneme recognition were recorded. At the level of whole-word recognition, autistic children exhibited reduced performance in both the auditory and audiovisual modalities. Importantly, autistic children showed reduced behavioral benefit from multisensory integration with whole-word recognition, specifically at low signal to noise ratios (SNRs). At the level of phoneme recognition, autistic children exhibited reduced performance relative to their TD peers in auditory, visual, and audiovisual modalities. However, and in contrast to their performance at the level of whole-word recognition, both autistic and TD children showed benefits from multisensory integration for phoneme recognition. In accordance with the principle of inverse effectiveness, both groups exhibited greater benefit at low SNRs relative to high SNRs. Thus, while autistic children showed typical multisensory benefits during phoneme recognition, these benefits did not translate to typical multisensory benefit of whole-word recognition in noisy e...

Sensitivity to audio‐visual synchrony and its relation to language abilities in children with and without ASD

“…Using naturalistic speech stimuli, this result extends prior work demonstrated a positive relationship between audiovisual speech processing and receptive language abilities [Patten et al, ]. Even though causality cannot be inferred from the present results, our data support the hypothesized relationship between language deficits observed in ASD and atypical multisensory processing [Stevenson et al, ]. Interpreted in this way, these results together with prior research may help to generate hypotheses about low level deficits that may precede or influence the development of deficits in higher level language abilities in ASD.…”

“…These differences have been reported from as early as 9 months of age and extend to children and adolescents, with atypical responses to mismatched auditory/visual signals in a McGurk paradigm that indicate reduced influence of visual information [Guiraud, Tomalski, Kushnerenko, Ribeiro, Davies, & Charman, ; Mongillo et al, ]. Children and adolescents with ASD also display atypical multisensory processing with younger participants failing to show preferences for synchronized audio‐visual presentations of socially salient stimuli [Bebko et al, ], and older participants failing to show expected behavioral facilitatory effects of a synchronous multimodal signal in the context of a noisy environment [Irwin et al, ; Foxe et al, ; Smith & Bennetto, ; Stevenson et al, a].…”

“…Impairments in detecting the temporal characteristics of multimodal signals have been documented extensively in ASD [see Stevenson et al, for a review]. A recent study has demonstrated that temporal processing is correlated with sensory integration, and the ability to accurately detect speech in noise in ASD only [Stevenson et al, b].…”

Shifts in Audiovisual Processing in Healthy Aging

“…Beyond this level the amount of behavioural benefit decreases, resulting in a pattern of behavioural gain where there is a “sweet spot” at which maximum gain occurs [49-51]. Indeed, there has been a good deal of evidence to suggest that the SNR at which maximal gain occurs during word recognition is relatively stable at adolescence [50] into adulthood [51], and across clinical populations [83-85]. …”

Multimodal Language Learning: How to Crack the Speech Code by Ear and by Eye

“…Nonetheless, more recent evidence converges toward the idea that typically developing humans are able to take advantage of the multimodality of the speech input to crack the speech signal (Crosse, Di Liberto, & Lalor, ; Peelle & Sommers, ; Venezia, Thurman, Matchin, George, & Hickok, ). For instance, different findings suggest that language learning impairments that are observed in children with developmental dyslexia, specific language impairment, or autistic spectrum disorders are correlated with children's difficulty in binding the visual speech stream with its temporally contingent auditory counterpart (Harrar et al., ; Pons, Andreu, Sanz‐Torrent, Buil‐Legaz, & Lewkowicz, ; Stevenson et al., ).…”