Electrophysiological Indices of Audiovisual Speech Perception in the Broader Autism Phenotype

Abstract

When a speaker talks, the consequences of this can both be heard (audio) and seen (visual). A novel visual phonemic restoration task was used to assess behavioral discrimination and neural signatures (event-related potentials, or ERP) of audiovisual processing in typically developing children with a range of social and communicative skills assessed using the social responsiveness scale, a measure of traits associated with autism. An auditory oddball design presented two types of stimuli to the listener, a clear exemplar of an auditory consonant–vowel syllable /ba/ (the more frequently occurring standard stimulus), and a syllable in which the auditory cues for the consonant were substantially weakened, creating a stimulus which is more like /a/ (the infrequently presented deviant stimulus). All speech tokens were paired with a face producing /ba/ or a face with a pixelated mouth containing motion but no visual speech. In this paradigm, the visual /ba/ should cause the auditory /a/ to be perceived as /ba/, creating an attenuated oddball response; in contrast, a pixelated video (without articulatory information) should not have this effect. Behaviorally, participants showed visual phonemic restoration (reduced accuracy in detecting deviant /a/) in the presence of a speaking face. In addition, ERPs were observed in both an early time window (N100) and a later time window (P300) that were sensitive to speech context (/ba/ or /a/) and modulated by face context (speaking face with visible articulation or with pixelated mouth). Specifically, the oddball responses for the N100 and P300 were attenuated in the presence of a face producing /ba/ relative to a pixelated face, representing a possible neural correlate of the phonemic restoration effect. Notably, those individuals with more traits associated with autism (yet still in the non-clinical range) had smaller P300 responses overall, regardless of face context, suggesting generally reduced phonemic discrimination.

Central Auditory Processing Disorders in Individuals with Autism Spectrum Disorders

“…Several publications have reported deterioration in speech perception in patients with ASDs. It has been advocated to be a component of the global sensory deficit ( 32 , 33 , 34 ). One interesting hypothesis regarding this issue is the integration deficit of the auditory and visual speech information.…”

Where on the face do we look during phonemic restoration: An eye-tracking study

“…For many years, there has been evidence that visual information about speech influences what listeners hear, including increasing identification of the speech signal in the context of background noise ( Sumby and Pollack, 1954 ; Grant et al, 1998 ). This influence of visual speech has been reported for a wide range of ages ( Irwin et al, 2017a , b ) for persons’ with typical and reduced hearing ( Sommers et al, 2005 ), for clinical populations such as for persons with autism ( Kuhl et al, 2005 ; Stevenson et al, 2014 ; Irwin et al, 2022 ), and for nonnative speakers of English ( Reisberg et al, 1987 ). The presence of visual articulatory information can also facilitate the perception of heard speech, speeding up cortical processing of the auditory signal ( van Wassenhove et al, 2005 ) and facilitating language processing ( MacDonald et al, 2000 ; Lachs and Pisoni, 2004 ).…”

“…Research on audiovisual communication has extended into child populations with similar results. Irwin et al (2017a , b) demonstrated that typically-developing children between the ages of 5–10 years show increased gaze to the mouth of a speaker in a range of audiovisual environments, including audiovisual speech, audiovisual speech in noise, and in an audiovisual mismatch condition.…”

Sketching the Landscape of Speech Perception Research (2000–2020): A Bibliometric Study

“…Second, research in the blue cohort concentrated on studies of cognitive processes in speech perception, such as audio-visual speech perception with application of the state-of-art brain science technologies ( Sato et al, 2010 ; Irwin et al, 2017 ). For example, Irwin et al (2017) tested typical developing children’s ability to detect the missing segment in spoken syllables with and without visual information and measured their behaviour and neural activities. They found that visual information attenuated the brain responses, suggesting that children were less sensitive to the missing segment when visual information of a face producing that segment was presented.…”