Infants and Adults Use Visual Cues to Improve Detection and Discrimination of Speech in Noise

Abstract

Purpose This study assessed the extent to which 6- to 8.5-month-old infants and 18- to 30-year-old adults detect and discriminate auditory syllables in noise better in the presence of visual speech than in auditory-only conditions. In addition, we examined whether visual cues to the onset and offset of the auditory signal account for this benefit. Method Sixty infants and 24 adults were randomly assigned to speech detection or discrimination tasks and were tested using a modified observer-based psychoacoustic procedure. Each participant completed 1–3 conditions: auditory-only, with visual speech, and with a visual signal that only cued the onset and offset of the auditory syllable. Results Mixed linear modeling indicated that infants and adults benefited from visual speech on both tasks. Adults relied on the onset–offset cue for detection, but the same cue did not improve their discrimination. The onset–offset cue benefited infants for both detection and discrimination. Whereas the onset–offset cue improved detection similarly for infants and adults, the full visual speech signal benefited infants to a lesser extent than adults on the discrimination task. Conclusions These results suggest that infants' use of visual onset–offset cues is mature, but their ability to use more complex visual speech cues is still developing. Additional research is needed to explore differences in audiovisual enhancement (a) of speech discrimination across speech targets and (b) with increasingly complex tasks and stimuli.

Development of the Mechanisms Underlying Audiovisual Speech Perception Benefit

“…Recently, Lalonde and colleagues [ 51 , 52 ] adapted an audiovisual benefit measure from the adult literature for use with infants and children. Specifically, these studies measured audiovisual speech detection benefit by comparing detection performance in auditory-only and audiovisual conditions.…”

“…Lalonde and Werner [ 51 ] also adapted the audiovisual speech detection task for use with 6- to 8-month-old infants. Using an observer-based psychophysical procedure [ 53 ], infants and adults were trained to respond when they heard an auditory syllable /mu/ presented at random intervals in a continuous noise.…”

“…Overall, these results suggest that there is little change over development (from 6 months to about 30 years) in audiovisual benefit to speech detection, at least for simple speech signals such as single syllables [ 51 ]. This early ability contrasts with the protracted development of audiovisual benefit to speech recognition in children and adolescents, e.g., [ 34 , 35 ].…”

“…Neurophysiological and behavioral evidence indicates that this large neural network subserves multiple mechanisms of audiovisual speech perception benefit in adults [ 62 , 63 , 64 , 65 ]. More specifically, in both the adult and developmental literatures, an important distinction has been made between use of phonetic information and use of other salient auditory–visual correspondences in speech, namely temporal cues [ 27 , 43 , 45 , 51 , 52 , 62 , 63 , 64 , 65 , 66 ]. Terminology varies across studies, but a similar distinction has been drawn between (1) general perceptual mechanisms related to auditory–visual temporal correspondences and (2) speech-specific mechanisms that constrain phonetic and lexical interpretation.…”

Visual speech cues speed processing and reduce effort for children listening in quiet and noise

“…However, primary school children may not benefit from AV presentation in the same way. It is known that, like adults, children do benefit from visual speech cues to some extent: beginning in infancy, children show an AV benefit for accuracy when listening in noise (Fort, Spinelli, Savariaux, & Kandel, 2012;Knowland, Evans, Snell, & Rosen, 2016;Lalonde & Holt, 2015Lalonde & Werner, 2019;Ross et al, 2011), when listening to vocoded speech (Maidment, Kang, Stewart, & Amitay, 2015), and when listening to words/nonwords where the initial consonant has been excised from the auditory track (Jerger, Damian, Tye-Murray, & Abdi, 2014). Furthermore, visual cues may enhance children's phonological processing, as evidenced by greater phonological priming effects in AV compared to AO modality (Jerger, Damian, Tye-Murray, & Abdi, 2017).…”