Audiovisual speech perception development at varying levels of perceptual processing

Lalonde, Kaylah; Holt, Rachael Frush

doi:10.1121/1.4945590

Cited by 35 publications

(38 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Visual speech consistently made it 1) harder to discriminate the non-intact from intact speech and 2) more likely to perceive the non-intact onsets as intact in both CHL and CNH for the easy to speechread onsets (/B/). These results agree with the finding that visual speech benefits both word discrimination and recognition (obtained in AX paradigm) in CNH [30]. For the hard-to-speechread onsets /G/, visual speech minimally (≤ 10%) affected both discrimination and identification in the CHL but only discrimination in the CNH.…”

Section: Discussionsupporting

confidence: 90%

“…Current, but scant, evidence indicates that: 1 ) Visual speech (salient contrasts) improved both phoneme discrimination and identification [6], and 2) Visual speech 2a ) produced adult-like benefit at an earlier age for word discrimination than word recognition as measured in an AX paradigm, and 2b ) benefited word recognition as measured in an AX paradigm more than word discrimination in adults but not in young children [30]. If we supplement this limited information with the auditory only evidence, the findings are mixed [31 –35].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Visual speech alters the discrimination and identification of non-intact auditory speech in children with hearing loss

Jerger

Damian

McAlpine

et al. 2017

International Journal of Pediatric Otorhinolaryngology

View full text Add to dashboard Cite

Objectives Understanding spoken language is an audiovisual event that depends critically on the ability to discriminate and identify phonemes yet we have little evidence about the role of early auditory experience and visual speech on the development of these fundamental perceptual skills. Objectives of this research were to determine 1) how visual speech influences phoneme discrimination and identification; 2) whether visual speech influences these two processes in a like manner, such that discrimination predicts identification; and 3) how the degree of hearing loss affects this relationship. Such evidence is crucial for developing effective intervention strategies to mitigate the effects of hearing loss on language development. Methods Participants were 58 children with early-onset sensorineural hearing loss (CHL, 53% girls, M = 9;4 yrs) and 58 children with normal hearing (CNH, 53% girls, M = 9;4 yrs). Test items were consonant-vowel (CV) syllables and nonwords with intact visual speech coupled to non-intact auditory speech (excised onsets) as, for example, an intact consonant/rhyme in the visual track (Baa or Baz) coupled to non-intact onset/rhyme in the auditory track (/–B/aa or /–B/az). The items started with an easy-to-speechread /B/ or difficult-to-speechread /G/ onset and were presented in the auditory (static face) vs. audiovisual (dynamic face) modes. We assessed discrimination for intact vs. non-intact different pairs (e.g., Baa:/–B/aa). We predicted that visual speech would cause the non-intact onset to be perceived as intact and would therefore generate more same—as opposed to different—responses in the audiovisual than auditory mode. We assessed identification by repetition of nonwords with non-intact onsets (e.g., /–B/az). We predicted that visual speech would cause the non-intact onset to be perceived as intact and would therefore generate more Baz—as opposed to az— responses in the audiovisual than auditory mode. Results Performance in the audiovisual mode showed more same responses for the intact vs. non-intact different pairs (e.g., Baa:/–B/aa) and more intact onset responses for nonword repetition (Baz for/–B/az). Thus visual speech altered both discrimination and identification in the CHL—to a large extent for the /B/ onsets but only minimally for the /G/ onsets. The CHL identified the stimuli similarly to the CNH but did not discriminate the stimuli similarly. A bias-free measure of the children’s discrimination skills (i.e., d’ analysis) revealed that the CHL had greater difficulty discriminating intact from non-intact speech in both modes. As the degree of HL worsened, the ability to discriminate the intact vs. non-intact onsets in the auditory mode worsened. Discrimination ability in CHL significantly predicted their identification of the onsets—even after variation due to the other variables was controlled. Conclusions These results clearly established that visual speech can fill in non-intact auditory speech, and this effect, in turn, made the non-intact onsets more difficult t...

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Introductionmentioning

confidence: 99%

Visual speech alters the discrimination and identification of non-intact auditory speech in children with hearing loss

Jerger

Damian

McAlpine

et al. 2017

International Journal of Pediatric Otorhinolaryngology

View full text Add to dashboard Cite

show abstract

“…However, they do not yet show adult patterns of visual attention. This may be because the audiovisual speech processing system is still immature in 12-month-olds and even develops during childhood (Knowland, Mercure, Karmiloff-Smith, Dick, & Thomas, 2014) and from childhood to adulthood (Lalonde & Holt, 2016).…”

Section: Discussionmentioning

confidence: 99%

Developmental Change in Sensitivity to Audiovisual Speech Congruency and Its Relation to Language in Infants

Imafuku

Myowa

2016

An International Journal of Psychological Sciences

View full text Add to dashboard Cite

In this study, we used eye-tracking to investigate selective visual attention paid to congruent and incongruent audiovisual speech of Japanese infants (6 to 12 months old) and adults. Infants' receptive and expressive language abilities at 12 months were measured through a questionnaire completed by their caregivers. We found that 6-month-olds looked at the mouth longer in the audiovisual congruent condition than in the incongruent condition, whereas 12-month-olds did not show any significant differences in time spent in looking at the mouth. Furthermore, the time spent looking at the mouth among 6-month-olds, in both the audiovisual congruent and incongruent conditions, was positively correlated with receptive language abilities with 12 month olds. These findings suggest that sensitivity to congruent and incongruent audiovisual speech changes during the first year, and attention to the speaker's mouth at 6 months are good predictors of later receptive language abilities.

show abstract

“…We paired three tasks involving auditory or visual discrimination with corresponding steady-state responses. All behavioral tasks required discrimination as opposed to detection, in order to target high-level perceptual judgments [32]. …”

Section: Introductionmentioning

confidence: 99%

Sustained Cortical and Subcortical Measures of Auditory and Visual Plasticity following Short-Term Perceptual Learning

et al. 2017

View full text Add to dashboard Cite

Short-term training can lead to improvements in behavioral discrimination of auditory and visual stimuli, as well as enhanced EEG responses to those stimuli. In the auditory domain, fluency with tonal languages and musical training has been associated with long-term cortical and subcortical plasticity, but less is known about the effects of shorter-term training. This study combined electroencephalography (EEG) and behavioral measures to investigate short-term learning and neural plasticity in both auditory and visual domains. Forty adult participants were divided into four groups. Three groups trained on one of three tasks, involving discrimination of auditory fundamental frequency (F0), auditory amplitude modulation rate (AM), or visual orientation (VIS). The fourth (control) group received no training. Pre- and post-training tests, as well as retention tests 30 days after training, involved behavioral discrimination thresholds, steady-state visually evoked potentials (SSVEP) to the flicker frequencies of visual stimuli, and auditory envelope-following responses simultaneously evoked and measured in response to rapid stimulus F0 (EFR), thought to reflect subcortical generators, and slow amplitude modulation (ASSR), thought to reflect cortical generators. Enhancement of the ASSR was observed in both auditory-trained groups, not specific to the AM-trained group, whereas enhancement of the SSVEP was found only in the visually-trained group. No evidence was found for changes in the EFR. The results suggest that some aspects of neural plasticity can develop rapidly and may generalize across tasks but not across modalities. Behaviorally, the pattern of learning was complex, with significant cross-task and cross-modal learning effects.

show abstract

Audiovisual speech perception development at varying levels of perceptual processing

Cited by 35 publications

References 69 publications

Visual speech alters the discrimination and identification of non-intact auditory speech in children with hearing loss

Visual speech alters the discrimination and identification of non-intact auditory speech in children with hearing loss

Developmental Change in Sensitivity to Audiovisual Speech Congruency and Its Relation to Language in Infants

Sustained Cortical and Subcortical Measures of Auditory and Visual Plasticity following Short-Term Perceptual Learning

Contact Info

Product

Resources

About