Developmental Change in Sensitivity to Audiovisual Speech Congruency and Its Relation to Language in Infants

Abstract

In this study, we used eye-tracking to investigate selective visual attention paid to congruent and incongruent audiovisual speech of Japanese infants (6 to 12 months old) and adults. Infants' receptive and expressive language abilities at 12 months were measured through a questionnaire completed by their caregivers. We found that 6-month-olds looked at the mouth longer in the audiovisual congruent condition than in the incongruent condition, whereas 12-month-olds did not show any significant differences in time spent in looking at the mouth. Furthermore, the time spent looking at the mouth among 6-month-olds, in both the audiovisual congruent and incongruent conditions, was positively correlated with receptive language abilities with 12 month olds. These findings suggest that sensitivity to congruent and incongruent audiovisual speech changes during the first year, and attention to the speaker's mouth at 6 months are good predictors of later receptive language abilities.

Demystifying infant vocal imitation: The roles of mouth looking and speaker’s gaze

“…That infants tend to look at mouths has been shown in several studies (Kubicek et al, 2014;Kushnerenko et al, 2013;Pons, Bosch, & Lewkowicz, 2015;Tenenbaum, Shah, Sobel, Malle, & Morgan, 2013;Tenenbaum, Sobel, & Sheinkope, 2015;Tsang, Atagi, & Johnson, 2018). In fact, 6-month-olds who focused on the speaker's mouth have showed better language acquisition at 12, 18 and 24 months (Imafuku & Myowa-Yamakoshi, 2016;Young, Merin, Rogers, & Ozonoff, 2009). Accordingly, it is possible that looking at a speaker's mouth can facilitate the processing of speech leading to effective vocal imitation, because it provides direct access to important redundant (e.g.…”

“…and 24 months of age (Imafuku & Myowa-Yamakoshi, 2016;Young et al, 2009). On the basis of these combined findings, we speculate that the relationship between mouth looking and later language development improvement is specifically due to vocal imitation: those who look at the mouth more tend to better imitate the sounds making up the phonemes required in words.…”

“…When infants receive redundant audiovisual information compared to audio information, their attention, encoding information and learning are facilitated (e.g. In fact, 6-month-olds who focused on the speaker's mouth have showed better language acquisition at 12, 18 and 24 months (Imafuku & Myowa-Yamakoshi, 2016;Young, Merin, Rogers, & Ozonoff, 2009). This explanation emphasizes the crucial role of not only hearing sound, but also lip reading for language acquisition.…”

Changes in selective attention to articulating mouth across infancy: Sex differences and associations with language outcomes

“…For example, visual information about speech articulation enhances phoneme discrimination at 6 months (Teinonen et al, 2008), which predicts later word-learning (Tsao et al, 2004). Secondly, this result points to a specificity in the association between mouth-looking and expressive language development only (Young et al, 2009;Tenenbaum et al, 2015; but see Imafuku & Myowa, 2016). Given the close and bidirectional relationship between infants' speech perception and production (DePaolis et al, 2011), this specificity might seem surprising.…”

Eyes can tell. Attention to the eyes and the mouth during audiovisual vowel processing in monolingual and bilingual infants

“…. Endnotes1 Similarly, 6-, but not 12-month-old Japanese-learning infants attend more to the mouth of a face that is mismatching the heard fluent speech(Imafuku & Myowa, 2016).2 Note that here we are referring to the vowel system in Castilian Spanish language and most Basque dialects because subjects are recruited from the region where those dialects are spoken.Other dialects may differ in their vowel systems. For instance, the Soule dialect of Basque has an additional vowel /ü/(Hualde, 1991).3 Despite that the two languages are rather distinct in terms of their syntactic properties (i.e., language Spanish follows a verb-object word order, while Basque relies on an object-verb order).…”