Audiovisual synchrony detection for fluent speech in early childhood: An eye‐tracking study

Abstract

During childhood, the ability to detect audiovisual synchrony gradually sharpens for simple stimuli such as flashbeeps and single syllables. However, little is known about how children perceive synchrony for natural and continuous speech. This study investigated young children's gaze patterns while they were watching movies of two identical speakers telling stories side by side. Only one speaker's lip movements matched the voices and the other one either led or lagged behind the soundtrack by 600 ms. Children aged 3–6 years (n = 94, 52.13% males) showed an overall preference for the synchronous speaker, with no age‐related changes in synchrony‐detection sensitivity as indicated by similar gaze patterns across ages. However, viewing time to the synchronous speech was significantly longer in the auditory‐leading (AL) condition compared with that in the visual‐leading (VL) condition, suggesting asymmetric sensitivities for AL versus VL asynchrony have already been established in early childhood. When further examining gaze patterns on dynamic faces, we found that more attention focused on the mouth region was an adaptive strategy to read visual speech signals and thus associated with increased viewing time of the synchronous videos. Attention to detail, one dimension of autistic traits featured by local processing, has been found to be correlated with worse performances in speech synchrony processing. These findings extended previous research by showing the development of speech synchrony perception in young children, and may have implications for clinical populations (e.g., autism) with impaired multisensory integration.

Multisensory Reading in Early Childhood: Systematic Review with Theoretical Guidance for Human Development Studies

“…Thirteen studies claimed that the intervention led to developmental gains but did not address/mention developmental theory. Three notable exceptions were noted in papers by Zhou et al (2022), Labat et al (2020), andSchröder et al (2020). Zhou et al (2022) commented on age-related sensitivity of audiovisual synchrony perception, but this was not supported by the authors' later findings.…”

“…Three notable exceptions were noted in papers by Zhou et al (2022), Labat et al (2020), andSchröder et al (2020). Zhou et al (2022) commented on age-related sensitivity of audiovisual synchrony perception, but this was not supported by the authors' later findings. Labat et al (2020), discussed, in relation to the start rotation principle that, sensorimotor learning is not age-related and did thus not further explore age differences in sensory per-ception.…”

“…The findings of our analysis are synthesized below. Zhou et al (2022) reported research that examined eye tracking in young children whilst watching an on-screen storyteller, and thus the term multisensory was used in the context of the study. Roskos, Burstein, & Byeong-Keun (2012) discussed "multisensory behaviours" (i.e., looking, touching, listening, moving, gesturing) for the purposes of creating a typology for a framework for observing young children's engagement with eBooks.…”

“…However, at its simplest, the term multisensory was defined as learning, or a teaching approach, that engages two or more senses. In thirteen of the eighteen papers using the term multisensory, we found that multisensory approaches to the teaching of reading were presented as involving the addition of a haptic or kinaesthetic element to the traditional phono-graphic approach to decoding (Oliva-Maza et al, 2021;Neumann et al, 2009;Bøga et al, 2021;Zettler-Greeley et al, 2018;DiLorenzo et al, 2011;Labat et al, 2015Labat et al, , 2020Ecalle et al, 2021;Schlesinger & Gray, 2017;Neumann, 2014;Bara et al, 2004;Zafrana et al, 2000;Zhou et al, 2022). This included activity such as tracing the letter shape in the air, associated actions or feeling a cut out letter shape made of sandpaper, for example.…”

“…The findings of our analysis are synthesised below. Zhou et al (2022) reported research that examined eye tracking in young children whilst watching an on-screen storyteller, and thus the term multisensory was used in the context of the study. Roskos, Burstein, & Byeong-Keun (2012) discussed 'multisensory behaviours' (i.e.…”