Newborns’ sensitivity to the visual aspects of infant-directed speech: Evidence from point-line displays of talking faces.

Abstract

The first time a newborn is held, he is attracted by the human's face. A talking face is even more captivating, as it is the first time he or she hears and sees another human talking. Older infants are relatively good at detecting the relationship between images and sounds when someone is addressing to them, but it is unclear whether this ability is dependent on experience or not. Using an intermodal matching procedure, we presented newborns with 2 silent point-line displays representing the same face uttering different sentences while they were hearing a vocal-only utterance that matched 1 of the 2 stimuli. Nearly all of the newborns looked longer at the matching point-line face than at the mismatching 1, with prior exposure to the stimuli (Experiment 1) or without (Experiment 2). These results are interpreted in terms of newborns' ability to extract common visual and auditory information of continuous speech events despite a short experience with talking faces. The implications are discussed in the light of the language processing and acquisition literature. (PsycINFO Database Record

Prosody in the Auditory and Visual Domains: A Developmental Perspective

“…Prosody and gestures also overlap in terms of which linguistic functions they are used for. Infants use visual correlates of prosody to segment the speech stream (e.g., Kitamura et al, 2014 ; Guellaï et al, 2016 ), to organize information at the discourse level (e.g., Nicoladis et al, 1999 ; Capone and McGregor, 2004 ; Mathew et al, 2017 ), and to express emotions, intentions, and beliefs ( Sullivan and Lewis, 2003 ; Esteve-Gibert and Prieto, 2014 ; Berman et al, 2016 ; Aureli et al, 2017 ; González-Fuente, 2017 ). Children are sensitive to the fact that visual cues convey relevant linguistic meaning, and experimental evidence shows that gestures are processed earlier and more accurately than prosodic or lexical cues ( Armstrong et al, 2014 ; Esteve-Gibert et al, 2017c ; Hübscher et al, 2017 ).…”

“…Results showed that infants reliably detect auditory and visual congruencies in the displays. It seems that this ability emerges early in development as newborns are already able to match a facial display to the corresponding speech stream ( Guellaï et al, 2016 ).…”

Categorical ERP repetition effects for human and furniture items in 7‐month‐old infants

“…By 3 months, infants are capable of categorizing animates and inanimates based on dynamic visual information (Arterberry & Bornstein, ). From birth on, they start associating lip movements with speech information, and facial expressions with emotional tone in voice (Guellai, Streri, Chopin, Rider, & Kitamura, ; Soken & Pick, ), and at 6.5 months, they match affective body movements with voice (Zieber, Kangas, Hock, & Bhatt, ). Hence, much of the existing infant literature suggests that infants show a special interest for information about humans in multiple sensory domains, and that they seem capable of combining such information from early on.…”

Perception of audiovisual infant directed speech

“…An early language environment presents considerable complexity to an infant sorting out phonetic categories, not only by its variability in speakers and speech registers, but also by the availability of information from multiple sensory modalities. Despite this complexity, new‐borns are able to perceive commonalities in visual and auditory information in continuous speech (Guellaï, Streri, Chopin, Rider & Kitamura, ). While still not a robust effect (Desjardins & Werker, ), classical studies show that infants will look longer at the face articulating a heard vowel compared to a non‐matching vowel (Aldridge, Braga, Walton & Bower, ; Kuhl & Meltzoff, ; Patterson & Werker, ).…”