Can a Signing Virtual Human Engage a Baby's Attention?

Abstract

The child developmental period of ages 6-12 months marks a widely understood "critical period" for healthy language learning, during which, failure to receive exposure to language can place babies at risk for language and reading problems spanning life. Deaf babies constitute one vulnerable population as they can experience dramatically reduced or no access to usable linguistic input during this period. Technology has been used to augment linguistic input (e.g., auditory devices; language videotapes) but research finds limitations in learning. We evaluated an AI system that uses an Avatar (provides language and socially contingent interactions) and a robot (aids attention to the Avatar) to facilitate infants' ability to learn aspects of American Sign Language (ASL), and asked three questions: (1) Can babies with little/no exposure to ASL distinguish among the Avatar's different conversational modes (Linguistic Nursery Rhymes; Social Gestures; Idle/nonlinguistic postures; 3rd person observer)? (2) Can an Avatar stimulate babies' production of socially contingent responses, and crucially, nascent language responses? (3) What is the impact of parents' presence/absence of conversational participation? Surprisingly, babies (i) spontaneously distinguished among Avatar conversational modes, (ii) produced varied socially contingent responses to Avatar's modes, and (iii) parents influenced an increase in babies' response tokens to some Avatar modes, but the overall categories and pattern of babies' behavioral responses remained proportionately similar irrespective of parental participation. Of note, babies produced the greatest percentage of linguistic responses to the Avatar's Linguistic Nursery Rhymes versus other Avatar conversational modes. This work demonstrates the potential for Avatars to facilitate language learning in young babies.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

“…This study was part of a larger project designed to develop a system called RAVE (Robot AVatar thermal Enhanced language learning tool). RAVE is aimed to be an augmentative learning tool that can provide linguistic input, in particular visual language inputs, to facilitate language learning during one widely recognized critical developmental period for language (ages 6-12 months [97]) [96,[98][99][100][101]. To this end, thermal IR imaging was used to determine the emotional arousal and attentional valence, providing new knowledge about when infants are most optimally "Ready to Learn", even before the onset of language production.…”

A Linguistic Analysis Study for Teaching American-English Pronunciation: A Case Study Research

“…There are a number of influential theories that attempt to explain this miraculous and rapid linguistic growth by representing various sets of evidence (Byers-Heinlein et al, 2020;Dambudzo, 2018;Stahl & Feigenson, 2018), yet what seems obvious in the first language acquisition process is that infants have extraordinary abilities to promptly develop phonetic linguistic structures despite their immature cognitive skills (Gavin, 2006;Ramirez et al, 2017;Ramirez & Kuhl, 2017). When a child is in the process of mastering the first language, the ability to differentiate sounds as distinctive speech units enables children to separate semantic units for meaning making and communication (Nasihati et al, 2019;Singh, 2021;Singh et al, 2018;Visser et al, 2021). Without further knowledge, an unknown language is perceived by infants as a stream of sound clusters without recognizing separate semantic units for meaning making (Nasr, 1997;Clark, 2016), yet the unsettling fact that still remains a mystery is the rapid and highly accurate pronunciation development during infancy.…”

Thermal infrared imaging reveals that 6-12 month-old babies show different autonomic response to interaction with robot and avatar

“…Infants were positioned on their parents' laps, comfortably seated on a non-swiveling chair. The experimental design consisted of an initial period of familiarization [5][6][7]29]. Here, babies could interact with the environment and the experimental setup.…”

“…Virtual human and robotic agents tend to occupy a growing prevalence in daily life, with applications ranging from entertainment, training, learning, to tutorial systems [1][2][3]. Anthropomorphic virtual agents can be a powerful technological tool to impact motivational outcomes and facilitate learning, particularly in young children [4][5][6][7]. Especially, robots and avatars might play a primary role in this regard [8,9].…”

“…In the present study, we conducted an original investigation of the psychophysiological correlates of autonomic nervous system responses that occurred when infants were interacting with a robot as compared to a virtual human/avatar. The data were drawn from a larger innovative project to create an effective natural language learning tool for young infants, be they hearing or deaf, during the critical periods of human learning and development, called RAVE (or, Robot, AVatar, thermal Enhanced language learning tool) [5][6][7]). However, entirely new scientific questions and hypotheses are here addressed, and novel analyses that render original scientific findings are conducted.…”