ChiCo: A Multimodal Corpus for the Study of Child Conversation

Abstract

The study of how children develop their conversational skills is an important scientific frontier at the crossroad of social, cognitive, and linguistic development with important applications in health, education, and child-oriented AI. While recent advances in machine learning techniques allow us to develop formal theories of conversational development in real-life contexts, progress has been slowed down by the lack of corpora that both approximate naturalistic interaction and provide clear access to children’s non-verbal behavior in face-to-face conversations. This work is an effort to fill this gap. We introduce ChiCo (for Child Conversation), a corpus we built using an online video chat system. Using a weakly structured task (a word-guessing game), we recorded 20 conversations involving either children in middle childhood (i.e., 6 to 12 years old) interacting with their caregivers (condition of interest) or the same caregivers interacting with other adults (a control condition), resulting in 40 individual recordings. Our annotation of these videos has shown that the frequency of children’s use of gaze, gesture, and facial expressions mirrors that of adults. Future modeling research can capitalize on this rich behavioral data to study how both verbal and non-verbal cues contribute to the development of conversational coordination

Understanding Children's Multimodal Conversational Development: Challenges and Opportunities

“…This data has been used to compare childcaregiver dynamics to that of adult dyads -both recorded using the same paradigm. They found both similarities (e.g., in terms of multimodal feedback signaling) and differences (e.g., in terms of multimodal alignment) between these dyads [2,3,22]. While this work has proven this costefficient data acquisition method to be promising, more research is needed to understand its full potential.…”

“…aspects are specific to the mediated one. 2 Further, the authors focused on school-age children; it is unknown whether a similarly cost-efficient design could be used for younger children as well.…”

Caregivers Exaggerate Their Lexical Alignment to Young Children Across Several Cultures

“…Another limitation is that we focused only on one aspect of alignment (lexical repetition) which provides a partial view of how interlocutors align to each other multimodally in social interaction (Rasenberg et al, 2020). A more comprehensive investigation would require using child-caregiver corpora that facilitate the study of multimodal face-toface conversations (e.g., Bodur et al, 2021Bodur et al, , 2022.…”

Social Interaction Data-sets in the Age of Covid-19: a Case Study on Digital Commensality

“…Moving on to social interaction studies, [16] describes deception detection in group video conversations and points out the importance of considering such a kind of interaction, as video calls are becoming increasingly common after the Covid-19 hit. [3] recorded, through video-calls, conversations between children and their caregivers to investigate gaze, gesture, and facial expressions. Similarly, [26] created a corpus of parentchildren speech interactions fully collected online and advocates for such an approach to overcome the limitations caused by lockdown policies.…”