Collaborative Storytelling between Robot and Child

Sun, Ming; Leite, Iolanda; Lehman, Jill Fain; Li, Boyang

doi:10.1145/3078072.3079714

Cited by 29 publications

(11 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [25], the authors implemented an intelligent virtual environment with many Embodied Conversational Agents, for improving speaking and listening skills of non-native English language learners. Finally, [50] reports about CAs to engage children in book reading activities and to create oral stories in a highly interactive manner.…”

Section: Related Workmentioning

confidence: 99%

Conversational Agents to Promote Children’s Verbal Communication Skills

Catania

Spitale

Cosentino

et al. 2021

Chatbot Research and Design

View full text Add to dashboard Cite

The fundamentals of verbal communication skills are developed during childhood, and existing studies pinpoint the benefits of stimulating language and expression skills from an early age. Our research is a preliminary evaluation of conversational technology to support this process. In this paper, we describe the design process of a speech-based conversational agent for children, which involved a Wizard-of-Oz empirical study with 20 primary school children aged 9-10 y.o. in order to identify the design guidelines for the automated version of the system. Our agent is called ISI, is integrated into a web application and exploits oral and visual interaction modes. ISI enables children to practice verbal skills related to the description of a person's physical characteristics. It provides opportunities for them to learn and use words and linguistic constructs. Also, ISI permits to develop their body awareness and selfexpression (when describing their self) or the attention to "the other" (when describing someone else). ISI engages users in a speech-based conversational flow composed of two main repeated steps. It talks to the children and stimulates them with questions about a specific part of their body (e.g., "What color is your hair?"). When the users describe the required feature adequately, ISI provides a cheerful real-time visual representation of the answer; otherwise, it provides hints.

show abstract

Section: Related Workmentioning

confidence: 99%

Conversational Agents to Promote Children’s Verbal Communication Skills

Catania

Spitale

Cosentino

et al. 2021

Chatbot Research and Design

View full text Add to dashboard Cite

show abstract

“…However, the interactive component of these scenarios remains limited, and has been the focus of recent investigation ( Paradeda et al, 2017 ; Park et al, 2017 ; Ligthart et al, 2020 ). In particular, Sun et al (2017) introduced a collaborative storytelling scenario in which both the robot and the child contribute to create a story. While most storytelling scenarios focus on the expressivity of the robot and the educational goals, in this work we introduced novel ways of introducing interactive engagement in robotic storytelling, through the use of screens that illustrate and support the story with engaging video snippets.…”

Section: Related Workmentioning

confidence: 99%

“Sequencing Matters”: Investigating Suitable Action Sequences in Robot-Assisted Autism Therapy

et al. 2022

View full text Add to dashboard Cite

Social robots have been shown to be promising tools for delivering therapeutic tasks for children with Autism Spectrum Disorder (ASD). However, their efficacy is currently limited by a lack of flexibility of the robot’s social behavior to successfully meet therapeutic and interaction goals. Robot-assisted interventions are often based on structured tasks where the robot sequentially guides the child towards the task goal. Motivated by a need for personalization to accommodate a diverse set of children profiles, this paper investigates the effect of different robot action sequences in structured socially interactive tasks targeting attention skills in children with different ASD profiles. Based on an autism diagnostic tool, we devised a robotic prompting scheme on a NAO humanoid robot, aimed at eliciting goal behaviors from the child, and integrated it in a novel interactive storytelling scenario involving screens. We programmed the robot to operate in three different modes: diagnostic-inspired (Assess), personalized therapy-inspired (Therapy), and random (Explore). Our exploratory study with 11 young children with ASD highlights the usefulness and limitations of each mode according to different possible interaction goals, and paves the way towards more complex methods for balancing short-term and long-term goals in personalized robot-assisted therapy.

show abstract

“…Finally, (Sun et al, 2017) investigated joint storytelling with children participants, where a child participant created a story by selecting from a predefined set of characters and settings, a human experimenter made suggestions on story continuations following a limited number of engagement strategies, and a puppeteered robot later helped child participants recall the stories they created in order to evaluate the efficacy of the engagement strategies. Our task setting is different in that it focuses on having an AI agent automatically generating story continuations without restrictions in settings or content rather than on evaluating strategies for humans to engage with other human storytellers.…”

Section: Storytelling With Robotsmentioning

confidence: 99%

Design and Analysis of a Collaborative Story Generation Game for Social Robots

Nichols

Gao²,

Vasylkiv

et al. 2021

Front. Comput. Sci.

View full text Add to dashboard Cite

Storytelling plays a central role in human socializing and entertainment, and research on conducting storytelling with robots is gaining interest. However, much of this research assumes that story content is curated. In this paper, we introduce the task of collaborative story generation, where an artificial intelligence agent, or a robot, and a person collaborate to create a unique story by taking turns adding to it. We present a collaborative story generation system which works with a human storyteller to create a story by generating new utterances based on the story so far. Our collaborative story generation system consists of a publicly-available large scale language model that was tuned on a dataset of writing prompts and short stories, and a ranker that samples from the language model and chooses the best possible output. We improve storytelling quality by optimizing the ranker’s sample size to strike a balance between quality and computational cost. Since latency can be detrimental to human-robot interaction, we examine the performance-latency trade-offs of our approach and find the optimal ranker sample size that strikes the best balance between quality and computational cost. We evaluate our system by having human participants play the collaborative story generation game and comparing the stories they create with our system to a naive baseline. Next, we conduct a detailed elicitation survey that sheds light on issues to consider when adapting our collaborative story generation system to a social robot. Finally, in a first step towards allowing human players to control the genre or mood of stories generated, we present preliminary work on steering story generation sentiment polarity with a sentiment analysis model. We find that our proposed method achieves a good balance of steering capability and text coherence. Our evaluation shows that participants have a positive view of collaborative story generation with a social robot and consider rich, emotive capabilities to be key to an enjoyable experience.

show abstract

Collaborative Storytelling between Robot and Child

Cited by 29 publications

References 22 publications

Conversational Agents to Promote Children’s Verbal Communication Skills

Conversational Agents to Promote Children’s Verbal Communication Skills

“Sequencing Matters”: Investigating Suitable Action Sequences in Robot-Assisted Autism Therapy

Design and Analysis of a Collaborative Story Generation Game for Social Robots

Contact Info

Product

Resources

About