Teaching Social Communication Skills Through Human-Agent Interaction

Abstract

There are a large number of computer-based systems that aim to train and improve social skills. However, most of these do not resemble the training regimens used by human instructors. In this paper, we propose a computer-based training system that follows the procedure of social skills training, a well-established method to decrease human anxiety and discomfort in social interaction, and acquire social skills. In this paper, we attempt to automate the process of social skills training by developing a dialogue system named the "automated social skills trainer," which teaches social communication skills through human-agent interaction. The system includes a virtual avatar that recognizes user speech and language information and gives feedback to users. Its design is based on conventional social skills training performed by human participants, including defining target skills, modeling, role-play, feedback, reinforcement, and homework. We performed a series of three experiments investigating 1) the advantages of using computer-based training systems compared to human-human interaction by subjectively evaluating nervousness, ease of talking, and ability to talk well, 2) the relationship between speech language features and human social skills, and 3) the effect of computer-based training using our proposed system. Results of our first experiment show that interaction with an avatar decreases nervousness and increases the user's subjective impression of their ability to talk well compared to interaction with an unfamiliar person. The experimental evaluation measuring the relationship between social skill and speech and language features shows that these features have a relationship with social skills. Finally, experiments measuring the effect of performing social skills training with the proposed application show that participants significantly improve their skill, as assessed by separate evaluators, by using the system for 50 minutes. A user survey also shows that the users thought our system is useful and easy to use, and interaction with the avatar felt similar to human to human interaction.

Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders

“…Based on our previous work [ 16 ], we related the following extracted features to speech and language: F0 variation, amplitude, voice quality, pauses, words per minute, words over six letters, and fillers.…”

“…We previously developed a system called an automated social skills trainer that completely adheres to SST’s basic training model through an embodied conversational agent [ 16 ]. Based on extracted audio features, this system provides feedback for improving users’ social skills.…”

“…One gap is related to modality. Our previous work [ 16 ] considered only acoustic and linguistic features, and yet visual information (e.g., facial expression, head pose, and posture) is another essential feature of human-based SST [ 2 , 3 ] and public speaking aids [ 17 ].…”

Design of an Agent-Based Learning Environment for High-Risk Doorstep Scam Victims

“…Virtual agents are used for many different types of training programs, among which are social skills. For example the automated social skills trainer (ASST) [28], in which human-agent interaction takes place via user speech and language interaction. The proposed training focuses mostly on communication, previous research has been done in the same field, however no comparable virtual trainings are found.…”

Designing and Applying a Pedagogical Interaction Model in the Smart Cloud Platform

“…interaction and communication. The smart class supports teachers' effective teaching and students' wellorganized communication (Tanaka et al, 2016). It uses the cloud service to make thinking collisions between the individuals which improve the abilities to communicate interindividually.…”