Affect sensing in an affective interactive e-theatre for autistic children

Abstract

Abstract:In this paper we describe an interactive emotional social virtual framework for autistic young people age 11 -14 to learn verbal and non-verbal emotional expression in role-play situations. In order to provide an interactive learning environment with some degree of automatic metaphorical understanding from open-ended text input, new developments on affect detection for the processing of several metaphorical languages have been presented. The emotional gesture and facial animation have been created for users' avatars and activated by affective states detected from users' text input. Emotion appraisal and affect detection functions have been embodied in an intelligent conversational agent, who interacts with human users for drama improvisation. We have also conducted user testing with 24 autistic children to evaluate the overall framework. The work has the potential to improve autistic young people's ability in language learning and their social connection with other people. It contributes to the conference themes on natural language processing, interactive affective social interfaces and user studies of intelligent interfaces.

ICTs in the Classroom, Multiliteracy and Special Education: A Required Interface

“…Several research studies were developed to help nonverbal autistic (Kanner, 1943) people communicate using smartphones to express their feelings and needs (Zhang, 2010). Other researchers developed applications to help students with communication difficulties with their learning inside and outside the classroom (El-Seoud, Samir, Karkar, Ja'Am, & Karam, 2014;Silva, Prado, Scardovelli, Bochi, Campos, & Frère, 2014;Scardovelli & Frere, 2015).…”