Emoty: an Emotionally Sensitive Conversational Agent for People with Neurodevelopmental Disorders

Abstract

Our research aims at exploiting the advances in conversational technology to support people with Neurodevelopmental Disorder (NDD). NDD is a group of conditions that are characterized by severe deficits in the cognitive, emotional and motor areas and produce severe impairments in communication and social functioning. This paper presents the design, technology and exploratory evaluation of Emoty, a spoken Conversational Agent (CA) created specifically for individuals with NDD. The goal of Emoty is to help these persons enhancing communication abilities related to emotional recognition and expression, which are fundamental in any form of human relationship. The system exploits emotion detection capabilities based on the semantics of the speech by calling the IBM Watson Tone Analyzer API and from the harmonic features of the audio thanks to an "all-of-us" Deep Learning model. The design and evaluation of Emoty are based on the close collaboration among computer engineers and specialists in NDD

Emerging Technologies of Natural Language‐Enabled Chatbots: A Review and Trend Forecast Using Intelligent Ontology Extraction and Patent Analytics

“…Both types of applications are focused on using chatbot as a virtual assistant serving a specific purpose, or using chatbot as an expert in a specific field to achieve the purpose of knowledge acquisition. ese applications for providing utility or productivity are progressing towards education [63,64], medical [65], emotional [66,67], and social services [68][69][70]. Under these conditions, the integration of socioemotional behavior and personality processing design principles can lead to a decisive competitive advantage [71].…”

The Conversational Agent “Emoty” Perceived by People with Neurodevelopmental Disorders: Is It a Human or a Machine?

“…A conversational agent (CA) is any dialogue system able to interact with a human through natural language [6]. Emoty [4] is a spoken, emotionally sensitive CA that has been specifically designed as an emotional facilitator and trainer for individuals with NDD. Emoty entertains the users with small talks and asks them to verbalize sentences expressing specific emotions (joy, sadness, fear, anger, surprise, and neutrality) with their tone of voice.…”

Boris: a Spoken Conversational Agent for Music Production for People with Motor Disabilities

“…We presented Boris, a conversational agent for entertainment capable of producing a tuned song from a user's melody thanks to an original artificial musical intelligence. Boris's strengthen is that it responds not only according to the semantic content of the speech but also to the pitch analysis result, as already seen in a previous study with children with neurodevelopmental disorders [3]. Boris was explicitly designed for people with motor disabilities, as they cannot generally use musical instruments or physical musical interfaces due to accessibility issues.…”