Deep Learning Systems for Estimating Visual Attention in Robot-Assisted Therapy of Children with Autism and Intellectual Disability

Abstract

Recent studies suggest that some children with autism prefer robots as tutors for improving their social interaction and communication abilities which are impaired due to their disorder. Indeed, research has focused on developing a very promising form of intervention named Robot-Assisted Therapy. This area of intervention poses many challenges, including the necessary flexibility and adaptability to real unconstrained therapeutic settings, which are different from the constrained lab settings where most of the technology is typically tested. Among the most common impairments of children with autism and intellectual disability is social attention, which includes difficulties in establishing the correct visual focus of attention. This article presents an investigation on the use of novel deep learning neural network architectures for automatically estimating if the child is focusing their visual attention on the robot during a therapy session, which is an indicator of their engagement. To study the application, the authors gathered data from a clinical experiment in an unconstrained setting, which provided low-resolution videos recorded by the robot camera during the child-robot interaction. Two deep learning approaches are implemented in several variants and compared with a standard algorithm for face detection to verify the feasibility of estimating the status of the child directly from the robot sensors without relying on bulky external settings, which can distress the child with autism. One of the proposed approaches demonstrated a very high accuracy and it can be used for off-line continuous assessment during the therapy or for autonomously adapting the intervention in future robots with better computational capabilities.

Identification of Autism Spectrum Disorder (ASD) using Autoencoder

“…al. [4] had reported that from ABIDE dataset brain morphometric features were derived with the help of FreeSurfer and were applied on 3 classifiers namely Random Forest (RF), Support Vector Machine (SVM) and Gradient Boosting Machine (GBM). Experiments were conducted on datasets of individual sites and were showing higher accuracy of 97%.…”

A Framework of Hybrid Force/Motion Skills Learning for Robots

“…In many cases, cognitionenhanced robot learning ability is required. In some scenarios, robots are expected to work together with human beings, where compliant performance and human-friendly user experience are desired, e.g., [2], [3]; while in others, robots are expected to autonomously conduct tasks like human beings, e.g., [4]. By using game theory and policy iteration, the human being's control objective can be estimated in the human-robot interaction process [5].…”

An Open-Source Social Robot Based on Compliant Soft Robotics for Therapy with Children with ASD

“…In the context of ASD therapy, SAR has shown great advances and potential benefits in the development and applications of therapies for CwASD [6][7][8][9]. Specifically, SAR has been used to assist the diagnostic process and practice and improve social skills, such as eye contact and joint attention [10,11], emotion recognition [12,13], imitation [14], sharing simple activities, and increasing self-initiated interaction [15] to encourage basic verbal and nonverbal communication [16].…”