Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social Communication and Behavioral Coaching

Abstract

BackgroundAutism spectrum disorder (ASD) is a childhood-onset neurodevelopmental disorder with a rapidly rising prevalence, currently affecting 1 in 68 children, and over 3.5 million people in the United States. Current ASD interventions are primarily based on in-person behavioral therapies that are both costly and difficult to access. These interventions aim to address some of the fundamental deficits that clinically characterize ASD, including deficits in social communication, and the presence of stereotypies, and other autism-related behaviors. Current diagnostic and therapeutic approaches seldom rely on quantitative data measures of symptomatology, severity, or condition trajectory.MethodsGiven the current situation, we report on the Brain Power System (BPS), a digital behavioral aid with quantitative data gathering and reporting features. The BPS includes customized smartglasses, providing targeted personalized coaching experiences through a family of gamified augmented-reality applications utilizing artificial intelligence. These applications provide children and adults with coaching for emotion recognition, face directed gaze, eye contact, and behavioral self-regulation. This preliminary case report, part of a larger set of upcoming research reports, explores the feasibility of the BPS to provide coaching in two boys with clinically diagnosed ASD, aged 8 and 9 years.ResultsThe coaching intervention was found to be well tolerated and rated as being both engaging and fun. Both males could easily use the system, and no technical problems were noted. During the intervention, caregivers reported improved non-verbal communication, eye contact, and social engagement during the intervention. Both boys demonstrated decreased symptoms of ASD, as measured by the aberrant behavior checklist at 24-h post-intervention. Specifically, both cases demonstrated improvements in irritability, lethargy, stereotypy, hyperactivity/non-compliance, and inappropriate speech.ConclusionSmartglasses using augmented reality may have an important future role in helping address the therapeutic needs of children with ASD. Quantitative data gathering from such sensor-rich systems may allow for digital phenotyping and the refinement of social communication constructs of the research domain criteria. This report provides evidence for the feasibility, usability, and tolerability of one such specialized smartglasses system.

Technology-Facilitated Diagnosis and Treatment of Individuals with Autism Spectrum Disorder: An Engineering Perspective

“…We have also seen research on using facial expressions recognition to obtain affective feedback [30,[111][112][113][114][115][116]. In some research, head orientation and movement are tracked using inertial sensors embedded in smart glasses instead of cameras [117]. While the tracking itself helps tremendously in producing realistic virtual avatar faces, there is still an open research issue in how to accurately understand the captured motion with respect to nonverbal communicative behaviors of a participant with the exception of eye gaze (and to some extend facial expressions).…”

Second Version of Google Glass as a Wearable Socio-Affective Aid: Positive School Desirability, High Usability, and Theoretical Framework in a Sample of Children with Autism

“…While there are many different types of assistive technology, the portability, capability, and ubiquity of smartphone and tablet devices has led to considerable growth in assistive apps for these devices [3,4]. More recent technological advances have resulted in lightweight smartglasses: face-worn computers with a visual display and in-built sensors [5-7] that can also deliver assistive apps [8,9]. …”

“…They are also sensor-rich and can collect a wide range of quantitative user data [9,11,12]. These data can be monitored and analyzed on a real-time basis, allowing for the smartglasses to dynamically change the user experience to optimize learning—effectively placing the user and the smartglasses in a closed feedback loop [8,13,14]. Given the proximity of smartglasses to the sensory organs contained in the human head, this type of computing may enable a higher level of human-computer interaction than other devices [13].…”

“…Given the proximity of smartglasses to the sensory organs contained in the human head, this type of computing may enable a higher level of human-computer interaction than other devices [13]. Smartglasses are already being developed as a social and behavioral communication aid for people with ASD [8,15,16]. …”

“…Glass, like its predecessor, is a head-mounted, wearable computer that has demonstrated utility in a variety of situations where operating a computer hands-free and while heads-up is of particular advantage. Glass has been creatively developed as a technology that can deliver social and cognitive skills coaching to children and adults with ASD [8]. To our knowledge, we have reported on the first studies of ASD-related software on the original Google Glass (Explorer Edition) [8,9,15,16], and here we present the first appearance of Glass (Enterprise Edition) in the literature.…”