Applying Machine Learning to Facilitate Autism Diagnostics: Pitfalls and Promises

Abstract

Machine learning has immense potential to enhance diagnostic and intervention research in the behavioral sciences, and may be especially useful in investigations involving the highly prevalent and heterogeneous syndrome of autism spectrum disorder. However, use of machine learning in the absence of clinical domain expertise can be tenuous and lead to misinformed conclusions. To illustrate this concern, the current paper critically evaluates and attempts to reproduce results from two studies (Wall et al., 2012a; Wall et al., 2012b) that claim to drastically reduce time to diagnose autism using machine learning. Our failure to generate comparable findings to those reported by Wall and colleagues using larger and more balanced data underscores several conceptual and methodological problems associated with these studies. We conclude with proposed best-practices when using machine learning in autism research, and highlight some especially promising areas for collaborative work at the intersection of computational and behavioral science.

Use of machine learning to improve autism screening and diagnostic instruments: effectiveness, efficiency, and multi‐instrument fusion

“…Machine learning (ML) provides novel opportunities for human behavior research and clinical translation, yet its application can have noted pitfalls (Bone et al, 2015). In this work, we fastidiously utilize ML to derive autism spectrum disorder (ASD) instrument algorithms in an attempt to improve upon widely-used ASD screening and diagnostic tools.…”

“…If we can identify items or constructs that appear to be optimal at discriminating different groups of children with ASD, then we can focus new efforts on developing measures that build upon those constructs. 1 Importantly, however, results of certain studies seeking to improve ASD diagnostic instruments through ML have been largely invalid due to errors in problem formulation and ML utilization (Bone et al, 2015). These issues include: flawed assertion that administration time for the Autism Diagnostic Observation Schedule (ADOS; Lord et al, 2000) is reduced by minimizing the number of codes used; classification from instrument diagnosis rather than BEC; insufficient validation; and lack of generalization of results in replication experiments.…”

“…

Background Machine learning (ML) provides novel opportunities for human behavior research and clinical translation, yet its application can have noted pitfalls (Bone et al, 2015). In this work, we fastidiously utilize ML to derive autism spectrum disorder (ASD) instrument algorithms in an attempt to improve upon widely-used ASD screening and diagnostic tools.

Methods

The data consisted of Autism Diagnostic Interview-Revised (ADI-R) and Social Responsiveness Scale (SRS) scores for 1,264 verbal individuals with ASD and 462 verbal individuals with non-ASD developmental or psychiatric disorders (DD), split at age 10.

…”

Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic

“…Adults with ASD exhibit restricted/repetitive patterns of behavior (APA, 2013), but computational efforts to quantify the restricted/repetitive dimension in real-world contexts are just beginning to emerge (Rouhizadeh et al, 2015;Bone et al, 2015;Goodwin et al, 2014). This knowledge gap makes adult impairments difficult to treat, and tracking the effectiveness of interventions that target RRBs is a significant challenge for clinicians and researchers.…”

Introduction to Technologies in the Daily Lives of Individuals with Autism

“…From the laboratory to the clinic, to the daily lives of individuals on the spectrum, the new era of technology development in autism research may very well be defined by the massive volume and mind-boggling complexity of dense technology-acquired metrics of behavior. To manage such information streams, we will need to look carefully towards more powerful statistical and data mining approaches such as those highlighted by Bone et al (2014). However, as noted by Bone et al (2014), perhaps even more important than access to the right machine learning algorithms in this next evolution of technology will be the right perspectives, representing knowledge of where collected data comes from, what it means, and how it can be realistically used in practice.…”