Machine Learning to Support Visual Inspection of Data: A Clinical Application

Abstract

Practitioners in pediatric feeding programs often rely on single-case experimental designs and visual inspection to make treatment decisions (e.g., whether to change or keep a treatment in place). However, researchers have shown that this practice remains subjective, and there is no consensus yet on the best approach to support visual inspection results. To address this issue, we present the first application of a pediatric feeding treatment evaluation using machine learning to analyze treatment effects. A 5-year-old male with autism spectrum disorder participated in a 2-week home-based, behavior-analytic treatment program. We compared interrater agreement between machine learning and expert visual analysts on the effects of a pediatric feeding treatment within a modified reversal design. Both the visual analyst and the machine learning model generally agreed about the effectiveness of the treatment while overall agreement remained high. Overall, the results suggest that machine learning may provide additional support for the analysis of single-case experimental designs implemented in pediatric feeding treatment evaluations.

Agreement between visual inspection and objective analysis methods: A replication and extension

“…For visual inspection of each phase comparison ( N = 198), raters responded “yes” or “no” to the following question: “Would the change observed from one phase to the next be indicative of functional control of the behaviour in the planned direction (i.e., increase or decrease) if it were reversed and replicated?” Raters also provided a continuous value from 0 (certainty of no effect in the planned direction) to 10 (certainty of an effect in the planned direction), with 0 to 4 corresponding to “No” and 5 to 10 corresponding to “Yes” (Taylor & Lanovaz, 2021). Five doctoral‐level behavior analysts with PhDs in psychology (one professor, four practitioners) made ratings based on the blinded AB graphs (see ExpertA.xlsx, ExpertB.xlsx, ExpertC.xlsx, ExpertD.xlsx, and ExpertE.xlsx for complete analyses).…”

Lexicon-Based Sentiment Analysis in Behavioral Research

“…Overall, the lexicon-based approach will be more practical, adequate, and insight generating for most behavior analysts than the learning-based approach. We therefore focus exclusively on lexicon-based approaches in this discussion, referring interested readers to more comprehensive works for an introduction to machine learning-based strategies (e.g., Liu, 2020) and their recent application to behavior analysis (Bailey et al, 2021;Lanovaz et al, 2020;Lanovaz & Hranchuk, 2021;Taylor & Lanovaz, 2021;Turgeon & Lanovaz, 2020).…”