Multimodal Ensemble Deep Learning to Predict Disruptive Behavior Disorders in Children

Abstract

Oppositional defiant disorder and conduct disorder, collectively referred to as disruptive behavior disorders (DBDs), are prevalent psychiatric disorders in children. Early diagnosis of DBDs is crucial because they can increase the risks of other mental health and substance use disorders without appropriate psychosocial interventions and treatment. However, diagnosing DBDs is challenging as they are often comorbid with other disorders, such as attention-deficit/hyperactivity disorder, anxiety, and depression. In this study, a multimodal ensemble three-dimensional convolutional neural network (3D CNN) deep learning model was used to classify children with DBDs and typically developing children. The study participants included 419 females and 681 males, aged 108–131 months who were enrolled in the Adolescent Brain Cognitive Development Study. Children were grouped based on the presence of DBDs (n = 550) and typically developing (n = 550); assessments were based on the scores from the Child Behavior Checklist and on the Schedule for Affective Disorders and Schizophrenia for School-age Children-Present and Lifetime version for DSM-5. The diffusion, structural, and resting-state functional magnetic resonance imaging (rs-fMRI) data were used as input data to the 3D CNN. The model achieved 72% accuracy in classifying children with DBDs with 70% sensitivity, 72% specificity, and an F1-score of 70. In addition, the discriminative power of the classifier was investigated by identifying the cortical and subcortical regions primarily involved in the prediction of DBDs using a gradient-weighted class activation mapping method. The classification results were compared with those obtained using the three neuroimaging modalities individually, and a connectome-based graph CNN and a multi-scale recurrent neural network using only the rs-fMRI data.

Evaluation of Risk of Bias in Neuroimaging-Based Artificial Intelligence Models for Psychiatric Diagnosis

“…Nearly half of the models (48.3% [268 of 555]) were found in studies authored by those with academic training in computers and data science (eTable 2 in Supplement 1). Schizophrenia (25.4% [141 of 555 models])…”

“…In the predictors domain, 187 of 555 models (33.7%; 95% CI, 29.9%- 37.6%) were rated with high ROB (Table 1). Defining predictors by knowing the outcome of these models was the unique source of the high ROB in this domain (ie, signaling question 2.2: were predictor assessments made without knowledge of outcome data?).…”

“…Defining predictors by knowing the outcome of these models was the unique source of the high ROB in this domain (ie, signaling question 2.2: were predictor assessments made without knowledge of outcome data?). In the outcome domain, high ROB was scored for 198 of 469 models (35.7%; 95% CI, 31.8%-39.7%) (Table 1). These models had a high ROB because the outcome knowledge of testing data sets was leaked into the predictors of the training set (ie, signaling question 3.3: were predictors excluded from the outcome definition?…”

“…High concern for clinical applicability in the participants domain was found for 2 of 555 models (0.4%; 95% CI, 0.00%-0.01%) . Furthermore, high concerns regarding clinical applicability were found for 199 of 555 models (35.9%; 95% CI, 31.9%-39.9%) in the predictors domain and for 217 of 555 models (39.1%; 95% CI, 35.0%-43.2%) in the outcome domain …”

“…Approximately half of the models (258 of 555 [46.48%; 95% CI, 42.3%- 50.6%]) were assessed for poor reporting quality in the technique domain across 3 signaling questions . The unique source for the degraded reporting quality in this domain was the absence of algorithmic details, which limited reproducibility (eg, signaling question: were details for algorithm development reported?).…”