ADHD diagnosis from multiple data sources with batch effects

Abstract

The Attention Deficit Hyperactivity Disorder (ADHD) affects the school-age population and has large social costs. The scientific community is still lacking a pathophysiological model of the disorder and there are no objective biomarkers to support the diagnosis. In 2011 the ADHD-200 Consortium provided a rich, heterogeneous neuroimaging dataset aimed at studying neural correlates of ADHD and to promote the development of systems for automated diagnosis. Concurrently a competition was set up with the goal of addressing the wide range of different types of data for the accurate prediction of the presence of ADHD. Phenotypic information, structural magnetic resonance imaging (MRI) scans and resting state fMRI recordings were provided for nearly 1000 typical and non-typical young individuals. Data were collected by eight different research centers in the consortium. This work is not concerned with the main task of the contest, i.e., achieving a high prediction accuracy on the competition dataset, but we rather address the proper handling of such a heterogeneous dataset when performing classification-based analysis. Our interest lies in the clustered structure of the data causing the so-called batch effects which have strong impact when assessing the performance of classifiers built on the ADHD-200 dataset. We propose a method to eliminate the biases introduced by such batch effects. Its application on the ADHD-200 dataset generates such a significant drop in prediction accuracy that most of the conclusions from a standard analysis had to be revised. In addition we propose to adopt the dissimilarity representation to set up effective representation spaces for the heterogeneous ADHD-200 dataset. Moreover we propose to evaluate the quality of predictions through a recently proposed test of independence in order to cope with the unbalancedness of the dataset.

Non-negative matrix factorization of multimodal MRI, fMRI and phenotypic data reveals differential changes in default mode subnetworks in ADHD

“…To provide data for this competition, one of the largest multisite data consortiums was initiated to provide open access to data from nearly a thousand children and adolescents with ADHD as well as age-matched controls. This dataset has been much published on in a short time [64] [45] [65] [8] [12] [48], allowing a direct comparison of the methodology and the common problems they all faced.…”

A High-Throughput Pipeline Identifies Robust Connectomes But Troublesome Variability

“…Third, there is some evidence machine learning approaches to mitigating batch e ects can be e ective as well, but so far only in fMRI data and only using four, rather than 17 studies [79]. Fourth, pre-processing strategies have been employed to improve multi-site reliability [80], so implementing methods such as these within NDMG could possibly mitigate some batch effect, at the risk of reducing accuracy and/or reliability [81].…”

The Neuro Bureau ADHD-200 Preprocessed Repository

“…The two biggest challenges for using the ADHD-200 Preprocessed data are head motion [53][54][55][56][57][58] and inter-site variation in the acquisition equipment, parameters, and experimental procedures [59,60]. A variety of different approaches have been proposed for addressing head motion in hyperkinetic populations [53,61], and in the ADHD-200 Sample in particular [56], that should be considered when analyzing the data.…”

“…At the very least, some statistic that characterizes individual motion (such as root mean square deviation [62]) should be included as a nuisance regressor in the group-level model [55,57]. Differences in the manner in which data was collected at each site can introduce addition and multiplicative effects (batch effects) to the data, which may obscure the underlying biological signal [59,60]. Including a regressor for acquisition protocol (see Tables 2 and 3 for a summary of the different protocols), the average pairwise correlation between all regions in the brain (GCOR) [63], or the whole-brain average of the feature under inquiry [60], have all been shown to be effective for dealing with inter-site variation.…”