Magnetic resonance imaging (MRI) has transformed our understanding of the human brain through well-replicated mapping of abilities to specific structures (for example, lesion studies) and functions1–3 (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI. A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping4 (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brain–behavioural phenotype associations5,6. Here we used three of the largest neuroimaging datasets currently available—with a total sample size of around 50,000 individuals—to quantify BWAS effect sizes and reproducibility as a function of sample size. BWAS associations were smaller than previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes. As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate). Smaller than expected brain–phenotype associations and variability across population subsamples can explain widespread BWAS replication failures. In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals.

Magnetic resonance imaging (MRI) continues to drive many important neuroscientific advances. However, progress in uncovering reproducible associations between individual differences in brain structure/function and behavioral phenotypes (e.g., cognition, mental health) may have been undermined by typical neuroimaging sample sizes (median N=25)1,2. Leveraging the Adolescent Brain Cognitive Development (ABCD) Study3 (N=11,878), we estimated the effect sizes and reproducibility of these brain wide associations studies (BWAS) as a function of sample size. The very largest, replicable brain wide associations for univariate and multivariate methods were r=0.14 and r=0.34, respectively. In smaller samples, typical for brain wide association studies, irreproducible, inflated effect sizes were ubiquitous, no matter the method (univariate, multivariate). Until sample sizes started to approach consortium levels, BWAS were underpowered and statistical errors assured. Multiple factors contribute to replication failures4,5,6; here, we show that the pairing of small brain behavioral phenotype effect sizes with sampling variability is a key element in widespread BWAS replication failure. Brain behavioral phenotype associations stabilize and become more reproducible with sample sizes of N>2,000. While investigator initiated brain behavior research continues to generate hypotheses and propel innovation, large consortia are needed to usher in a new era of reproducible human brain wide association studies.

The 21-site Adolescent Brain Cognitive Development (ABCD) study provides an unparalleled opportunity to characterize functional brain development via resting-state functional connectivity (RSFC) and to quantify relationships between RSFC and behavior. This multi-site data set includes potentially confounding sources of variance, such as differences between data collection sites and/or scanner manufacturers, in addition to those inherent to RSFC (e.g., head motion). The ABCD project provides a framework for characterizing and reproducing RSFC and RSFC-behavior associations, while quantifying the extent to which sources of variability bias RSFC estimates. We quantified RSFC and functional network architecture in 2,188 9–10-year old children from the ABCD study, segregated into demographically-matched discovery (N =1,166) and replication datasets (N = 1,022). We found RSFC and network architecture to be highly reproducible across children. We did not observe strong effects of site; however, scanner manufacturer effects were large, reproducible, and followed a “short-to-long” association with distance between regions. Accounting for potential confounding variables, we replicated that RSFC between several higher-order networks was related to general cognition. In sum, we provide a framework for how to characterize RSFC-behavior relationships in a rigorous and reproducible manner using the ABCD dataset and other large multi-site projects.

Attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) are commonly comorbid, share genetic liability, and often exhibit overlapping cognitive impairments. Clarification of shared and distinct cognitive effects while considering comorbid symptoms across disorders has been lacking. In the current study, children ages 7-15 years assigned to three diagnostic groups:ADHD (n = 509), ASD (n = 97), and controls (n = 301) completed measures spanning the cognitive domains of attention/arousal, working memory, set-shifting, inhibition, and response variability. Specific processes contributing to response variability were examined using a drift diffusion model, which separately quantified drift rate (i.e., efficiency of information processing), boundary separation (i.e., speed-accuracy trade-offs), and non-decision time. Children with ADHD and ASD were impaired on attention/arousal, processing speed, working memory, and response inhibition, but did not differ from controls on measures of delayed reward discounting, set-shifting, or interference control. Overall, impairments in the ASD group were not attributable to ADHD symptoms using either continuous symptom measures or latent categorical grouping approaches. Similarly, impairments in the ADHD group were not attributable to ASD symptoms. When specific RT parameters were considered, children with ADHD and ASD shared impairments in drift rate. However, children with ASD were uniquely characterized by a wider boundary separation. Findings suggest a combination of overlapping and unique patterns of cognitive impairment for children with ASD as compared to those with ADHD, particularly when the processes underlying reaction time measures are considered separately.

DSM-5 Autism Spectrum Disorder (ASD) comprises a set of neurodevelopmental disorders characterized by deficits in social communication and interaction and repetitive behaviors or restricted interests, and may both affect and be affected by multiple cognitive mechanisms. This study attempts to identify and characterize cognitive subtypes within the ASD population using our Functional Random Forest (FRF) machine learning classification model. This model trained a traditional random forest model on measures from seven tasks that reflect multiple levels of information processing. 47 ASD diagnosed and 58 typically developing (TD) children between the ages of 9 and 13 participated in this study. Our RF model was 72.7% accurate, with 80.7% specificity and 63.1% sensitivity. Using the random forest model, the FRF then measures the proximity of each subject to every other subject, generating a distance matrix between participants. This matrix is then used in a community detection algorithm to identify subgroups within the ASD and TD groups, and revealed 3 ASD and 4 TD putative subgroups with unique behavioral profiles. We then examined differences in functional brain systems between diagnostic groups and putative subgroups using resting-state functional connectivity magnetic resonance imaging (rsfcMRI). Chi-square tests revealed a significantly greater number of between group differences (p < .05) within the cingulo-opercular, visual, and default systems as well as differences in inter-system connections in the somato-motor, dorsal attention, and subcortical systems. Many of these differences were primarily driven by specific subgroups suggesting that our method could potentially parse the variation in brain mechanisms affected by ASD.