Magnetic resonance imaging (MRI) has transformed our understanding of the human brain through well-replicated mapping of abilities to specific structures (for example, lesion studies) and functions1–3 (for example, task functional MRI (fMRI)). Mental health research and care have yet to realize similar advances from MRI. A primary challenge has been replicating associations between inter-individual differences in brain structure or function and complex cognitive or mental health phenotypes (brain-wide association studies (BWAS)). Such BWAS have typically relied on sample sizes appropriate for classical brain mapping4 (the median neuroimaging study sample size is about 25), but potentially too small for capturing reproducible brain–behavioural phenotype associations5,6. Here we used three of the largest neuroimaging datasets currently available—with a total sample size of around 50,000 individuals—to quantify BWAS effect sizes and reproducibility as a function of sample size. BWAS associations were smaller than previously thought, resulting in statistically underpowered studies, inflated effect sizes and replication failures at typical sample sizes. As sample sizes grew into the thousands, replication rates began to improve and effect size inflation decreased. More robust BWAS effects were detected for functional MRI (versus structural), cognitive tests (versus mental health questionnaires) and multivariate methods (versus univariate). Smaller than expected brain–phenotype associations and variability across population subsamples can explain widespread BWAS replication failures. In contrast to non-BWAS approaches with larger effects (for example, lesions, interventions and within-person), BWAS reproducibility requires samples with thousands of individuals.

Magnetic resonance imaging (MRI) continues to drive many important neuroscientific advances. However, progress in uncovering reproducible associations between individual differences in brain structure/function and behavioral phenotypes (e.g., cognition, mental health) may have been undermined by typical neuroimaging sample sizes (median N=25)1,2. Leveraging the Adolescent Brain Cognitive Development (ABCD) Study3 (N=11,878), we estimated the effect sizes and reproducibility of these brain wide associations studies (BWAS) as a function of sample size. The very largest, replicable brain wide associations for univariate and multivariate methods were r=0.14 and r=0.34, respectively. In smaller samples, typical for brain wide association studies, irreproducible, inflated effect sizes were ubiquitous, no matter the method (univariate, multivariate). Until sample sizes started to approach consortium levels, BWAS were underpowered and statistical errors assured. Multiple factors contribute to replication failures4,5,6; here, we show that the pairing of small brain behavioral phenotype effect sizes with sampling variability is a key element in widespread BWAS replication failure. Brain behavioral phenotype associations stabilize and become more reproducible with sample sizes of N>2,000. While investigator initiated brain behavior research continues to generate hypotheses and propel innovation, large consortia are needed to usher in a new era of reproducible human brain wide association studies.

The 21-site Adolescent Brain Cognitive Development (ABCD) study provides an unparalleled opportunity to characterize functional brain development via resting-state functional connectivity (RSFC) and to quantify relationships between RSFC and behavior. This multi-site data set includes potentially confounding sources of variance, such as differences between data collection sites and/or scanner manufacturers, in addition to those inherent to RSFC (e.g., head motion). The ABCD project provides a framework for characterizing and reproducing RSFC and RSFC-behavior associations, while quantifying the extent to which sources of variability bias RSFC estimates. We quantified RSFC and functional network architecture in 2,188 9–10-year old children from the ABCD study, segregated into demographically-matched discovery (N =1,166) and replication datasets (N = 1,022). We found RSFC and network architecture to be highly reproducible across children. We did not observe strong effects of site; however, scanner manufacturer effects were large, reproducible, and followed a “short-to-long” association with distance between regions. Accounting for potential confounding variables, we replicated that RSFC between several higher-order networks was related to general cognition. In sum, we provide a framework for how to characterize RSFC-behavior relationships in a rigorous and reproducible manner using the ABCD dataset and other large multi-site projects.

The Adolescent Brain Cognitive Development Study (ABCD), a 10 year longitudinal neuroimaging study of the largest population based and demographically distributed cohort of 9-10 year olds (N=11,877), was designed to overcome reproducibility limitations of prior child mental health studies. Besides the fantastic wealth of research opportunities, the extremely large size of the ABCD data set also creates enormous data storage, processing, and analysis challenges for researchers. To ensure data privacy and safety, researchers are not currently able to share neuroimaging data derivatives through the central repository at the National Data Archive (NDA). However, sharing derived data amongst researchers laterally can powerfully accelerate scientific progress, to ensure the maximum public benefit is derived from the ABCD study. To simultaneously promote collaboration and data safety, we developed the ABCD-BIDS Community Collection (ABCC), which includes both curated processed data and software utilities for further analyses. The ABCC also enables researchers to upload their own custom-processed versions of ABCD data and derivatives for sharing with the research community. This NeuroResource is meant to serve as the companion guide for the ABCC. In section we describe the ABCC. Section II highlights ABCC utilities that help researchers access, share, and analyze ABCD data, while section III provides two exemplar reproducibility analyses using ABCC utilities. We hope that adoption of the ABCC’s data-safe, open-science framework will boost access and reproducibility, thus facilitating progress in child and adolescent mental health research.

No abstract