Publisher Correction: Reproducible brain-wide association studies require thousands of individuals

Marek, Scott; Tervo‐Clemmens, Brenden; Calabro, Finnegan J.; Montez, David F.; Kay, Benjamin P.; Hatoum, Alexander S.; Donohue, Meghan Rose; Foran, William; Miller, Ruth H.; Hendrickson, Timothy; Malone, Stephen M.; Kandala, Sridhar; Feczko, Eric; Miranda-Domínguez, Óscar; Graham, Alice M.; Earl, Eric; Perrone, Anders; Cordova, Michaela; Doyle, Olivia; Moore, Lucille A; Conan, Greg; Uriarte, Johnny; Snider, Kathy; Lynch, Benjamin J.; Wilgenbusch, James C.; Pengo, Thomas; Tam, Angela; Chen, Jianzhong; Newbold, Dillan J.; Zheng, Annie; Seider, Nicole A.; Van, Andrew N.; Metoki, Athanasia; Chauvin, Roselyne J.; Laumann, Timothy O.; Greene, Deanna J.; Petersen, Steven E.; Garavan, Hugh; Thompson, Wesley K.; Nichols, Thomas E.; Yeo, B. T. Thomas; Barch, Deanna M.; Luna, Beatríz; Fair, Damien A.; Dosenbach, Nico U.F.

doi:10.1038/s41586-022-04692-3

Cited by 35 publications

(35 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Importantly, this cost savings becomes even more pronounced as subject sizes increase further to 100,000 subjects and beyond. Recent estimates have shown that large samples (n > 2,000) are required to discover robust brain-wide associations with behavior 1 . Our results demonstrate similar effects, showing how large sample sizes are helpful, but up to a point.…”

Section: Resultsmentioning

confidence: 99%

“…In many studies these effects are further compounded by suboptimal reliability in neuroimaging when best practices are not followed 9,11,46,[53][54][55][56][57] . We caution readers against interpreting the small effect sizes that are commonly found in large-scale samples 1,58 as definitive indications of weak or null effects. Furthermore, the replication crisis in neuroimaging is likely due in part to the greater variability in estimated effect sizes that stems from suboptimal joint reliability.…”

Section: Biomarker Discovery Depends On Joint Reliabilitymentioning

confidence: 96%

“…Biomedical researchers are increasingly recognizing that measurement reliability is a critical determinant of the reproducibility of scientific findings, as it mediates the relationship between sample size, statistical power, and replication between studies 1,2,6,7,[21][22][23][24][25][26] . In response, across a growing number of biological disciplines, researchers are arduously working to optimize the reliability of their assays or tools of choice (e.g., genetics, multimodal MRI, EEG) [3][4][5]27 .…”

Section: Introductionmentioning

confidence: 99%

“…Largely, critiques of low reproducibility in neuroimaging studies of individual differences have focused on the inadequacies of MRI data 1,3,46,47 , preprocessing pipelines 8 , and analytic methods 11,48,49 . However, optimizing reliability of one side of the equation (e.g., neuroimaging) without addressing reliability of the other (e.g., phenotyping) leads to underpowered and irreproducible results.…”

Section: Introductionmentioning

confidence: 99%

“… Summary Paragraph Biomarkers of behavior and psychiatric illness for cognitive and clinical neuroscience remain out of reach 1–4 . Suboptimal reliability of biological measurements, such as functional magnetic resonance imaging (fMRI), is increasingly cited as a primary culprit for discouragingly large sample size requirements and poor reproducibility of brain-based biomarker discovery 1,5–7 . In response, steps are being taken towards optimizing MRI reliability and increasing sample sizes 8–11 , though this will not be enough.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Suboptimal phenotypic reliability impedes reproducible human neuroscience

Nikolaidis

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

Biomarkers of behavior and psychiatric illness for cognitive and clinical neuroscience remain out of reach. Suboptimal reliability of biological measurements, such as functional magnetic resonance imaging (fMRI), is increasingly cited as a primary culprit for discouragingly large sample size requirements and poor reproducibility of brain-based biomarker discovery. In response, steps are being taken towards optimizing MRI reliability and increasing sample sizes, though this will not be enough. Optimizing biological measurement reliability and increasing sample sizes are necessary but insufficient steps for biomarker discovery; this focus has overlooked the ″other side of the equation″ — the reliability of clinical and cognitive assessments — which are often suboptimal or unassessed. Through a combination of simulation analysis and empirical studies using neuroimaging data, we demonstrate that the joint reliability of both biological and clinical/cognitive phenotypic measurements must be optimized in order to ensure biomarkers are reproducible and accurate. Even with best–case scenario high reliability neuroimaging measurements and large sample sizes, we show that suboptimal reliability of phenotypic data (i.e., clinical diagnosis, behavioral and cognitive measurements) will continue to impede meaningful biomarker discovery for the field. Improving reliability through development of novel assessments of phenotypic variation is needed, but it is not the sole solution. We emphasize the potential to improve the reliability of established phenotypic methods through aggregation across multiple raters and/or measurements, which is becoming increasingly feasible with recent innovations in data acquisition (e.g., web– and smart–phone-based administration, ecological momentary assessment, burst sampling, wearable devices, multimodal recordings). We demonstrate that such aggregation can achieve better biomarker discovery for a fraction of the cost engendered by large–scale samples. Although the current study has been motivated by ongoing developments in neuroimaging, the prioritization of reliable phenotyping will revolutionize neurobiological and clinical endeavors that are focused on brain and behavior.

show abstract

Section: Resultsmentioning

confidence: 99%