How sample size influences the replicability of task-based fMRI

Bo, Turner; Ej, Paul; Mb, Miller; Barbey, Aron K.

doi:10.1101/136259

Cited by 8 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Still, Paul et al demonstrated recently that even large fMRI studies (i.e. N = 100) do not produce group results with good reliability (Paul, Turner, Miller, & Barbey, 2017). Furthermore, previous studies showed that group-level stability is not indicative of individual stability (Raemaekers et al, 2007;van den Bulk et al, 2013;Vetter, Pilhatsch, Weigelt, Ripke, & Smolka, 2015), while research focusing on within-subject reliability over time has produced mixed results.…”

Section: Introductionmentioning

confidence: 99%

Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects

et al. 2019

View full text Add to dashboard Cite

To cast valid predictions of future behavior or diagnose disorders, the reliable measurement of a "biomarker" such as the brain activation to prospective reward is a prerequisite. Surprisingly, only a small fraction of functional magnetic resonance imaging (fMRI) studies report or cite the reliability of brain activation maps involved in group analyses. Here, using simulations and exemplary longitudinal data of 126 healthy adolescents performing an intertemporal choice task, we demonstrate that reproducing a group activation map over time is not a sufficient indication of reliable measurements at the individual level. Instead, selecting regions based on significant main effects at the group level may yield estimates that fail to reliably capture individual variance in the subjective evaluation of an offer. Collectively, our results call for more attention on the reliability of supposed biomarkers at the level of the individual. Thus, caution is warranted in employing brain activation patterns prematurely for clinical applications such as diagnosis or tailored interventions before their reliability has been conclusively established by large-scale studies. To facilitate assessing and reporting of the reliability of fMRI contrasts in future studies, we provide a toolbox that incorporates common measures of global and local reliability..

show abstract

Section: Introductionmentioning

confidence: 99%

Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Our study has four main limitations. First, the sample size (n = 13) is relatively small, compared to the recommendations for a typical task-based FMRI study (n = 30;Turner et al, 2018). Our sample size was not based on a formal sample size calculation because we were unable to find effect sizes for an FMRI experiment similar to ours.…”

Section: Methodological Considerationsmentioning

confidence: 92%

Functional MRI of Native and Non-native Speech Sound Production in Sequential German-English Bilinguals

Treutler

Sörös

2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Bilingualism and multilingualism are highly prevalent. Non-invasive brain imaging has been used to study the neural correlates of native and non-native speech and language production, mainly on the lexical and syntactic level. Here, we acquired continuous fast event-related FMRI during visually cued overt production of exclusively German and English vowels and syllables. We analyzed data from 13 university students, native speakers of German and sequential English bilinguals. The production of non-native English sounds was associated with increased activity of the left primary sensorimotor cortex, bilateral cerebellar hemispheres (lobule VI), left inferior frontal gyrus, and left anterior insula compared to native German sounds. The contrast German > English sounds was not statistically significant. Our results emphasize that the production of non-native speech requires additional neural resources already on a basic phonological level in sequential bilinguals.

show abstract

“…For generalizability considerations, there should also be a floor for both subject and trial sample sizes as a rule of thumb. Various recommendations have been proposed for the minimum number of subjects, ranging (at present) from 20 (Thirion et al, 2007) to 100 (Turner et al, 2018); these numbers are likely to depend strongly on experimental design, tasks, region(s) of interest and other specific considerations. Similarly, no single minimum number of trials can be recommended across all studies, for the same reasons.…”

Section: Beyond Efficiency: Trial Counts For Generalizability Replicability Power and Reliabilitymentioning

confidence: 99%

Hyperbolic trade-off: the importance of balancing trial and subject sample sizes in neuroimaging

Chen

Pine

Brotman

et al. 2021

Preprint

View full text Add to dashboard Cite

Big data initiatives have gained popularity for leveraging a large sample of subjects to study a wide range of effect magnitudes in the brain. On the other hand, most task-based FMRI designs feature relatively small number of subjects, so that resulting parameter estimates may be associated with compromised precision. Nevertheless, little attention has been given to another important dimension of experimental design, which can equally boost a study's statistical efficiency: the trial sample size. Here, we systematically explore the different factors that impact effect uncertainty, drawing on evidence from hierarchical modeling, simulations and an FMRI dataset of 42 subjects who completed a large number of trials of a commonly used cognitive task. We find that, due to the presence of relatively large cross-trial variability: 1) trial sample size has nearly the same impact as subject sample size on statistical efficiency; 2) increasing both trials and subjects improves statistical efficiency more effectively than focusing on subjects alone; 3) trial sample size can be leveraged with the number of subjects to improve the cost-effectiveness of an experimental design; 4) for small trial sample sizes, rather than the common practice of condition-level modeling through summary statistics, trial-level modeling may be necessary to accurately assess the standard error of an effect estimate. Lastly, we make practical recommendations for improving experimental designs across neuroimaging and behavioral studies.

show abstract

How sample size influences the replicability of task-based fMRI

Cited by 8 publications

References 54 publications

Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects

Addressing the reliability fallacy in fMRI: Similar group effects may arise from unreliable individual effects

Functional MRI of Native and Non-native Speech Sound Production in Sequential German-English Bilinguals

Hyperbolic trade-off: the importance of balancing trial and subject sample sizes in neuroimaging

Contact Info

Product

Resources

About