The statistical crisis in science: how is it relevant to clinical neuropsychology?

Gelman, Andrew; Geurts, Hilde M.

doi:10.1080/13854046.2016.1277557

Cited by 34 publications

(25 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In light of this empirical consideration, our own findings of poor reliability might be considered as reflecting spurious results that can arise when VOI-based structural brain behavior correlations are performed in healthy adult cohorts with possibly noisy measurements. Thus, together and in line with the recent replication crisis Gelman and Geurts 2017), the present study emphasizes that low effect size in noisy brain-behavior correlation should be taken with caution rather than indicative of a robust feature of brain-behavior relationship.…”

Section: Significant Structural Brain Behavior Correlation and The Nementioning

confidence: 51%

Searching for behavior relating to grey matter volume in a-priori defined right dorsal premotor regions: Lessons learned

et al. 2017

View full text Add to dashboard Cite

(max 250 words)Recently, we showed that the functional heterogeneity of the right dorsal premotor (PMd) cortex could be better understood by dividing it into five subregions that showed different behavioral associations according to task-based activations studies. The present study investigated whether the revealed behavioral profile could be corroborated and complemented by a structural brain behavior correlation approach in two healthy adults cohorts.Grey matter volume within the five volumes of interest (VOI-GM) was computed using voxelbased morphometry. Associations between the inter-individual differences in VOI-GM and performance across a range of neuropsychological tests were assessed in the two cohorts with and without correction for demographical variables. Additional analyses were performed in random smaller subsamples drawn from each of the two cohorts.In both cohorts, correlation coefficients were low; only few were significant and a considerable number of correlations were counterintuitive in their directions (i.e., higher performance related to lower GMV). Furthermore, correlation patterns were inconsistent between the two cohorts.Subsampling revealed that correlation patterns could vary widely across small samples and that negative correlations were as likely as positive correlations.Thus, the structural brain/behavior approach did not corroborate the functional profiles of the PMd subregions inferred from activation studies, suggesting that local recruitment by fMRI studies does not necessarily imply covariance of local structure with behavioral performance in healthy adults.We discuss the limitations of such studies and related recommendations for future studies.

show abstract

Section: Significant Structural Brain Behavior Correlation and The Nementioning

confidence: 51%

Searching for behavior relating to grey matter volume in a-priori defined right dorsal premotor regions: Lessons learned

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Our study therefore speaks directly to the “replication crisis” that is currently being highlighted in psychology and neuroscience ( Forstmeier et al, 2017 , Gelman and Geurts, 2017 , Ioannidis, 2005 , Loken and Gelman, 2017 , Munafò et al, 2017 , Pashler and Wagenmakers, 2012 ). In the field of psychology, for example, a large-scale collaborative initiative reported that it could only successfully replicate less than 40% of original effects from a representative set of one hundred randomly selected studies ( Open Science Collaboration, 2015 ).…”

Section: Introductionmentioning

confidence: 70%

The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings

et al. 2018

View full text Add to dashboard Cite

This study investigated how sample size affects the reproducibility of findings from univariate voxel-based lesion-deficit analyses (e.g., voxel-based lesion-symptom mapping and voxel-based morphometry). Our effect of interest was the strength of the mapping between brain damage and speech articulation difficulties, as measured in terms of the proportion of variance explained. First, we identified a region of interest by searching on a voxel-by-voxel basis for brain areas where greater lesion load was associated with poorer speech articulation using a large sample of 360 right-handed English-speaking stroke survivors. We then randomly drew thousands of bootstrap samples from this data set that included either 30, 60, 90, 120, 180, or 360 patients. For each resample, we recorded effect size estimates and p values after conducting exactly the same lesion-deficit analysis within the previously identified region of interest and holding all procedures constant. The results show (1) how often small effect sizes in a heterogeneous population fail to be detected; (2) how effect size and its statistical significance varies with sample size; (3) how low-powered studies (due to small sample sizes) can greatly over-estimate as well as under-estimate effect sizes; and (4) how large sample sizes (N ≥ 90) can yield highly significant p values even when effect sizes are so small that they become trivial in practical terms. The implications of these findings for interpreting the results from univariate voxel-based lesion-deficit analyses are discussed.

show abstract

“…Another problem is related to statistics, where Gelman and Geurts (2017) explained three examples of findings in social psychology indicating "a replication crisis". One of them is related to a statistical error in removing data, for instance, few par-ticipants are removed from the analysis with a certain purpose, either to confirm a hypothesis or to remove outliers in order to have normally distributed data.…”

Section: Challenges Of Replication In Psychologymentioning

confidence: 99%

“…Based on the explanation of replication study issues in Indonesia and the recommendations to resolve the issues as suggested by Asendorpt et al (2013) and Gelman and Geurts (2017), it can be concluded that the essential thing to support the increase of replication studies and to strengthen the findings of replication studies in Indonesia is by increasing the understanding of research methods and basic statistical methods. Psychology lecturers in Indonesia should be open to information and new understanding of replication studies as an effort to strengthen the results of study and theoretical concepts.…”

Section: Scientific Communicationmentioning

confidence: 99%

Experiment Replication: A Proposed Solution for Developing Psychological Research in Indonesia

2019

View full text Add to dashboard Cite

Is it possible that psychology can be a strong as natural science? Having replication studies could be the answer to this question. Philosophically, a replication is ‘the heart of any science,’ however it receives a little attention from social science. In Indonesia, there are three major problems: (1) only few number of researchers implement replication studies; (2) only few replication studies present strong evidence; and (3) only a small number of replication studies have been published. This might occur because the knowledge on how to conduct a replication study is inaccessible to most psychology researchers in Indonesia. This article explains a definition of a replication study, types of replications, and strategies to conduct replication experiments. I will explain how to conduct a replication study, starting from determining and reviewing reference articles to designing a replication study.

show abstract

The statistical crisis in science: how is it relevant to clinical neuropsychology?

Cited by 34 publications

References 66 publications

Searching for behavior relating to grey matter volume in a-priori defined right dorsal premotor regions: Lessons learned

Searching for behavior relating to grey matter volume in a-priori defined right dorsal premotor regions: Lessons learned

The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings

Experiment Replication: A Proposed Solution for Developing Psychological Research in Indonesia

Contact Info

Product

Resources

About