An Updated Guide to Robust Statistical Methods in Neuroscience

Abstract

There is a vast array of new and improved methods for comparing groups and studying associations that offer the potential for substantially increasing power, providing improved control over the probability of false positives, and yielding a deeper and more nuanced understanding of data. These new techniques effectively deal with four insights into when and why conventional methods can be unsatisfactory. But for the non-statistician, this vast array of techniques for comparing groups and studying associations can seem daunting. This article briefly reviews when and why conventional methods can have relatively low power and yield misleading results. The main goal is to suggest guidelines regarding the use of modern techniques that improve upon classic approaches such as Pearson's correlation, ordinary linear regression, ANOVA, and ANCOVA. This updated version includes recent advances dealing with effect sizes, including situations where there is a covariate. The R code, figures, and accompanying notebooks have been updated as well.

Dorsal brain activity reflects the severity of menopausal symptoms

“…33 Therefore, we used a nonparametric approach with Monte Carlo resampling, such as bootstrapping 34 and permutation approaches, 35 which are standard procedures in the neuroimaging data analysis. 36,37 First, to examine the relationship among three clinical parameters (KKSI [ie, level of unpleasantness], and FSH and LH levels), correlation analysis was performed using a nonparametric percentile bootstrapping approach. 34,38,39 For each pair of parameters, the Pearson coefficient was calculated by resampling with replacement data across all participants for 5,000 times.…”

“…Data with a small number of samples are usually unstable, potentially skewed, and have nonnormal distributions, necessitating assessments using nonparametric statistics 33 . Therefore, we used a nonparametric approach with Monte Carlo resampling, such as bootstrapping 34 and permutation approaches, 35 which are standard procedures in the neuroimaging data analysis 36,37 . First, to examine the relationship among three clinical parameters (KKSI [ie, level of unpleasantness], and FSH and LH levels), correlation analysis was performed using a nonparametric percentile bootstrapping approach 34,38,39 .…”

Corticospinal excitability changes during muscle relaxation and contraction in motor imagery

“…In each experiment, multiple comparisons were performed using a Bias correction and accelerated (BCA) bootstrapped paired t-test with Holm's adjustment. The BCA bootstrapped paired t-tests were performed using 5000 bootstrap samples, as recommended by Wilcox and Rousselet (Wilcox & Rousselet, 2023), rather than using traditional t-tests. Additionally, we reported the number of participants who exhibited larger MEP amplitude than the control trial (or the number of participants who demonstrated larger MEP amplitude in control at contraction MI compared with the control at relaxation MI in Experiment 2).…”

An introduction to the bootstrap: a versatile method to make inferences by using data-driven simulations.

“…Also, the shorter CIs obtained with the percentile bootstrap compared to the bootstrap-t CIs, for similar coverage, suggest it could be beneficial to use the percentile bootstrap in conjunction with trimmed means. More generally, inferences about trimmed means are robust to outliers, which can have devastating effects on the power of methods using the mean (Wilcox, 2022;Wilcox & Rousselet, 2023). Finally, based on simulations, it seems that to make inferences about the mean or when trimming less than 20%, the bootstrap-t technique performs well in many situations and should be preferred over the percentile bootstrap; when trimming at least 20%, the standard bootstrap approach is recommended (Wilcox, 2022).…”