Taking Parametric Assumptions Seriously Arguments for the Use of Welch’s F-test instead of the Classical F-test in One-way ANOVA (in press for the International Review of Social Psychology)

Abstract

Student's t-test and classical F-test ANOVA rely on the assumptions that two or more samples are independent, and that independent and identically distributed residuals are normal and have equal variances between groups. We focus on the assumptions of normality and equality of variances, and argue that these assumptions are often unrealistic in the field of psychology. We underline the current lack of attention to these assumptions through an analysis of researchers' practices. Through Monte Carlo simulations we illustrate the consequences of performing the classic parametric F-test for ANOVA when the test assumptions are not met on the Type I error rate and statistical power. Under realistic deviations from the assumption of equal variances the classic F-test can yield severely biased results and lead to invalid statistical inferences. We examine two common alternatives to the F-test, namely the Welch's ANOVA (W-test) and the Brown-Forsythe test (F*-test). Our simulations show that under a range of realistic scenariosthe W-test is a better alternative and we therefore recommend using the W-test by default when comparing means. We provide a detailed example explaining how to perform the W-test in SPSS and R. We summarize our conclusions in practical recommendations that researchers can use to improve their statistical practices.

Exploring the Human Side of Meteorology: A Brief Report on the Psychology of Meteorologists

“…Welch's ANOVA was chosen following recent recommendations (Delacre et al 2018) that the test be used in place of the regular F-test-based oneway ANOVA. For these, we calculated effect size (eta-squared, ηp 2 ), and statistical power.…”

In This Together? Exploring Moderating and Mediating Effects of Shared Racial Fate on the Predictive Role of Racial Socialization and Discrimination on Black Activism

“…Levene’s test was used to examine the assumption of homogeneity of variance for comparative analyses (i.e., t- tests and ANOVA) and as expected, there was evidence of unequal variances. Given this, we used Welch tests for comparative analyses as default given their robustness against violations to the assumption of homogeneity of variances (Delacre et al, 2017, 2019). The assumption of homoscedasticity for regression analyses was assessed by examining the residual plots, which indicated no clear violations of homoscedasticity.…”

“…Educational Attainment: Welch’s one-way ANOVA, which is robust against variance heterogeneity and unequal sample sizes, was used to compare study variables on participants’ education attainment (Delacre et al, 2019). Composite groups were created to optimize the number of cases across cells for the comparative analysis.…”

Comparison of early career researchers and senior career researchers as peer reviewers: A questionnaire survey in China

“…First, the demographic data were described to analyse the samples on the whole. Then, using SPSS Version 26, after preliminary checking of the normality of the data and its homogeneity of variance (Delacre et al, 2019), Welch's test for analysis of variance (Welch's ANOVA) was conducted to investigate the heterogeneity between ECRs and SCRs.…”