Modified tests for comparison of group means under heteroskedasticity and non-normality caused by outlier(s)

Abstract

There are several approximate tests proposed such as Welch's F-test (W), the Parametric Bootstrap Test (PB) and Generalized F-test (GF) for comparing several group means under heteroskedasticity. These tests are powerful and have nominal type 1 error rates but they are not performing satisfactorily under non-normality caused by outlier(s). To handle this problem, we investigate tests that are powerful and provide nominal type 1 error rates by using robust estimators both for location and scale parameters. The performance of the modified tests are examined with Monte-Carlo simulation studies. Results of simulations clearly indicate that Generalized F-test modified with Huber's M-estimators achieves the nominal type 1 error rate and provide higher power than alternative methods.

Conditions of the Central-Limit Theorem Are Rarely Satisfied in Empirical Psychological Studies

“…The study also discussed two possible modifications of a procedure commonly employed in the studies to better satisfy the conditions of the CLT (see https://osf.io/kn8mh/ for computer simulations of these modified procedures). Some commonly used parametric statistical tests, such as the t-test and the ANOVA (see also Tan and Tabatabai, 1985;Fan and Hancock, 2012;Cavus et al, 2017 for the robust-ANOVA) are robust to some extent against some types of nonnormality of the population (see Lumley et al, 2002 for a review) but not against some other types of non-normality, e.g., heavy tails and outliers (Cressie and Whitford, 1986;Wilcox, 2012). Note that there are some non-parametric statistical tests that do not use a normality assumption for the population but these non-parametric tests are not universally more robust than the parametric tests.…”

“…Experimental Psychologists should check their row data and should discuss “effects” only if the effects are clear in their data. If they want to make inferences based on the results of statistical analyses, more modern statistical methods should be considered: e.g., Robust statistics (Tan and Tabatabai, 1985 ; Huber and Ronchetti, 2009 ; Fan and Hancock, 2012 ; Wilcox, 2012 ; Cavus et al, 2017 ).…”

A revised generalized F-test for testing the equality of group means under non-normality caused by skewness

“…Yazici and Cavus (2021) discussed the performance the GF test under various conditions. Cavus et al (2017) proposed the modified GF (MGF) test by replacing the maximum likelihood estimators of the sample mean and variance with Huber (1964)'s M-estimators to overcome non-normality caused by outlier(s). They conducted a Monte-Carlo simulation study to show the performance of the MGF in terms of power and Type I error probability.…”

“…It is clearly concluded that MGF has a serious problem controlling the Type I error probability when RGF outperforms others for most of the scenarios. Cavus et al (2017) showed that the MGF is the superior test for heteroscedasticity and non-normality caused by outliers. However, it could not maintain the performance for non-normality caused by skewness in terms of Type I error probability.…”

“…They may lead to lose power and fail to control Type I error probability of the tests which are used to test the equality of group means under heteroscedasticity. Cavus et al (2017) modified the GF test to obtain a powerful test for non-normality is caused by outliers. However, it is concluded by the Monte-Carlo simulation study that MGF does not maintain its performance when non-normality caused by skewness.…”