A Monte Carlo Comparison of Robust MANOVA Test Statistics

Abstract

Multivariate Analysis of Variance (MANOVA) is a popular statistical tool in the social sciences, allowing for the comparison of mean vectors across groups. MANOVA rests on three primary assumptions regarding the population: (a) multivariate normality, (b) equality of group population covariance matrices and (c) independence of errors. When these assumptions are violated, MANOVA does not perform well with respect to Type I error and power. There are several alternative test statistics that can be considered including robust statistics and the use of the structural equation modeling (SEM) framework. This simulation study focused on comparing the performance of the P test statistics with fifteen other test statistics across seven manipulated factors. These statistics were evaluated across 12,076 different conditions in terms of Type I error and power. Results suggest that when assumptions were met, the standard MANOVA test functioned well. However, when assumptions were violated, it performed poorly, whereas several of the alternatives performed better. Discussion focuses on advice for selecting alternatives in practice. This study's focus on all these in one simulation and the 3 group case should be helpful to the practitioner making methodological sections.

Enfoques de aprendizaje en estudiantes universitarios de Buenos Aires

“…Se revisó, primero, la homogeneidad de covarianzas, que resultó verificada (m de Box = 49,396; p = 0,008), de acuerdo con Huberty y Petroskey (2000). Luego se examinó la homogeneidad de varianzas, la cual se verificó solo para una de ambas variables dependientes (p profundo = 0,153; p superficial < 0,001), llevando ello a utilizar para el análisis el estadístico Traza de Hotelling, considerado robusto en casos de heterocedasticidad (Finch y French, 2013). Posteriormente, se examinó el cumplimiento del supuesto de multicolinealidad entre variables dependientes (r = -0,381; p < 0,01), el cual quedó verificado (Tabachnick y Fidell, 2013).…”

Comparison of Multivariate Means across Groups with Ordinal Dependent Variables: A Monte Carlo Simulation Study

“…Simulation study results have shown clearly that the performance of Pillai's Trace, Wilk's Lambda, Hotelling-Lawley's Trace and Roy's Greatest Root can be severely compromised when there are violations in the assumption of equality of covariance matrices [2,17,20,21]. In particular, when group covariance matrices are not equivalent in the two groups case, and the smaller group has the larger variance, Type I error rates will be inflated, whereas when the larger group has the larger variance power is reduced.…”

“…In particular, when group covariance matrices are not equivalent in the two groups case, and the smaller group has the larger variance, Type I error rates will be inflated, whereas when the larger group has the larger variance power is reduced. In addition, the presence of skewed dependent variables has also been shown to be related to reductions in power for correctly identifying group differences [2,20,22]. The results of these prior studies has also indicated that no one of the standard MANOVA test statistics is clearly optimal under in all situations when the assumptions are violated, though all generally perform poorly in such situations [18,19,23].…”

“…With respect to MANOVA in the nonmultivariate normal case, nearly all of the simulation work has focused on continuous data that is skewed and/or kurtotic. In response to this type of data structure, a number of robust statistics have been suggested for use, of which several (e.g., Yao's F test, the Brown-Forsythe F test) have been shown to perform better (maintain the nominal Type I error rate and render power above 0.8) than the standard MANOVA tests when the dependent values are skewed [2,3,[24][25][26]. In contrast, very little research has been conducted to investigate how MANOVA methods work when the dependent variables are ordinal in nature.…”

“…A number of statistical methods have been designed for conducting MANOVA when one or more of these assumptions are not met. Much of the research examining these approaches has focused on the case of nonnormal continuous data [e.g., [2][3][4][5][6][7][8]. However, in some situations, the researcher may be interested in simultaneously comparing the means of multiple ordinal variables, such as responses to likert questionnaire items, across groups.…”