Effects of Covariance Heterogeneity on Three Procedures for Analyzing Multivariate Repeated Measures Designs

Abstract

Empirical Type I error and power rates were estimated for (a) the doubly multivariate model, (b) the Welch-James multivariate solution developed by Keselman, Carriere and Lix (1993) using Johansen's results (1980), and for (c) the multivariate version of the modified Brown-Forsythe (1974) procedure. The performance of these procedures was investigated by testing within-blocks sources of variation in a multivariate split-plot design containing unequal covariance matrices. The results indicate that the doubly multivariate model did not provide effective Type I error control while the Welch-James procedure provided robust and powerful tests of the within-subjects main effect, however, this approach provided liberal tests of the interaction effect. The results also indicate that the modified Brown-Forsythe procedure provided robust tests of within-subjects main and interaction effects, especially when the design was balanced or when group sizes and covariance matrices were positively paired.

Parametric and nonparametric bootstrap methods for general MANOVA

“…Vallejo et al [35] have compared different multivariate procedures with respect to covariance heterogeneity, under the assumption of multivariate normality. In an extensive simulation study involving heterogeneous covariance matrices and nonnormal distributions, Vallejo and Ato [33] investigated the behavior of different small-sample approximations for multivariate tests.…”

Analysis of Type I Error Rates of Univariate and Multivariate Procedures in Repeated Measures Designs

“…However, when neither the sphericity nor the homogeneity assumption is met (because the data deviate from normality, and the design is unbalanced), the specialized literature does not recommend the use of these alternative procedures because they do not adequately control Type I error rates (Vallejo and Lozano, 2006;Vallejo et al, 2006a). To tackle these problems, new methods of data analysis such as the Improved General Approximate approach (IGA) developed by Huynh (1978) and modified by Algina and Oshima (1995), the Welch-James procedure (WJ) developed by Johansen (1980) and implemented in the field of repeated measures designs by Keselman et al (1993), the multivariate version of the Brown-Forsythe procedure (BF; originally developed in 1974) proposed by Vallejo et al (2001), the bootstrap technique (BT) developed by Efron and Tibshiriani (1993), and the linear mixed model (LMM) have been developed.…”

A Comparison of Procedures for the Analysis of Multivariate Repeated Measurements

“…The number of repeated measurements was set at T = 4 and 6, while the number of dependent variables was set at P = 2, 3, 4, and 6. These conditions reflect the range of simulation parameters that have been investigated in previous research on methods for the analysis of multivariate repeated measures data (Boik, 1991;Lix et al, 2003;Vallejo et al, 2001).…”

“…Vallejo, Fidalgo, and Fernandez (2001) extended the Brown and Forsythe (1974) procedure to the case of multivariate repeated measurements, but a recent modification proposed by Vallejo and Ato (2006), to address the conservative Type I error properties of this procedure for testing within-subjects effects, was adopted in the current study.…”

“…Previous research on methods for the analysis of multivariate repeated measures data when covariances are heterogeneous has focused on the properties of DMM tests of multivariate within-subjects main and interaction effects and robust alternatives to DMM tests. Robust alternatives include Johansen's (1980) Lix, 1997;Lix, Algina, & Keselman, 2003;Vallejo, Fidalgo, & Fernandez, 2001). These ADF tests have been implemented with least-squares estimators when the data follow a multivariate normal distribution, as well as with trimmed estimators (i.e., trimmed means and Winsorized covariances) for the case when the data follow a multivariate heavy-tailed or skewed distribution.…”