Comparison of Test Statistics of Nonnormal and Unbalanced Samples for Multivariate Analysis of Variance in terms of Type-I Error Rates

Abstract

In this study, we investigate how Wilks’ lambda, Pillai’s trace, Hotelling’s trace, and Roy’s largest root test statistics can be affected when the normal and homogeneous variance assumptions of the MANOVA method are violated. In other words, in these cases, the robustness of the tests is examined. For this purpose, a simulation study is conducted in different scenarios. In different variable numbers and different sample sizes, considering the group variances are homogeneous σ12=σ22=⋯=σg2 and heterogeneous (increasing) σ12<σ22<⋯<σg2, random numbers are generated from Gamma(4-4-4; 0.5), Gamma(4-9-36; 0.5), Student’s t(2), and Normal(0; 1) distributions. Furthermore, the number of observations in the groups being balanced and unbalanced is also taken into account. After 10000 repetitions, type-I error values are calculated for each test for α = 0.05. In the Gamma distribution, Pillai’s trace test statistic gives more robust results in the case of homogeneous and heterogeneous variances for 2 variables, and in the case of 3 variables, Roy’s largest root test statistic gives more robust results in balanced samples and Pillai’s trace test statistic in unbalanced samples. In Student’s t distribution, Pillai’s trace test statistic gives more robust results in the case of homogeneous variance and Wilks’ lambda test statistic in the case of heterogeneous variance. In the normal distribution, in the case of homogeneous variance for 2 variables, Roy’s largest root test statistic gives relatively more robust results and Wilks’ lambda test statistic for 3 variables. Also in the case of heterogeneous variance for 2 and 3 variables, Roy’s largest root test statistic gives robust results in the normal distribution. The test statistics used with MANOVA are affected by the violation of homogeneity of covariance matrices and normality assumptions particularly from unbalanced number of observations.

Understanding Behavioral Regulation Towards Physical Activity Participation: Do We Need a Paradigm Shift to Close the Gender Gap?

“…A two-way multivariate analysis of variance (MANOVA) was carried out to examine differences in types of motivational regulation (intrinsic, integrated, identified, introjected, external, or amotivation) across Gender (man, woman) and Age Group (12-15, 16-18, 19-21, 22-25 years). We used the Wilks' Lambda test for multivariate analysis based on the characteristics of our statistical design [37]. Further univariate analysis of variance (ANOVA) and post-hoc tests using Bonferroni correction were applied to individual dependent variables when significant main or interaction effects were found in multivariate analysis.…”

Reply to letter to the editor “effect of thrombocytopenia on development of posthemorrhagic hydrocephalus in neonates: few concerns”

“…The authors criticize our work in a few aspects, some of which I agree with but others not. They criticized not using a multivariate logistic regression which I agree with but the fact that we have an unbalanced sample size, B1 (37) vs B2 (143), would have shown biased results as the test statistics used are affected by the violation of the homogeneity and normality assumptions of the covariance matrices, in particular from the unbalanced number of observations [1]. We are aiming to continue our documentation and observations to verify our findings in further publications using multivariate analysis.…”

Nursery origin and population connectivity of swordfish Xiphias gladius in the North Pacific Ocean

“…Nursery‐specific differences in otolith core element:Ca ratios (Mg: Ca, Sr:Ca, Ba:Ca) were tested using a multivariate analysis of variance (MANOVA) model with nursery as a fixed factor. Wilks' lambda test statistic was used to test for significance as it is the most robust to deviations from assumption of homogeneity of variance and covariance (Ates et al ., 2019). Univariate tests for each of the three element:Ca ratios were performed using analysis of variance (ANOVA).…”