The Behrens–Fisher problem with covariates and baseline adjustments

Abstract

The Welch-Satterthwaite t-test is one of the most prominent and often used statistical inference method in applications. The method is, however, not flexible with respect to adjustments for baseline values or other covariates, which may impact the response variable. Existing analysis of covariance methods are typically based on the assumption of equal variances across the groups. This assumption is hard to justify in real data applications and the methods tend to not control the type-1 error rate satisfactorily under variance heteroscedasticity. In the present paper, we tackle this problem and develop unbiased variance estimators of group specific variances, and especially of the variance of the estimated adjusted treatment effect in a general analysis of covariance model. These results are used to generalize the Welch-Satterthwaite t-test to covariates adjustments. Extensive simulation studies show that the method accurately controls the nominal type-1 error rate, even for very small sample sizes, moderately skewed distributions and under variance heteroscedasticity. A real data set motivates and illustrates the application of the proposed methods.

Multivariate analysis of covariance for heterogeneous and incomplete data.

“…The results of the present study in conjunction with those from previous research (Cao et al, 2020; Zimmermann et al, 2020) indicate that when covariances are heterogeneous, the rejection rates of the common MANCOVA statistics are severely distorted. Specifically, the type I error rates are extremely conservative or extremely liberal when cell sizes and covariance matrices are positively and negatively paired, respectively.…”

“…When the normality and homogeneity assumptions are met, both univariate and multivariate tests for main and interaction effects in ANCOVA and MANCOVA models yield accurate p values; see Rheinheimer and Penfield (2001), Cao et al (2020), and the simulation results presented in this article. However, when the homogeneity assumption is not met, the effects are essentially the same as those found with analysis of variance (ANOVA) and multivariate ANOVA (MANOVA).…”

“…An alternative approach to correct the conventional standard errors is to use tests based on the heteroscedasticity-consistent covariance matrix estimator of White (1980). However, inferences based on most robust covariance estimators may behave poorly when sample sizes are small, as Long and Ervin (2000) and others (e.g., Cao et al, 2020) have demonstrated using Monte Carlo simulations. A different kind of solution is to use simulation-based methods (Ananda & Weerahandi, 1997; Friedrich & Pauly, 2018; Gamage et al, 2004; Konietschke et al, 2015; Xu, 2015; Zimmermann et al, 2020).…”

Analysis of covariance under variance heteroscedasticity in general factorial designs

“…We note that is the ”classical” variance estimator for each group‐specific submodel and therefore the result follows, see Cao et al 17 Both the estimators and of the treatment effects as well as their consistent variance‐covariance matrix estimators can now be used for the derivation of statistical procedures. This will be explained in the next section.…”

“…The variance components are model constants and we therefore prefer to estimate them on a group‐specific level using methods of moments. We follow the idea from Cao et al 17 and estimate them using the corresponding submodels. Let denote the vector of 1's and let denote the matrices of the covariates for each group separately.…”