A cautionary note on the use of the Analysis of Covariance (ANCOVA) in classification designs with and without within-subject factors

Abstract

A number of statistical textbooks recommend using an analysis of covariance (ANCOVA) to control for the effects of extraneous factors that might influence the dependent measure of interest. However, it is not generally recognized that serious problems of interpretation can arise when the design contains comparisons of participants sampled from different populations (classification designs). Designs that include a comparison of younger and older adults, or a comparison of musicians and non-musicians are examples of classification designs. In such cases, estimates of differences among groups can be contaminated by differences in the covariate population means across groups. A second problem of interpretation will arise if the experimenter fails to center the covariate measures (subtracting the mean covariate score from each covariate score) whenever the design contains within-subject factors. Unless the covariate measures on the participants are centered, estimates of within-subject factors are distorted, and significant increases in Type I error rates, and/or losses in power can occur when evaluating the effects of within-subject factors. This paper: (1) alerts potential users of ANCOVA of the need to center the covariate measures when the design contains within-subject factors, and (2) indicates how they can avoid biases when one cannot assume that the expected value of the covariate measure is the same for all of the groups in a classification design.

Building Blocks of Mathematical Learning: Virtual and Tangible Manipulatives Lead to Different Strategies in Number Composition

“…To test for conditional differences, we used an ANCOVA with the Post-Test scores as the dependent variable, the Pre-Test as the covariate, and the Group as the independent variable. ANCOVA is advocated in this type of context because it controls for minor variations in the Pre-Test scores (Oakes and Feldman, 2001;Schneider et al, 2015). The assumptions of the ANCOVA were satisfied (as noted above, the covariate levels did not differ between conditions, and homogeneity of slopes held, as verified by running an ANOVA and customizing the model to include the interaction between the covariate and independent variable, p = 0.5).…”

How age affects memory task performance in clinically normal hearing persons

“…These outcomes are confirmed by univariate analyses (ANCOVA) for the Reading Span Test (age: F(2,115) = 20.54, p < .001, partial η 2 = .26; PTT: F (1,115) = .18, p = .67, partial η 2 < .01), Digit Span forward (age: F(2,115) = 4.86, p < .01, partial η 2 = .08; PTT: F(1,115) = 1.48, p = .23, partial η 2 = .01), and Digit Span backward Test (age: F (2,115) = 5.90, p < .01, partial η 2 = .09; PTT: F(1,115) = 1.26, p = .26, partial η 2 = .01). As both the MANCOVA and the three ANCOVAs suggest that the stringent participant selection has reduced the contribution of PTT as a covariate to memory task performance, one-way ANOVAs are conducted to determine if and how age affects performance on the memory tasks without taking PTT into account (Schneider, Avivi-Reich, & Mozuraitis, 2015). These analyses reveal a significant effect of age group on Reading Span data (F(2,116) = 28.78, p < .001), and on logarithmically transformed Digit Span forward (F(2,116) = 10.83, p < .001) and backward data (F(2,116) = 17.60, p < .001).…”

The impact of age-related hearing loss and lateralized auditory attention on spatiotemporal parameters of gait during dual-tasking among community dwelling older adults

“…Concerning the effects of aging on lateralized organization of motor functions, findings depend on the action in question. For example, in upper-limb function preservation of lateralized capacities has been documented (Sebastjan, Skrzek, Ignasiak, & Slawinska, 2017). As for walking, the situation is quite different.…”

“…A C C E P T E D M A N U S C R I P T 12 covariate. As suggested by Schneider et al (Schneider, Avivi-Reich, & Mozuraitis, 2015) all values were centered before used as covariates. Also, data were scrutinized to assure compliance of all ANCOVA assumptions, which were met.…”