Multiple Imputation of Multilevel Missing Data—Rigor Versus Simplicity

Abstract

Multiple imputation is widely accepted as the method of choice to address itemnonresponse in surveys. However, research on imputation strategies for the hierarchical structures that are typically found in the data in educational contexts is still limited. While a multilevel imputation model should be preferred from a theoretical point of view if the analysis model of interest is also a multilevel model, many practitioners prefer a fixed effects imputation model with dummies for the clusters since these models are easy to set up with standard imputation software. In this article, we theoretically and empirically evaluate the impacts of this simplified approach. We illustrate that the cluster effects that are often of central interest in educational research can be biased if a fixed effects imputation model is used. We show that the potential bias depends on three quantities: the amount of missingness, the intraclass correlation, and the cluster size. We argue that the bias for the random effects can be substantial while the bias for the fixed effects will be negligible in most real-data situations. We further illustrate this with an application using data from the German National Educational Panel Survey.

Biases in multilevel analyses caused by cluster-specific fixed-effects imputation

“…We derive analytically why the variance of the random effects in the analysis model is positively biased when a cluster-specific fixedeffects imputation model, instead of a multilevel imputation model, is used. Further, we find that beyond the three factors governing this bias that were already identified in Drechsler (2015) (for the special case of random intercept models), the bias also depends on the mean and variance of the observed data (which are governed by the missing data mechanism). We present support for these findings using simulation studies and a real data application.…”

“…To our knowledge, the impact on random effects if fixedeffects models with cluster-specific slopes are used for imputation has not yet been studied analytically, despite the demand for such research (Drechsler, 2015;Lüdtke et al, 2017;Grund et al, 2016). Our paper closes this research gap by comparing cluster-specific fixed-effects imputation and multilevel imputation and generalizing the evaluations to all types of random coefficient models.…”

“…Drechsler (2015) theoretically explained the bias found in the simulations of van Buuren (2011) and illustrated that the bias depends on the cluster size, the missing data rate, and the intra-class correlation (ICC), which, in random intercepts models, is the proportion of variance between clusters relative to the total variance. Like van Buuren (2011), he only focused on random intercept models.…”

“…In educational research, it is often the random effects themselves (or derivatives, such as the intra-class correlation) that are of particular interest when measuring the school effect (Lenkeit, 2012;Nye et al, 2004;McCaffrey et al, 2004b). The impacts on random effects were addressed in later papers but the authors either only focused on random intercept models (Drechsler, 2015;Lüdtke et al, 2017;Zhou et al, 2016), or the evaluations were limited to running simulation studies without analytical derivations to identify which factors influence the bias observed in the simulation studies (van Buuren, 2011;Enders et al, 2016;Grund et al, 2016). In random intercept models, it is assumed that within a cluster, the average intercept deviates from the global intercept by a cluster-specific random value.…”

Using mHealth to Increase Treatment Utilization Among Recently Incarcerated Homeless Adults (Link2Care): Protocol for a Randomized Controlled Trial (Preprint)

“…If participants do not have transportation, bus passes will be mailed to the participants' desired location so that they can attend follow-up visits (many local shelters offer onsite mailboxes). Should high rates of missing data occur, we will employ multiple imputation methods designed for longitudinal data, [85] such as R packages mice [86] and pan. [87] Other studies comparing usual care to a smart phone intervention have observed equal rates of attrition across study arms.…”