Simple imputation methods versus direct likelihood analysis for missing item scores in multilevel educational data

Abstract

Missing data, such as item responses in multilevel data, are ubiquitous in educational research settings. Researchers in the item response theory (IRT) context have shown that ignoring such missing data can create problems in the estimation of the IRT model parameters. Consequently, several imputation methods for dealing with missing item data have been proposed and shown to be effective when applied with traditional IRT models. Additionally, a nonimputation direct likelihood analysis has been shown to be an effective tool for handling missing observations in clustered data settings. This study investigates the performance of six simple imputation methods, which have been found to be useful in other IRT contexts, versus a direct likelihood analysis, in multilevel data from educational settings. Multilevel item response data were simulated on the basis of two empirical data sets, and some of the item scores were deleted, such that they were missing either completely at random or simply at random. An explanatory IRT model was used for modeling the complete, incomplete, and imputed data sets. We showed that direct likelihood analysis of the incomplete data sets produced unbiased parameter estimates that were comparable to those from a complete data analysis. Multipleimputation approaches of the two-way mean and corrected item mean substitution methods displayed varying degrees of effectiveness in imputing data that in turn could produce unbiased parameter estimates. The simple random imputation, adjusted random imputation, item means substitution, and regression imputation methods seemed to be less effective in imputing missing item scores in multilevel data settings.Keywords Item response theory . Multilevel data . Missing data . Imputation methods Multilevel data in education settings can contain complex patterns of nested sources of variability. For instance, suppose that exercise items nested in courses or chapters with varying difficulty levels are presented to students nested in schools or classes with varying ability levels. For each student, scores to the items, along with person and item properties, can be recorded. The collected data are clustered or multilevel in nature, consisting of the properties of schools, students, chapters, and items and the students' item scores (e.g., binary, pass/fail) on attempted items. These data can be modeled statistically-say, using item response theory (IRT; van der Linden & Hambleton, 1997)-to explain and understand student characteristics in relation to the item properties.De Boeck and Wilson (2004) described IRT models within the framework of generalized linear mixed models (GLMMs) or nonlinear mixed models (NLMMs), also accounting for more complex multilevel structures than the structure of measurement occasions within subjects,

Multiple imputation for ordinal longitudinal data with monotone missing data patterns

“…Rather than imputing missing measurements, Mallinckrodt et al [32] suggested the use of a direct likelihood approach to deal with incomplete correlated data under the ignorable assumption. Here, the observed cases are analysed without any analyst's adjustments, that is, without imputation nor deletion, by the use of models that provide a framework where clustered data can be analysed by including both fixed and random effects in the model (in case of GLMMs for non-Gaussian data) [25]. The authors in [25] further showed that DL analysis of incomplete datasets produced unbiased Table 3.…”

“…Here, the observed cases are analysed without any analyst's adjustments, that is, without imputation nor deletion, by the use of models that provide a framework where clustered data can be analysed by including both fixed and random effects in the model (in case of GLMMs for non-Gaussian data) [25]. The authors in [25] further showed that DL analysis of incomplete datasets produced unbiased Table 3. Standard errors (Std Err), Bias and mean squared error (MSE) estimates from fully conditional specification (FCS) and multivariate normal imputation methods (MVNI).…”

Imputation Methods to Deal With Missing Responses in Computerized Adaptive Multistage Testing

“…In PMI, the mean score of the observed responses of the person is calculated, and the missing values are replaced with values that are generated from random distributions defined by the mean score (Kadengye, Cools, Ceulemans, & Van den Noortgate, 2012). More specifically, the arithmetic mean of an examinee’s observed scores is computed as follows:…”

“…where PM p is the mean of the observed scores for person p , I rp is the total number of item responses r for person p , and Y pi is the observed response of person p for item i (Kadengye et al, 2012). To add random noise into imputation, PM p is used as a probability to make a random draw from the Bernoulli distribution (Sijtsma & van der Ark, 2003).…”

Analysis of Simple Data Imputation in Disease Dataset

“…The rule of imputation is to get the predictive value as close as possible to the missing value, in other words, the imputation tries to minimize the value between the missing value and the predicted value of the missing value. There are two types of data imputation techniques, namely simple imputation [1] and an approximation approach. Simple imputation uses general statistical values, e.g., zero values, mean, median, and random values, while the approximation approach uses prediction values based on other values in the same variable.…”