The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates

Abstract

Methods of uniform differential item functioning (DIF) detection have been extensively studied in the complete data case. However, less work has been done examining the performance of these methods when missing item responses are present. Research that has been done in this regard appears to indicate that treating missing item responses as incorrect can lead to inflated Type I error rates (false detection of DIF). The current study builds on this prior research by investigating the utility of multiple imputation methods for missing item responses, in conjunction with standard DIF detection techniques. Results of the study support the use of multiple imputation for dealing with missing item responses. The article concludes with a discussion of these results for multiple imputation in conjunction with other research findings supporting its use in the context of item parameter estimation with missing data.Missing data are a common problem in many data analysis contexts, including those associated with item response data. For example, respondents may simply overlook one or more items on an instrument and neglect to provide a response. In other cases, individuals may not respond to an item for some systematic reason, such as being unsure of the answer to a test question. A number of methods Correspondence should be addressed to Holmes Finch,

Multiple Imputation of missing values in exploratory factor analysis of multidimensional scales: estimating latent trait scores

“…We suggest how the multiple imputation approach can be used in the context of EFA. It should be pointed out that while multiple imputation has been studied in unidimensional situations (see, for example, Finch, 2008Finch, , 2011, our approach is useful in multidimensional situations: to our knowledge no previous work has been done on this specific situation, which is frequent in real applied research. The key step in our procedure is to simultaneously rotate the K copies of data obtained after multiple imputation, so that the K factor scores for each individual are comparable (i.e., the average between the K factor score estimates of an individual can be computed to obtain the final factor score estimation of the individual).…”

“…When this approach is not feasible, imputation of missing data appears as an advisable alternative. Imputation of missing data in IRT has been studied in the context of unidimensional models (Ayala, Plake, & Impara, 2001;DeMars, 2003;Finch, 2008Finch, , 2011Huisman & Molenaar, 2001;Sijtsma & Van der Ark, 2003). Recently, Wolkowitz and Skorupski (2013) proposed a single imputation approach intended to estimate statistical properties of items but not factor scores.…”

Impact of simple substitution methods for missing data on Classical test theory difficulty and discrimination

“…For their part, Kamakura and Wedel (2000) developed a method to deal with MD in exploratory factor analysis, as well as Song and Lee (2002) in structural equation modeling. Other authors like Banks (2015) and Finch (2011) have discussed the case of MD on differential item functioning, and B. Zhang and Walker (2008) studied MD with person-fit statistics.…”

Investigating the effects of missing data treatments on item response theory vertical scaling

“…Some studies examined the impact of missing data on person parameter estimates (De Ayala et al, 2001;Huisman & Molenaar, 2001;Ludlow & O'leary, 1999;Mislevy & Wu, 1988). Others investigated the impact of missing responses on item parameter estimates (Finch, 2008;Oshima, 1994), differential item functioning (Finch, 2011;Rousseau, Bertrand, & Boiteau, 2004), and equating (Shin, 2009). However, little empirical research has been conducted to explore the practical impact of missing data in the context of IRT.…”

“…In addition, modeling Pimentel, 2006;Holman & Glas, 2005;Lord, 1974aLord, , 1983Mislevy & Wu, 1996;Pimentel, 2005;Rose et al, 2010) and imputation methods (Finch, 2008;Huisman & Molenaar, 2001) have been investigated. The impact of missing data have been studied mostly on proficiency and item parameter estimates, but also on DIF (Finch, 2011;Rousseau et al, 2004) and equating (Liou & Cheng, 1995;Moses et al, 2011;Shin, 2009). This section summarizes the previous research and divides it into two categories: studies on proficiency estimates and studies on item parameter estimates.…”