Uncovering Substantive Patterns in Student Responses in International Large-Scale Assessments—Comparing a Latent Class to a Manifest DIF Approach

Effects of Population Heterogeneity on Accuracy of DIF Detection

“…Within group heterogeneity has also been examined with linguistically diverse students using data from large-scale assessments administered within and across countries (Ercikan et al, 2014 [this issue];Oliveri, Ercikan, Lyons-Thomas, & Holtzman, in press;Oliveri, Ercikan, Zumbo, & Lawless, 2014;Solano-Flores & Li, 2009). Solano-Flores and Li (2009) demonstrated a great degree of variation in the mathematics scores on a set of NAEP assessments that could be attributed to item by dialect by person interactions highlighting heterogeneity within the same language group due to dialect differences.…”

“…Solano-Flores and Li (2009) demonstrated a great degree of variation in the mathematics scores on a set of NAEP assessments that could be attributed to item by dialect by person interactions highlighting heterogeneity within the same language group due to dialect differences. Oliveri et al (2014) conducted a study using data from the Progress in International Reading Literacy Study (PIRLS) 2006 administered to examinees from Kuwait, Qatar, Hong Kong, and Chinese Taipei. PIRLS is an international test of reading that is administered to linguistically heterogeneous examinees.…”

Comparing DIF methods for data with dual dependency

“…When type I error can be reasonably controlled under these conditions, applied researchers might prefer using the simple model (i.e., LR) for its ease of implementation and interpretation. A number of previous studies conducting DIF analysis on large-scale assessments ignored person clustering effect (e.g., Babiar 2011;Choi et al 2015;Hauger and Sireci 2008;Innabi and Dodeen 2006;Mahoney 2008;Mesic 2012;Ockey 2007;Oliveri et al 2014;Sandilands et al 2013). Therefore, evaluating the trade-off between complex versus simple modeling of DIF may provide supporting evidence for the findings of these studies.…”

Invariance analyses in large-scale studies

“…The Bayes factor does not provide interpretable outcomes when improper priors are used, since the outcomes will depend on the unknown normalising constants. However, a fractional Bayes factor (FBF) can be computed, which normalises the outcome by using a fraction of the data (O'Hagan, 1995 [55]). The Bayes factor is computed as the probability of the data given the null hypothesis divided by the probability of the data given the alternative hypothesis:…”