The Performance of the Semigeneralized Partial Credit Model for Handling Item-Level Missingness

Zhou, Sherry; Huggins-Manley, Anne Corinne

doi:10.1177/0013164420918392

Cited by 6 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, the model defined in Equation ( 12 ) can be interpreted as an IRT model for a variable

that has three categories: Category 0 (observed incorrect):

, Category 1 (observed correct):

, and Category 2 (missing item response):

(see [ 43 , 69 , 70 ]).…”

Section: Statistical Models For Handling Missing Item Responsesmentioning

confidence: 99%

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Robitzsch

2021

EJIHPE

View full text Add to dashboard Cite

Missing item responses are prevalent in educational large-scale assessment studies such as the programme for international student assessment (PISA). The current operational practice scores missing item responses as wrong, but several psychometricians have advocated for a model-based treatment based on latent ignorability assumption. In this approach, item responses and response indicators are jointly modeled conditional on a latent ability and a latent response propensity variable. Alternatively, imputation-based approaches can be used. The latent ignorability assumption is weakened in the Mislevy-Wu model that characterizes a nonignorable missingness mechanism and allows the missingness of an item to depend on the item itself. The scoring of missing item responses as wrong and the latent ignorable model are submodels of the Mislevy-Wu model. In an illustrative simulation study, it is shown that the Mislevy-Wu model provides unbiased model parameters. Moreover, the simulation replicates the finding from various simulation studies from the literature that scoring missing item responses as wrong provides biased estimates if the latent ignorability assumption holds in the data-generating model. However, if missing item responses are generated such that they can only be generated from incorrect item responses, applying an item response model that relies on latent ignorability results in biased estimates. The Mislevy-Wu model guarantees unbiased parameter estimates if the more general Mislevy-Wu model holds in the data-generating model. In addition, this article uses the PISA 2018 mathematics dataset as a case study to investigate the consequences of different missing data treatments on country means and country standard deviations. Obtained country means and country standard deviations can substantially differ for the different scaling models. In contrast to previous statements in the literature, the scoring of missing item responses as incorrect provided a better model fit than a latent ignorable model for most countries. Furthermore, the dependence of the missingness of an item from the item itself after conditioning on the latent response propensity was much more pronounced for constructed-response items than for multiple-choice items. As a consequence, scaling models that presuppose latent ignorability should be refused from two perspectives. First, the Mislevy-Wu model is preferred over the latent ignorable model for reasons of model fit. Second, in the discussion section, we argue that model fit should only play a minor role in choosing psychometric models in large-scale assessment studies because validity aspects are most relevant. Missing data treatments that countries can simply manipulate (and, hence, their students) result in unfair country comparisons.

show abstract

“…Hence, the model defined in Equation ( 12 ) can be interpreted as an IRT model for a variable

that has three categories: Category 0 (observed incorrect):

, Category 1 (observed correct):

, and Category 2 (missing item response):

(see [ 43 , 69 , 70 ]).…”

Section: Statistical Models For Handling Missing Item Responsesmentioning

confidence: 99%

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Robitzsch

2021

EJIHPE

View full text Add to dashboard Cite

show abstract

“…First and foremost, the aims of this article were not focused on the recovery of true item and person parameters, which would best be fulfilled through simulation methods, and were instead focused on demonstrating a process to use when true parameters are unknown. Large simulation studies have been conducted on multiple models for semiordered data, including the semi-PCM, for a variety of purposes well beyond that of neutral categories (Zhou & Huggins-Manley, 2018), and the results align with those in the smaller simulation study in Huggins-Manley et al (2017). Namely, the semi-PCM can generally recover true item and person parameters to a similar degree as the PCM, with the exception of slope and intercept parameters of the unordered response category.…”

Section: Resultsmentioning

confidence: 58%

“…Namely, the semi-PCM can generally recover true item and person parameters to a similar degree as the PCM, with the exception of slope and intercept parameters of the unordered response category. In comparison with the NRM, the ordered category item parameters are generally recovered better by the semi-PCM than are nominal category item parameters by the semi-PCM or NRM (Zhou & Huggins-Manley, 2018). These simulation results are to be expected as the PCM, semi-PCM, and NRM are nested models, with the PCM being the least complex and the NRM being the most complex.…”

Section: Resultsmentioning

confidence: 97%

Applying Unidimensional Models for Semiordered Data to Scale Data With Neutral Responses

Cohn

Huggins-Manley

2019

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

The purpose of this study is to evaluate whether a recently developed semiordered model can be used to explore the functioning of neutral response options in rating scale data. Huggins-Manley, Algina, and Zhou developed a class of unidimensional models for semiordered data within scale items (i.e., items with both ordered response categories and an additional nominal response category) and found promising results when applying them to scale data with Not Applicable response categories. In this study, we extended the application of the semi–partial credit model (PCM) to evaluate whether the semi-PCM can be used to calibrate potentially unordered neutral responses in rating scale data, and if so, how the approach compares with alternate methods of dealing with the neutral response option. Findings indicate that the semi-PCM can (a) assist practitioners in evaluating the ordered or unordered nature of neutral responses and (b) provide a viable alternative for θ estimation in the presence of an unordered neutral category. The process used in this study also provides a methodological framework for researchers and practitioners to use when dealing with neutral responses in their own data.

show abstract

“…Second, it can be cumbersome for Monte Carlo simulation studies to simulate step parameters or crossover parameters since a value for one parameter may affect whether a value for another parameter is realistic (e.g., leading to cross-over of CRFs that are atypical). Thus, sometimes a very small variance or limited range (e.g., Zhou & Huggins-Manley, 2020) or even fixed values (e.g., Kim & Paek, 2017) for each parameter is used, which adversely affects generalizability and makes it difficult to study how variability in CRFs may affect the performance of studied approaches.…”

Section: Nominal Generalized Partial Credit and Partial Credit Modelsmentioning

confidence: 99%

A note on the interpretation and simulation of reparameterized intercepts in constrained versions of the nominal response model

Falk¹

2021

TQMP

View full text Add to dashboard Cite

This is a brief expository paper on reparameterized intercepts under constrained variants of the nominal response model, including the generalized partial credit and partial credit models. Such parameterizations are commonly found in item response theory software packages such as flexMIRT®, IRTPRO, and OpenMx / rpf, and both these models are highly popular in educational and psychological testing. A heuristic graphical interpretation is provided. We give examples of how intercepts may be easily generated for Monte Carlo simulation studies, including a brief study to increase generalizability and explore limitations of a recently developed information matrix test to detect misspecification when collapsing adjacent response categories.

show abstract

The Performance of the Semigeneralized Partial Credit Model for Handling Item-Level Missingness

Cited by 6 publications

References 26 publications

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

On the Treatment of Missing Item Responses in Educational Large-Scale Assessment Data: An Illustrative Simulation Study and a Case Study Using PISA 2018 Mathematics Data

Applying Unidimensional Models for Semiordered Data to Scale Data With Neutral Responses

A note on the interpretation and simulation of reparameterized intercepts in constrained versions of the nominal response model

Contact Info

Product

Resources

About