A Model Fit Statistic for Generalized Partial Credit Model

Abstract

Investigating the fit of a parametric model is an important part of the measurement process when implementing item response theory (IRT), but research examining it is limited. A general nonparametric approach for detecting model misfit, introduced by J. Douglas and A. S. Cohen (2001), has exhibited promising results for the two-parameter logistic model and Samejima s graded response model. This study extends this approach to test the fit of generalized partial credit model (GPCM). The empirical Type I error rate and power of the proposed method are assessed for various test lengths, sample sizes, and type of assessment. Overall, the proposed fit statistic performed well under the studied conditions in that the Type I error rate was not inflated and the power was acceptable, especially for moderate to large sample sizes. A further advantage of the nonparametric approach is that it provides a convenient graphical display of possible misfit.

An Assessment of the Nonparametric Approach for Evaluating the Fit of Item Response Models

“…The minimization procedure proceeded iteratively until the χ 2 statistic stopped decreasing. For the GPCM, least squares were used on the logit deviates (Liang & Wells, ). In other words, although the relationship between θ and each step ICC was nonlinear, the relationship between the logit deviate, for each step ICC and θ was linear, allowing the use of least squares to estimate the slope and difficulty for each step ICC.…”

“…This article focused on studying an approach for assessing model fit proposed by Douglas and Cohen () that compares a nonparametrically derived ICC to the parametric ICC. This nonparametric approach, hereafter referred to as RISE (Root Integrated Squared Error), has exhibited controlled Type I error rates and adequate power in the dichotomous and polytomous case (Li & Wells, ; Liang & Wells, , ; Wells & Bolt, ). Beyond its attractive statistical properties, an additional advantage of the nonparametric approach is that it provides a convenient graphical representation of model misfit.…”

Using Bayesian Nonparametric Item Response Function Estimation to Check Parametric Model Fit

“…The true thetas of three samples (500, 2,000, and 5,000) were randomly generated from the standard normal distribution. One hundred replications were conducted for each condition to get a generalizable result (Liang & Wells, 2009; Kuhfeld, 2019).…”

“…In contrast, nonparametric methods (e.g., Ramsay, 1991) estimate the IRF without assuming any of its mathematical form and can adapt to irregularities in the data. Those nonparametric IRF estimation methods are often used to check the model fit of the parametric models by assessing the functional shape of departures from the parametric models (Junker & Sijtsma, 2001; Lee et al, 2009; Liang et al, 2014; Liang & Wells, 2009; Sijtsma & Junker, 2006; Stout, 2001; Wells & Bolt, 2008). An advantages of nonparametric model fit methods, compared with chi-square-based model fit methods (McKinley & Mills, 1985; Orlando & Thissen, 2000; Yen, 1981), are that nonparametric model fit methods can provide graphical representation of the misfit (Liang & Wells, 2009).…”

Performance of the Generalized S‐X2 Item Fit Index for Polytomous IRT Models

“…Numerous statistical procedures have been developed to evaluate item fit under an IRT model, and goodness‐of‐fit studies have been conducted and reported in the voluminous IRT literature (Bock, 1972; Douglas & Cohen, 2001; Glas & Suarez‐Falcon, 2003; Liang & Wells, 2007; McKinley & Mills, 1985; Orlando & Thissen, 2000, 2003; Sinharay, 2003, 2005; Stone, 2000; Stone & Zhang, 2003; Suarez‐Falcon & Glas, 2003; Wells, 2004; Yen, 1981). Among them, several Chi‐square‐based item‐level goodness‐of‐fit indices using significance tests such as Yen's Q 1 for dichotomous items, the traditional log‐likelihood Chi‐square, G 2 , for both dichotomous and polytomous items (McKinley & Mills), and Orlando and Thissen's S‐X 2 for dichotomous items have been used for IRT applications.…”