Asymptotically Correct Standardization of Person-Fit Statistics Beyond Dichotomous Items

Abstract

The [Formula: see text] statistic (Drasgow et al. in Br J Math Stat Psychol 38:67-86, 1985) is one of the most popular person-fit statistics (Armstrong et al. in Pract Assess Res Eval 12(16):1-10, 2007). Snijders (Psychometrika 66:331-342, 2001) derived the asymptotic null distribution of [Formula: see text] when the examinee ability parameter is estimated. He also suggested the [Formula: see text] statistic, which is the asymptotically correct standardized version of [Formula: see text]. However, Snijders (Psychometrika 66:331-342, 2001) only considered tests with dichotomous items. In this paper, the asymptotic null distribution of [Formula: see text] is derived for mixed-format tests (those that include both dichotomous and polytomous items). The asymptotically correct standardized version of [Formula: see text], which can be considered as the extension of [Formula: see text] to such tests, is suggested. The Type I error rate and power of the suggested statistic are examined from several simulated datasets. The suggested statistic is computed using a real dataset. The suggested statistic appears to be a satisfactory tool for assessing person fit for mixed-format tests.

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

“…The adjustment of Snijders () was also applied to the infit ( W ) and outfit ( U ) statistics (e.g., Wright & Masters, ) by Magis, Beland and Raiche () and to the standardized extended caution indices (Tatsuoka, ) by Sinharay (). Sinharay () extended the adjustment of Snijders (), which applies only to dichotomous items, to tests involving both dichotomous and polytomous items.…”

“…Magis et al. (, ), Sinharay (, ), Snijders (), and van Krimpen‐Stoop and Meijer () used the maximum likelihood estimate (MLE), the weighted likelihood estimate (WLE; Warm, ), and the modal a posteriori (MAP) estimate of the examinee ability with the adjustment of Snijders (). Magis et al.…”

Robust Maximum Marginal Likelihood (RMML) Estimation for Item Response Theory Models

“…This method is flexible in that different weighting mechanisms can be used in the same general framework. In this study the weights are derived from the normalized p-value of Sinharay's (2016a) person fit statistic ! " * , but it does not necessarily have to be so.…”

How to Compare Parametric and Nonparametric Person‐Fit Statistics Using Real Data

“…A comparison of PFSs for polytomous items using real data may be of interest. Parametric PFSs for polytomous items were suggested by researchers such as Drasgow et al (1985) and Sinharay (2016c) and nonparametric PFSs were suggested by researchers such as Emons (2008). Sixth, WLEs (Warm, 1989) were used as the ability estimates in this article.…”