Assessing person fit using l&lt;SUP align="right"&gt;*&lt;/SUP&gt;&lt;SUB align="right"&gt;z and the posterior predictive model checking method for dichotomous item response theory models

Sinharay, Sandip

doi:10.1504/ijqre.2015.071730

Cited by 7 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Improvement on this Type I error inflation is a possible future research topic. For the 1% (or smaller) significance level, it is possible to use critical values based on simulations that led to Type I error rates close to the nominal level in de la Torre and Deng (), Sinharay (, ), and van Krimpen‐Stoop and Meijer (). Finally, further research along the lines of Conijn, Emons and Sijtsma (), Conrad et al .…”

Section: Discussionmentioning

confidence: 99%

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

Sinharay

2016

Brit J Math & Statis

Self Cite

View full text Add to dashboard Cite

Snijders (2001, Psychometrika, 66, 331) suggested a statistical adjustment to obtain the asymptotically correct standardized versions of a specific class of person-fit statistics. His adjustment has been used to obtain the asymptotically correct standardized versions of several person-fit statistics including the lz statistic (Drasgow et al., 1985, Br. J. Math. Stat. Psychol., 38, 67), the infit and outfit statistics (e.g., Wright & Masters, 1982, Rating scale analysis, Chicago, IL: Mesa Press), and the standardized extended caution indices (Tatsuoka, 1984, Psychometrika, 49, 95). Snijders (2001), van Krimpen-Stoop and Meijer (1999, Appl. Psychol. Meas., 23, 327), Magis et al. (2012, J. Educ. Behav. Stat., 37, 57), Magis et al. (2014, J. Appl. Meas., 15, 82), and Sinharay (2015b, Psychometrika, doi:10.1007/s11336-015-9465-x, 2016b, Corrections of standardized extended caution indices, Unpublished manuscript) have used the maximum likelihood estimate, the weighted likelihood estimate, and the posterior mode of the examinee ability with the adjustment of Snijders (2001). This paper broadens the applicability of the adjustment of Snijders (2001) by showing how other ability estimates such as the expected a posteriori estimate, the biweight estimate (Mislevy & Bock, 1982, Educ. Psychol. Meas., 42, 725), and the Huber estimate (Schuster & Yuan, 2011, J. Educ. Behav. Stat., 36, 720) can be used with the adjustment. A simulation study is performed to examine the Type I error rate and power of two asymptotically correct standardized person-fit statistics with several ability estimates. A real data illustration follows.

show abstract

Section: Discussionmentioning

confidence: 99%

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

Sinharay

2016

Brit J Math & Statis

Self Cite

View full text Add to dashboard Cite

show abstract

“…De la Torre and Deng () used the PPMC method, but with the less powerful

l_{z}

rather than with

l_{z}^{*}

. The MCMC algorithm and the PPMC method were used in PFA by Glas and Meijer () with several PFSs, Sinharay () with

l_{z}

, and Sinharay () with

l_{z}^{*}

.…”

Section: A Generalized Resampling‐based Pfa Approachmentioning

confidence: 99%

“…The remaining draws were thinned by retaining every fourth draw to lead to 1,000 draws from the posterior distribution of ability for each examinee. Further details on the MCMC algorithm can be found in Sinharay ().…”

Section: A Simulation Study With Lz*mentioning

confidence: 99%

See 1 more Smart Citation

Assessment of Person Fit Using Resampling‐Based Approaches

Sinharay

2016

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

and Deng suggested a resampling-based approach for person-fit assessment (PFA). The approach involves the use of the l * z statistic, a corrected expected a posteriori estimate of the examinee ability, and the Monte Carlo (MC) resampling method. The Type I error rate of the approach was closer to the nominal level than that of the traditional approach of using l * z along with the assumption of a standard normal null distribution. This article suggests a generalized resamplingbased approach for PFA that allows one to employ l * z or another person-fit statistic (PFS) based on item response theory, the corrected expected a posteriori estimate or another ability estimate, and the MC method or another resampling method. The suggested approach includes the approach of de la Torre and Deng as a special case. Several approaches belonging to the generalized approach perform very similarly to the approach of de la Torre and Deng's in two simulation studies and in applications to three real data sets, irrespective of the PFS used. The generalized approach promises to be useful to those interested in resampling-based PFA.

show abstract

“…The

L_{z}

statistic has been investigated in simulation studies as a person‐fit DM under the PP method in a Bayesian 3PL IRT model by de la Torre and Deng (2008). The L statistic has been investigated using a Bayesian three‐parameter normal ogive (3PNO) IRT model by Glas and Meijer (2003) and a 3PL IRT model by Sinharay (2015). In all these studies, it was seen that under the PP method, the power of L and

L_{z}

to detect model violations was relatively low.…”

Section: Introductionmentioning

confidence: 99%

A New Bayesian Person‐Fit Analysis Method Using Pivotal Discrepancy Measures

Combs

2022

J Educational Measurement

View full text Add to dashboard Cite

A common method of checking person‐fit in Bayesian item response theory (IRT) is the posterior‐predictive (PP) method. In recent years, more powerful approaches have been proposed that are based on resampling methods using the popular Lz∗$L_{z}^{*}$ statistic. There has also been proposed a new Bayesian model checking method based on pivotal discrepancy measures (PDMs). A PDM T is a discrepancy measure that is a pivotal quantity with a known reference distribution. A posterior sample of T can be generated using standard Markov chain Monte Carlo output, and a p‐value is obtained from probability bounds computed on order statistics of the sample. In this paper, we propose a general procedure to apply this PDM method to person‐fit checking in IRT models. We illustrate this using the Lz$L_{z}$ and Lz∗$L_{z}^{*}$ measures. Simulation studies are done comparing these with the PP method and one of the more recent resampling methods. The results show that the PDM method is more powerful than the PP method. Under certain conditions, it is more powerful than the resampling method, while in others, it is less. The PDM method is also applied to a real data set.

show abstract

Assessing person fit using l<SUP align="right">*</SUP><SUB align="right">z and the posterior predictive model checking method for dichotomous item response theory models

Cited by 7 publications

References 27 publications

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

Assessment of Person Fit Using Resampling‐Based Approaches

A New Bayesian Person‐Fit Analysis Method Using Pivotal Discrepancy Measures

Contact Info

Product

Resources

About