Assessment of Person Fit Using Resampling‐Based Approaches

Sinharay, Sandip

doi:10.1111/jedm.12101

Cited by 17 publications

(18 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Clearly, additional research is needed. Researchers concerned about this issue are advised to conduct parametric or non-parametric bootstrapping (see Sinharay, 2016, for similar advice in the context of item response theory person fit measurement).…”

Section: Discussionmentioning

confidence: 99%

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

Reise

Kim

Mansolf

et al. 2016

Multivariate Behavioral Research

134

195

View full text Add to dashboard Cite

Although the structure of the Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask: (1) for what percentage of individuals is a unidimensional model adequate, (2) what additional percentage of individuals can be modeled with multidimensional specifications, and (3) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS; Yuan & Bentler, 2000) to examine the structure of the RSES in a large, publicly available dataset. A distance measure, ds, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. But, based on dr values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as “unmodelable” due to their significant residuals in all models considered. Finally, analysis of ds revealed that some, but not all, of the superior fit of the bifactor model is owed to that model’s ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.

show abstract

Section: Discussionmentioning

confidence: 99%

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

Reise

Kim

Mansolf

et al. 2016

Multivariate Behavioral Research

134

195

View full text Add to dashboard Cite

show abstract

“…Specifically, it is shown how the asymptotically correct standardized PFSs can be computed with ability estimates other than the MLE, WLE, and MAP so as to have an asymptotic standard normal null distribution. Note that de la Torre and Deng (), Sinharay (), and Tendeiro and Meijer () used the EAP and a corrected version of the EAP with

l_{z}^{*}

, but did not examine whether the null distribution of the resulting

l_{z}^{*}

differs from the standard normal distribution and used simulations to obtain the null distribution.…”

Section: Introductionmentioning

confidence: 99%

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

Sinharay

2016

Brit J Math & Statis

Self Cite

View full text Add to dashboard Cite

Snijders (2001, Psychometrika, 66, 331) suggested a statistical adjustment to obtain the asymptotically correct standardized versions of a specific class of person-fit statistics. His adjustment has been used to obtain the asymptotically correct standardized versions of several person-fit statistics including the lz statistic (Drasgow et al., 1985, Br. J. Math. Stat. Psychol., 38, 67), the infit and outfit statistics (e.g., Wright & Masters, 1982, Rating scale analysis, Chicago, IL: Mesa Press), and the standardized extended caution indices (Tatsuoka, 1984, Psychometrika, 49, 95). Snijders (2001), van Krimpen-Stoop and Meijer (1999, Appl. Psychol. Meas., 23, 327), Magis et al. (2012, J. Educ. Behav. Stat., 37, 57), Magis et al. (2014, J. Appl. Meas., 15, 82), and Sinharay (2015b, Psychometrika, doi:10.1007/s11336-015-9465-x, 2016b, Corrections of standardized extended caution indices, Unpublished manuscript) have used the maximum likelihood estimate, the weighted likelihood estimate, and the posterior mode of the examinee ability with the adjustment of Snijders (2001). This paper broadens the applicability of the adjustment of Snijders (2001) by showing how other ability estimates such as the expected a posteriori estimate, the biweight estimate (Mislevy & Bock, 1982, Educ. Psychol. Meas., 42, 725), and the Huber estimate (Schuster & Yuan, 2011, J. Educ. Behav. Stat., 36, 720) can be used with the adjustment. A simulation study is performed to examine the Type I error rate and power of two asymptotically correct standardized person-fit statistics with several ability estimates. A real data illustration follows.

show abstract

“…Third, the use of the statistics with simulation‐based critical values, (or simulation‐based null distribution) is a possible area of further research. For example, Sinharay () suggested how one can perform person‐fit assessment using simulation‐based critical values of the popular

l_{z}^{*}

person‐fit statistic (Snijders, ) whose null distribution deviates slightly from its theoretical (standard normal) null distribution. A similar approach can be used with the statistics considered here.…”

Section: Discussionmentioning

confidence: 99%

A New Statistic for Detection of Aberrant Answer Changes

Sinharay

Duong

Wood

2017

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

As noted by Fremer and Olson, analysis of answer changes is often used to investigate testing irregularities because the analysis is readily performed and has proven its value in practice. Researchers such as Belov, Sinharay and Johnson, van der Linden and Jeon, van der Linden and Lewis, and Wollack, Cohen, and Eckerly have suggested several statistics for detection of aberrant answer changes. This article suggests a new statistic that is based on the likelihood ratio test. An advantage of the new statistic is that it follows the standard normal distribution under the null hypothesis of no aberrant answer changes. It is demonstrated in a detailed simulation study that the Type I error rate of the new statistic is very close to the nominal level and the power of the new statistic is satisfactory in comparison to those of several existing statistics for detecting aberrant answer changes. The new statistic and several existing statistics were shown to provide useful information for a real data set. Given the increasing interest in analysis of answer changes, the new statistic promises to be useful to measurement practitioners.

show abstract

Assessment of Person Fit Using Resampling‐Based Approaches

Cited by 17 publications

References 31 publications

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

The choice of the ability estimate with asymptotically correct standardized person‐fit statistics

A New Statistic for Detection of Aberrant Answer Changes

Contact Info

Product

Resources

About