How to Compare Parametric and Nonparametric Person‐Fit Statistics Using Real Data

Sinharay, Sandip

doi:10.1111/jedm.12155

Cited by 9 publications

(9 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We finally considered two indicators that were based on the similarity of response patterns from different examinees (Maynes, 2017 ; Zopluoglu, 2017 ). The first indicator (PT) was formed on basis of the number of identical responses and the second indicator (PI) on basis of the number of identical incorrect responses; for earlier studies on the performance of some of these indicators in the field of educational testing, see Karabatsos ( 2003 ), Tendeiro and Meijer ( 2014 ), Kim et al ( 2017 ), Sinharay ( 2017 ), and Man et al ( 2018 ). In the following, we will briefly review the employed indicators.…”

Section: Indicators Of Cheatingmentioning

confidence: 99%

The Detection of Cheating on E-Exams in Higher Education—The Performance of Several Old and Some New Indicators

Ranger

Schmidt

Wolgast³

2020

Front. Psychol.

View full text Add to dashboard Cite

In this paper, we compare the performance of 18 indicators of cheating on e-exams in higher education. Basis of the study was a field experiment. The experimental setting was a computer assisted mock exam in an introductory course on psychology conducted at a university. The experimental manipulation consisted in inducing two forms of cheating (pre-knowledge, test collusion) in a subgroup of the examinees. As indicators of cheating, we consider well-established person-fit indices (e.g., the U3 statistic), but also several new ones based on process data (e.g., response times). The indicators were evaluated with respect to their capability to separate the subgroup of the cheaters from the remaining examinees. We additionally employed a classification tree for detecting the induced cheating behavior. With this proceeding, we aimed at investigating the detectability of cheating in the day-to-day educational setting where conditions are suboptimal (e.g., tests with low psychometric quality are used). The indicators based on the number of response revisions and the response times were capable to indicate the examinees who cheated. The classification tree achieved an accuracy of 0.95 (sensitivity: 0.42/specificity: 0.99). In the study, the number of revisions was the most important predictor of cheating. We additionally explored the performance of the indicators to predict the specific form of cheating. The specific form was identified with an accuracy of 0.93.

show abstract

Section: Indicators Of Cheatingmentioning

confidence: 99%

The Detection of Cheating on E-Exams in Higher Education—The Performance of Several Old and Some New Indicators

Ranger

Schmidt

Wolgast³

2020

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…In the context of person‐fit analysis using item scores, researchers such as Meijer and Tendeiro () and Sinharay () emphasized the importance of first assessing the overall fit of the IRT model before performing any person‐fit analysis. A similar strategy should be used for person‐fit analysis using response times.…”

Section: Conclusion and Recommendationsmentioning

confidence: 99%

A New Person‐Fit Statistic for the Lognormal Model for Response Times

Sinharay

2018

J Educational Measurement

Self Cite

View full text Add to dashboard Cite

Response‐time models are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular response‐time models, and suggests a simple person‐fit statistic for the model. The distribution of the statistic under the null hypothesis of no misfit is proved to be a χ2 distribution. A simulation study and a real data example demonstrate the usefulness of the suggested statistic.

show abstract

“…Based on G p and G n p , response patterns can be classified as severely inconsistent, suggesting that the resulting total score is probably invalid or “‘suspect.” We computed cutoff values using a bootstrap procedure based on the graded response model (i.e., an item response theory for polytomous data) with 500 replications and a one-sided α-level of .05 (e.g., Seo & Weiss, 2013; Sinharay, 2017). The latent trait values and item parameters for generating the bootstrap replications were estimated using the “mirt” R package (Chalmers, 2012), using semiparametric Davidian curves to account for a skewed distribution of the latent trait values (Woods & Lin, 2009).…”

Section: Methodsmentioning

confidence: 99%

Determining at What Age Children Provide Sound Self-Reports: An Illustration of the Validity-Index Approach

2019

View full text Add to dashboard Cite

In psychological assessment of children, it is pivotal to establish from what age on self-reports can complement or replace informant reports. We introduce a psychometric approach to estimate the minimum age for a child to produce self-report data that is of similar quality as informant data. The approach makes use of statistical validity indicators such as person-fit and long-string indices, and can be readily applied to data commonly collected in psychometric studies of child measures. We evaluate and illustrate the approach, using self-report and informant-report data of the PedsQL, a pediatric health-related quality of life measure, from 651 child–mother pairs. To evaluate the approach, we tested various hypotheses about the validity of the self-report data, using the [Formula: see text] person-fit index as the validity indicator and the mother informant-data as a benchmark for validity. Results showed that [Formula: see text] discriminated between self-reports of younger and older children, between self-reports of children that completed the PedsQL alone or with a parent, and between self-reports and informant reports. We conclude that the validity-index approach has good potential for future applications. Future research should further evaluate the approach for different types of questionnaires (e.g., personality inventories) and using different validity indices (e.g., response-bias indices).

show abstract

How to Compare Parametric and Nonparametric Person‐Fit Statistics Using Real Data

Cited by 9 publications

References 40 publications

The Detection of Cheating on E-Exams in Higher Education—The Performance of Several Old and Some New Indicators

The Detection of Cheating on E-Exams in Higher Education—The Performance of Several Old and Some New Indicators

A New Person‐Fit Statistic for the Lognormal Model for Response Times

Determining at What Age Children Provide Sound Self-Reports: An Illustration of the Validity-Index Approach

Contact Info

Product

Resources

About