No abstract

The multiple-group categorical factor analysis (FA) model and the graded response model (GRM) are commonly used to examine polytomous items for differential item functioning to detect possible measurement bias in educational testing. In this study, the multiple-group categorical factor analysis model (MC-FA) and multiple-group normal-ogive GRM models are unified under the common framework of discretization of a normal variant. We rigorously justify a set of identified parameters and determine possible identifiability constraints necessary to make the parameters just-identified and estimable in the common framework of MC-FA. By doing so, the difference between categorical FA model and normal-ogive GRM is simply the use of two different sets of identifiability constraints, rather than the seeming distinction between categorical FA and GRM. Thus, we compare the performance on DIF assessment between the categorical FA and GRM approaches through simulation studies on the MC-FA models with their corresponding particular sets of identifiability constraints. Our results show that, under the scenarios with varying degrees of DIF for examinees of different ability levels, models with the GRM type of identifiability constraints generally perform better on DIF detection with a higher testing power. General guidelines regarding the choice of just-identified parameterization are also provided for practical use.

Cognitive diagnostic computerized adaptive testing (CD-CAT) has been suggested by researchers as a diagnostic tool for assessment and evaluation. Although model-based CD-CAT is relatively well researched in the context of large-scale assessment systems, this type of system has not received the same degree of research and development in small-scale settings, such as at the course-based level, where this system would be the most useful. The main obstacle is that the statistical estimation techniques that are successfully applied within the context of a large-scale assessment require large samples to guarantee reliable calibration of the item parameters and an accurate estimation of the examinees' proficiency class membership. Such samples are simply not obtainable in course-based settings. Therefore, the nonparametric item selection (NPS) method that does not require any parameter calibration, and thus, can be used in small educational programs is proposed in the study. The proposed nonparametric CD-CAT uses the nonparametric classification (NPC) method to estimate an examinee's attribute profile and based on the examinee's item responses, the item that can best discriminate the estimated attribute profile and the other attribute profiles is then selected. The simulation results show that the NPS method outperformed the compared parametric CD-CAT algorithms and the differences were substantial when the calibration samples were small.

Thurstonian paired comparison models, identifiability, multiple judgment,

The purpose of this study was to find means to increase the power of the Cognitive Abilities Screening Instrument, Chinese version (CASI C-2.0) in the screening of dementia. In assessing the performance of the CASI in dementia screening, it is a common practice to determine a cutoff score for the total CASI score and report the respective sensitivity and specificity. In this paper, we showed that an alternative scoring system, i.e., a weighted sum of the scores from the 9 domains in the CASI C-2.0, may improve its effectiveness in screening. In particular, short-term memory and orientation appeared to be the 2 most relevant domains and their combined score was shown to be more effective than the total score in screening dementia. We also showed that such a scoring system could be kept the same across examinees with different demographic backgrounds. With the weighted scoring system of the 9 domains for patients from the targeted population, we hope to make the CASI a clinically more powerful tool in screening dementia.