Item Response Theory Models for Ipsative Tests With Multidimensional Pairwise Comparison Items

Abstract

There is re-emerging interest in adopting forced-choice items to address the issue of response bias in Likert-type items for noncognitive latent traits. Multidimensional pairwise comparison (MPC) items are commonly used forced-choice items. However, few studies have been aimed at developing item response theory models for MPC items owing to the challenges associated with ipsativity. Acknowledging that the absolute scales of latent traits are not identifiable in ipsative tests, this study developed a Rasch ipsative model for MPC items that has desirable measurement properties, yields a single utility value for each statement, and allows for comparing psychological differentiation between and within individuals. The simulation results showed a good parameter recovery for the new model with existing computer programs. This article provides an empirical example of an ipsative test on work style and behaviors.

Item Selection and Exposure Control Methods for Computerized Adaptive Testing with Multidimensional Ranking Items

“…By following the BTL framework, W.‐C. Wang and colleagues (Wang, Qiu, Chen, & Ro, ) proposed the Rasch ipsative model (RIM), which has specific objectivity (Rasch, ). With specific objectivity, RIM allows comparisons between a person's profile on the same scale instead of a person's scores , so we use the RIM framework for the measurement model in CAT.…”

“…(By contrast, the Thurstonian model for multidimensional ranking items [Brown, ; Brown & Maydeu‐Olivares, ] has neither test‐free specific objectivity [W.‐C. Wang et al., ] nor a unique utility parameter for each statement. Therefore, it is not suitable for ranking items in CAT.)…”

“…The probability of each pairwise comparison (e.g., person n preferring S 1 to S 2 P ni [ S 1 > S 2 ]) is defined in RIM (W.‐C. Wang et al., ) as follows: …”

Obituary

“…Wang's several major contributions towards the goal of extending the general Rasch model to many specific models deserve at least brief mention here. First, he developed new models and advanced existing theories and frameworks (e.g., Wang & Wilson, ); for example, he developed new classes of IRT models for rater errors (Wang & Wilson, ; Wang, Wilson, & Shih, ), ipsative tests with forced‐choice items (Wang, Qiu, Chen, Ro, & Jin, ), and examinee‐selected items to ensure test fairness (Wang, Jin, Qiu, & Wang, ); he advanced cognitive diagnosis models (CDMs) by developing, evaluating, and demonstrating novel models and methods (Wang & Qiu, in press). Second, he built, validated, and adapted various frameworks and scales, such as the Marital Needs Scale, the Principal Instructional Management Rating Scale (PIMRS), the Student Research Experience Questionnaire (SREQ), and the Stroke Rehabilitation Assessment of Movement (STREAM) scale for stroke patients.…”

“…He guided each of his PhD students to focus on one idiosyncratic characteristic at a time, identify the issue, and then solve the problem. This dedication yielded impactful new knowledge, including models for randomness in subjective judgements for rating scale items (Wang & Wu, ; Wang et al ., ), testlet items (Huang & Wang, ), various rater errors (Hung & Wang, ; Wang & Liu, ; Wang, Su, & Qiu, ), multilevel data structures (Wang & Qiu, ), higher‐order latent traits (Huang & Wang, ; Huang, Wang, Chen, & Su, ), CDMs (Li & Wang, ), unfolding models for Likert‐type items (Liu & Wang, ; Wang & Wu, ), response styles (Chen, Jin, & Wang, ; Jin & Wang, ; Liu & Wang, in press), ipsative items (Wang et al ., ), examinee‐selected items (Liu & Wang, ), and test‐takers with inattentive response behaviour (Jin, Chen, & Wang, ).…”