Illustration of Multilevel Explanatory IRT Model DIF Testing with the Creative Thinking Scale

Abstract

Creativity has been well studied in the past several decades, and numerous measures have been developed to assess creativity. However, validity evidence associated with each measure is often mixed. In particular, the social consequence aspect of validity has received little attention. This is partly due to the difficulty of testing for differential item functioning (DIF) within the traditional classical test theory framework, which still remains the most popular approach to assessing creativity. Hence, this study provides an example of examining differential item functioning using multilevel explanatory item response theory models. The Creative Thinking Scale was tested for DIF in a sample of 1043 10th-12th graders. Results revealed significant uniform and non-uniform DIF for some items. Differentially functioning items are able to produce measurement bias and should be either deleted or modeled. The detailed implications for researchers and practitioners are discussed.

Estimating Explanatory Extensions of Dichotomous and Polytomous Rasch Models: The eirm Package in R

“…In this model, the common variability among items and persons can be explained using a set of item and person properties. There are also different variants of explanatory IRT models, such as explanatory IRT models for polytomous response data [15,16], person-by-item models with an interaction between item and person properties to examine differential item or facet functioning [17][18][19], multidimensional models [20], multilevel models [21,22], and item response tree models [23,24].…”

Academic Self-Efficacy Inventory: Using Rasch Analysis to Develop and Evaluate Self-Efficacy Rating Scales

“…Rasch model, also known as the one-parameter logistic model, is a psychometric model within the framework of item response theory (IRT) (Bond & Fox, 2015;Qian & Wang, 2020;Lipovetsky, 2021). Rasch model was conceptualized and developed by the mathematician Georg Rasch after identifying certain issues by utilizing unprocessed examination results (raw test scores) (Bond & Fox, 2015;Elliot et al, 2016).…”