The Crit coefficient in Mokken scale analysis: a simulation study and an application in quality-of-life research

Abstract

Purpose In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. Methods We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health. Results We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations. Discussion Even in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision.

Comparing the Automatic Item Selection Procedure and Exploratory Factor Analysis in Determining Factor Structure

“…Reference ranges have been defined for the interpretation of the crit values obtained. According to this definition, crit<40 is interpreted as suitable, 40≤crit<80 is interpreted as suspicious, and crit>80 is interpreted as seriously incompatible (Crișan, Tendeiro, & Meijer, 2021;Emons et al, 2012).…”

Advances in nonparametric item response theory for scale construction in quality-of-life research

“…When applied to real-data analysis, the previous four steps often provide detailed information on dimensionality, local independence, monotonicity, and invariant item ordering, and the researcher finds herself confronted with the question how to weigh and combine the information to draw a conclusion about the scale(s). In this special section, Crisan et al [ 31 ] critically discussed a summary measure called crit that was proposed as a heuristic tool to help researchers finding their way in the output of a Mokken Scale Analysis [ 23 ]. The four analysis steps pertain to the situation in which a test or questionnaire is administered once to a sample from the population of interest, but in repeated measurement, the issue of response shift—that is, a change in patients’ perspective on the meaning of an item—may reveal itself through a change in item ordering at the individual level.…”

Non-parametric measurement for patient-reported outcomes

“…io/ y7xud/), the paper firstly offers point estimates and standard errors for scalability coefficients for clustered data; and it secondly extends the automated item selection procedure with a formal test of whether a minimal scalability coefficient is exceeded (taking uncertainty of the point estimate into account). The second paper [11] explores the Crit coefficient, which is an index of violations of common assumptions in Mokken scale analysis. Discussed for dichotomous items, the two simulation studies and worked example show that the index performs well for certain conditions.…”