A two-step, test-guided Mokken scale analysis, for nonclustered and clustered data

Abstract

Purpose Mokken scale analysis (MSA) is an attractive scaling procedure for ordinal data. MSA is frequently used in health-related quality of life research. Two of MSA's prime features are the scalability coefficients and the automated item selection procedure (AISP). The AISP partitions a (large) set of items into scales based on the observed item scores; the resulting scales can be used as measurement instruments. There exist two issues in MSA: First, point estimates, standard errors, and test statistics for scalability coefficients are inappropriate for clustered item scores, which are omnipresent in quality of life research data. Second, the AISP insufficiently takes sampling fluctuation of Mokken’s scalability coefficients into account. As per standard instruction, since Affiliation 2 appears to be Private address, the street details have been deleted. Please check if action taken is appropriate and amend if necessary. Methods We solved both issues by providing point estimates and standard errors for the scalability coefficients for clustered data and by implementing a Wald-based significance test in the AISP algorithm, resulting in a test-guided AISP (T-AISP), that is available for both nonclustered and clustered test scores. Results We integrated the T-AISP into a two-step, test-guided MSA for scale construction, to guide the analysis for nonclustered and clustered data. The first step is performing a T-AISP and select the final scale(s). For clustered data, within-group dependency is investigated on the final scale(s). In the second step, the strength of the scale(s) is determined and further analyses are performed. The procedure was demonstrated on clustered item scores obtained from administering a questionnaire on quality of life in schools to 639 students nested in 30 classrooms. Conclusions We developed a two-step, test-guided MSA for scale construction that takes into account sample fluctuation of all scalability coefficients and that can be applied to item scores obtained by a nonclustered or clustered sampling design.

The Crit coefficient in Mokken scale analysis: a simulation study and an application in quality-of-life research

“…From a substantive perspective this approach might not be ideal, as it causes loss of information. For researchers who wish to analyze such data using Mokken scale analysis, we refer to Koopman et al [ 4 ], who proposed point estimates, standard errors, and test statistics for scalability coefficients for nested data. These authors incorporated their proposed methods into what they called a two-step, test-guided MSA procedure for scale construction.…”

Advances in nonparametric item response theory for scale construction in quality-of-life research

“…After determining a preliminary division of items in one or more item sets, for each set the assumptions of the nonparametric IRT models are investigated. In their contribution to the special section, Koopman et al [ 19 ] discuss item selection based on scalability coefficients for clustered data common in much health research.…”

Range-Preserving Confidence Intervals and Significance Tests for Scalability Coefficients in Mokken Scale Analysis

“…One-sided significance tests (or one-sided confidence intervals) are appropriate to evaluate the two criteria of a Mokken scale (Eq. 1; Koopman et al, 2021). For the first criterion, the null hypothesis is H ij = 0 and the alternative hypothesis is H ij > 0 for each item pair (i, j ).…”

“…2), whereas one-sided significance tests are useful to test the two criteria of a Mokken scale (Eq. 1; Koopman et al, 2021). If the sampling distribution of the scalability coefficients is skewed, Wald-based confidence intervals and significance tests may be biased.…”