Conducting Measurement Invariance Tests with Ordinal Data: A Guide for Social Work Researchers

Abstract

The validity of measures across groups is a major concern for social work researchers and practitioners. Many social workers use scales, or sets of questionnaire items, with ordinal response options. However, a review of social work literature indicates the appropriate treatment of ordinal data in measurement invariance tests is rare; only 3 of 57 articles published in 26 social work journals over the past 12 years used proper testing procedures. This article synthesizes information from the literature and provides recommendations for appropriate measurement invariance procedures with ordinal data. Method: We use data from the Cebu Longitudinal Health and Nutrition Survey to demonstrate applications of invariance testing with ordinal data. Using a robust weighted least squares estimator and polychoric correlation matrix, we examine invariance of a 10-item Perceived Stress Scale (PSS) across 2 young adult groups defined by health status. We describe 2 competing approaches: a 4-step approach, in which factor loadings and thresholds are tested and constrained separately; and a 3-step approach, in which loadings and thresholds are tested and constrained in tandem. Results: Both approaches lead to the same conclusion that the 2 dimensions of the PSS are noninvariant across health status. In the absence of invariance, mean scores on the PSS factors cannot be validly compared across groups, nor should latent variables be used in the hypothesis testing across the 2 groups. Readers are directed to online resources. Conclusions: Careful examination of social work scales is likely to reveal fit or noninvariance problems across some groups. Use of appropriate methods for invariance testing will reduce misuse of measures in practice and improve the rigor and quality of social work publications.

Psychometric properties and measurement invariance of the Beck hopelessness scale (BHS): results from a German representative population sample

“…(Iliceto, Fino, Sabatello, and Candilera [45] established measurement invariance regarding age in a larger model including the BHS, using a Likert scale and Iliceto and Fino tested for general model invariance in two random subsamples.) Measurement bias can lead to erroneous application and interpretation of cut-off scores, denying individuals in distress proper treatment [46]. Empirical findings could furthermore erroneously be generalized across groups.…”

A Measure of Comprehensive Airman Fitness: Construct Validation and Invariance Across Air Force Service Components

“…If full invariance could not be established at a particular step, an assessment of partial invariance was conducted (N. Bowen & Masa, 2015;Byrne, Shavelson, & Muthen, 1989). Dimitrov (2010) described partial invariance as "a situation in which there is no perfect invariance for specific parameters, but neither is there evidence of their complete inequality" (p.127).…”

Academic Self‐efficacy Among Junior High School Students in Ghana: Evaluating Factor Structure and Measurement Invariance Across Gender

“…The factor loadings of .38 to .69 were above the ≥ .30 cutoff, similar to those of the boys‐only model (see Table ). Following the adequacy of the one‐factor model for the two gender groups, we proceeded with the invariance testing (Bowen & Masa, ).…”

“…The invariance tests assessed whether the parameters of the academic self‐efficacy measurement model were statistically identical for boys and girls at baseline and follow‐up. First, we examined separate models for boys and girls to confirm the adequacy of the model for each gender (Bowen & Masa, ). Next, we conducted gender invariance tests by estimating a succession of three nested models, starting with the least constrained to the most constrained model: configural, metric, and scalar models (Millsap & Olivera‐Aguilar, ; Vandenberg & Lance, ).…”