Examining the Structural Validity of Stereotype Content Scales – A Preregistered Re-Analysis of Published Data and Discussion of Possible Future Directions

Abstract

The Stereotype Content Model (SCM) plays a prominent role in social perception research when comparing the evaluation of different targets on warmth and competence dimensions. However, there is scarce information on the SCM's measurement properties. Thus, in this article, we provide a comprehensive test of the SCM's structural validity (i.e., reliability, dimensionality, cross-group comparability of measurement properties). We re-analysed published SCM data from English speaking participants (study 1: 78 datasets from 43 original publications, N = 20,819) and German participants (study 2: 29 datasets from 23 original publications, N = 10,854). We used confirmatory factor analyses to assess the scales' reliability and dimensionality as well as measurement invariance assessment to examine cross-group comparability as a precondition for meaningful and valid mean-value comparison. We found on average good reliabilities of the SCM scales. In contrast, about 35% of all 1093 examined SCM measurement models presented adequate scale dimensionality, and regarding the scales' crossgroup comparability, we found (partial) scalar measurement invariance in about 11% of all cases. These findings indicate considerable validity concerns in published SCM research, as a meaningful and valid measurement of warmth and competence was not given in approximately two thirds of all cases, and mean-value comparisons were potentially biased due to lacking cross-group comparability for about eight out of nine cases. We propose future directions to improve the measurement quality and validity in SCM research and invite fellow researchers to constructively discuss these ideas.

Stereotype content of occupational groups in Germany

“…We aim to advance the research on occupational stereotypes and the SCM in two important ways: First, by addressing methodological critiques directed toward the SCM scales through the development of a context-specific warmth and competence measure, the examination of the SCM's dimensionality, and the application of advanced analytical approaches for mean value comparison that guarantee measurement invariance as a precondition for the valid interpretation of results (Asparouhov & Muthén, 2014;Friehs et al, 2022;Kotzur et al, 2019).…”

“…Previous research has found German and English SCM scales to perform poorly in confirmatory factor analyses and measurement invariance assessments (indicating that the dimensionality of warmth and competence was not given and that (latent) mean-value comparisons were biased; Friehs et al, 2022;Kotzur et al, 2019Kotzur et al, , 2020. Consequently, we developed a new SCM scale with a specific focus on assessing the warmth and competence of occupational groups.…”

“…Second, re-analyses showed that confirmatory factor analyses of German and English warmth and competence scales did not show acceptable structural validity in a vast majority of cases (Friehs et al, 2022;Kotzur et al, 2019Kotzur et al, , 2020. And finally, because the statistical preconditions for meaningful (latent) meanvalue comparison (i.e., partial) scalar measurement invariance; Vandenberg & Lance, 2000) were found to be oftentimes not fulfilled (Friehs et al, 2022;Kotzur et al, 2020). In the absence of measurement invariance, comparisons on SCM dimensions between social groups are biased because the scales' conceptual understanding, measurement units, or points of zero might differ between groups (Boer et al, 2018).…”

“…First, a review of SCM publications showed that the initial scale development was methodologically questionable and resulted in a wide variety of applied warmth and competence indicators and scales, often with insufficient structural validity assessments (Halkias & Diamantopoulos, 2020). Second, re-analyses showed that confirmatory factor analyses of German and English warmth and competence scales did not show acceptable structural validity in a vast majority of cases (Friehs et al, 2022;Kotzur et al, 2019Kotzur et al, , 2020. And finally, because the statistical preconditions for meaningful (latent) meanvalue comparison (i.e., partial) scalar measurement invariance; Vandenberg & Lance, 2000) were found to be oftentimes not fulfilled (Friehs et al, 2022;Kotzur et al, 2020).…”

“…These findings raise doubts as to whether current SCM scales really measure what they intent to measure, and whether meaningful and valid conclusions can be drawn from the findings (Flake et al, 2017). Though the SCM is not the only theoretical framework in which measurement issues have been reported (Hester et al, 2022;Hussey & Hughes, 2020), future SCM research would profit greatly from the development of well-performing stereotype content scales, a careful examination of the scales' dimensionality, reliability and invariance, and the application of more suitable analytical approaches in stereotype content research (Friehs et al, 2022). One recent and exemplary work complying with these demands is Halkias and Diamantopoulos (2020), who present a thoroughly developed SCM measure for the evaluation of products and countries of origin in marketing research.…”