Examining the structural validity of stereotype content scales – A preregistered re-analysis of published data and discussion of possible future directions

Abstract

The stereotype content model (SCM), which defines warmth and competence as fundamental dimensions of social perception, plays a prominent role in contemporary research. Recently, researchers suggested that the SCM scales currently utilised in English contexts might perform less well than previously assumed (Friehs et al., 2021). This was particularly the case when it came to meeting prerequisites for mean-value comparisons, which are the kinds of analyses that SCM scales are mostly submitted to. We build on this research by investigating the scale properties of SCM measures in the German language context. Thus, we investigated the reliability, dimensionality and cross-target group measurement equivalence of German SCM scales in 29 published data sets (N = 10,854) using a preregistered analysis protocol. Confirmatory factor analyses of 507 SCM measurement models showed that the reliability of the used scales was on average good and that they showed adequate dimensionality in 35.10 % of all cases. We additionally assessed (partial) scalar measurement equivalence as a prerequisite for meaningful mean-value comparisons and found evidence for it in 11.44% of all cases. Our findings echo those from the English context and indicate that the currently utilised German scales perform less well than we would have hoped. Moreover, our findings contribute to a debate about how to measure stereotype content, and we call on all researchers to invest in scale development efforts to ensure highly reliable and valid social perception research in Germany and elsewhere.

Stereotype content of occupational groups in Germany

“…Previous research has found German and English SCM scales to perform poorly in confirmatory factor analyses and measurement invariance assessments (indicating that the dimensionality of warmth and competence was not given and that (latent) mean‐value comparisons were biased; Friehs et al, 2022; Kotzur et al, 2019, 2020). Consequently, we developed a new SCM scale with a specific focus on assessing the warmth and competence of occupational groups.…”

“…First, a review of SCM publications showed that the initial scale development was methodologically questionable and resulted in a wide variety of applied warmth and competence indicators and scales, often with insufficient structural validity assessments (Halkias & Diamantopoulos, 2020). Second, re‐analyses showed that confirmatory factor analyses of German and English warmth and competence scales did not show acceptable structural validity in a vast majority of cases (Friehs et al, 2022; Kotzur et al, 2019, 2020). And finally, because the statistical preconditions for meaningful (latent) mean‐value comparison (i.e., partial) scalar measurement invariance; Vandenberg & Lance, 2000) were found to be oftentimes not fulfilled (Friehs et al, 2022; Kotzur et al, 2020).…”

“…Second, re‐analyses showed that confirmatory factor analyses of German and English warmth and competence scales did not show acceptable structural validity in a vast majority of cases (Friehs et al, 2022; Kotzur et al, 2019, 2020). And finally, because the statistical preconditions for meaningful (latent) mean‐value comparison (i.e., partial) scalar measurement invariance; Vandenberg & Lance, 2000) were found to be oftentimes not fulfilled (Friehs et al, 2022; Kotzur et al, 2020). In the absence of measurement invariance, comparisons on SCM dimensions between social groups are biased because the scales’ conceptual understanding, measurement units, or points of zero might differ between groups (Boer et al, 2018).…”

“…These findings raise doubts as to whether current SCM scales really measure what they intent to measure, and whether meaningful and valid conclusions can be drawn from the findings (Flake et al, 2017). Though the SCM is not the only theoretical framework in which measurement issues have been reported (Hester et al, 2022; Hussey & Hughes, 2020), future SCM research would profit greatly from the development of well‐performing stereotype content scales, a careful examination of the scales’ dimensionality, reliability and invariance, and the application of more suitable analytical approaches in stereotype content research (Friehs et al, 2022). One recent and exemplary work complying with these demands is Halkias and Diamantopoulos (2020), who present a thoroughly developed SCM measure for the evaluation of products and countries of origin in marketing research.…”

“…We aim to advance the research on occupational stereotypes and the SCM in two important ways: First, by addressing methodological critiques directed toward the SCM scales through the development of a context‐specific warmth and competence measure, the examination of the SCM's dimensionality, and the application of advanced analytical approaches for mean value comparison that guarantee measurement invariance as a precondition for the valid interpretation of results (Asparouhov & Muthén, 2014; Friehs et al, 2022; Kotzur et al, 2019). Second, by presenting a current overview of occupational stereotypes in Germany using a large and heterogeneous sample and a parsimonious and comprehensive theoretical framework (Fiske et al, 2002; He et al, 2019).…”