Psychometric properties of sum scores and factor scores differ even when their correlation is 0.98: A response to Widaman and Revelle

The shortcomings of equal weights estimation and the composite equivalence index in PLS-SEM

“…Finally, extending on current discussions in the psychometrics literature (e.g. McNeish and Wolf, 2020;McNeish, 2023;Widaman and Revelle, 2023b), we show by means of a simulation study and an empirical example that following the recommendations implied by the CEI will generally lead to models with inferior out-ofsample predictive accuracyan important goal in PLS-SEM analyses (Danks et al, 2023).…”

“…Yet, the CEI suggested using equal weights in practically all instances in our simulation, thereby recommending a demonstratively inferior scoring method whose disadvantages have been broadly discussed in prior literature (e.g. McNeish and Wolf, 2020; Lastovicka and Thamodaran, 1991; Andersson and Yang-Wallentin, 2020; McNeish, 2023). In sum, adoption of the CEI and equal weights would decrease the methodological rigor of research.…”

“…Hence, assuming equal weights estimation as a quasi-defaultas practically implied by the CEIreintroduces various problems that have long been overcome through the implementation of estimators that do not impose artificial constraints on the model. More importantly, McNeish (2023) shows that analyses based on equal versus differentiated weights can cause large variability in conclusions, even when the corresponding scores are very highly correlatedhigher than the suggested CEI threshold of 0.95. Specifically, differentiated weights produce scores that have higher correlations with true scores, higher sensitivity and higher reliability even at correlations as high as 0.98.…”

“…Not surprisingly, researchers have noted that sumscores are too imprecise to be used in rigorous empirical research in most conditionsand that their use EJM 58,13 should be restricted to very specific conditions (e.g. McNeish and Wolf, 2020;McNeish, 2023).…”

Summed versus estimated factor scores: Considering uncertainties when using observed scores.

“…Here, we use the term latent score to refer collectively to CTT true scores and LVs 1 . Recent discussions have focused on the nature of summed scores (SS) versus estimated factor scores (e.g., McNeish, 2022; McNeish & Wolf, 2020; Widaman & Revelle, 2022, 2023). We extend this discussion by considering the use of observed scores by substantive researchers in terms of scoring and classifying individuals as well as estimating relations among LVs (see also DiStefano et al, 2009; Grice & Harris, 1998; Horn, 1965; Tucker, 1971).…”