A model and its fit lie in the eye of the beholder: Long live the sum score

Edelsbrunner, Peter A.

doi:10.3389/fpsyg.2022.986767

Cited by 10 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yb =~a * Y ' fit <-sem ( mlm , data = exampleData , cluster = " g " ) # Obtain factor scores from lavaan lavMles <-lavPredict ( fit , level = 2 , method = " Bartlett " , se = " standard " ) lavEaps <-lavPredict ( fit , level = 2 , method = " regression " , se = " standard " ) lavMles . se <-attributes ( lavMles ) $se [ [1]] lavMles <-cbind ( lavMles , rep ( lavMles . se [1] , J ) ) colnames ( lavMles ) <-c ( " MLE " , " SE_MLE " ) lavEaps .…”

Section: Discussionmentioning

confidence: 99%

“…se <-attributes ( lavMles ) $se [ [1]] lavMles <-cbind ( lavMles , rep ( lavMles . se [1] , J ) ) colnames ( lavMles ) <-c ( " MLE " , " SE_MLE " ) lavEaps . se <-attributes ( lavEaps ) $se [ [1]] lavEaps <-cbind ( lavEaps , rep ( lavEaps .…”

Section: Discussionmentioning

confidence: 99%

“…se [1] , J ) ) colnames ( lavMles ) <-c ( " MLE " , " SE_MLE " ) lavEaps . se <-attributes ( lavEaps ) $se [ [1]] lavEaps <-cbind ( lavEaps , rep ( lavEaps . se [1] , J ) ) colnames ( lavEaps ) <-c ( " EAP " , " SE_EAP " )…”

Section: Discussionmentioning

confidence: 99%

“…se <-attributes ( lavEaps ) $se [ [1]] lavEaps <-cbind ( lavEaps , rep ( lavEaps . se [1] , J ) ) colnames ( lavEaps ) <-c ( " EAP " , " SE_EAP " )…”

Section: Discussionmentioning

confidence: 99%

“…One way of obtaining individual scores on these dimensions is estimating them based on statistical models. This approach may thus be called model-based (for a discussion of this and other approaches that do not necessarily rely on statistical models, see [1][2][3]). There are different methods for obtaining factor score estimates from a model, with the two best known being the Bartlett or Maximum Likelihood (ML) methods [4] and the regression or Bayesian methods [5,6].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Cautionary Note Regarding Multilevel Factor Score Estimates from Lavaan

Zitzmann

2023

Psych

View full text Add to dashboard Cite

To compute factor score estimates, lavaan version 0.6–12 offers the function lavPredict( ) that can not only be applied in single-level modeling but also in multilevel modeling, where characteristics of higher-level units such as working environments or team leaders are often assessed by ratings of employees. Surprisingly, the function provides results that deviate from the expected ones. Specifically, whereas the function yields correct EAP estimates of higher-level factors, the ML estimates are counterintuitive and possibly incorrect. Moreover, the function does not provide the expected standard errors. I illustrate these issues using an example from organizational research where team leaders are evaluated by their employees, and I discuss these issues from a measurement perspective.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

“…se <-attributes ( lavEaps ) $se [ [1]] lavEaps <-cbind ( lavEaps , rep ( lavEaps . se [1] , J ) ) colnames ( lavEaps ) <-c ( " EAP " , " SE_EAP " )…”

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Cautionary Note Regarding Multilevel Factor Score Estimates from Lavaan

Zitzmann

2023

Psych

View full text Add to dashboard Cite

show abstract

A comparison of the multilevel MIMIC model to the multilevel regression and mixed ANOVA model for the estimation and testing of a cross‐level interaction effect: A simulation study

Kessels

Moerbeek

2023

Biometrical J

View full text Add to dashboard Cite

When observing data on a patient-reported outcome measure in, for example, clinical trials, the variables observed are often correlated and intended to measure a latent variable. In addition, such data are also often characterized by a hierarchical structure, meaning that the outcome is repeatedly measured within patients. To analyze such data, it is important to use an appropriate statistical model, such as structural equation modeling (SEM). However, researchers may rely on simpler statistical models that are applied to an aggregated data structure. For example, correlated variables are combined into one sum score that approximates a latent variable. This may have implications when, for example, the sum score consists of indicators that relate differently to the latent variable being measured. This study compares three models that can be applied to analyze such data: the multilevel multiple indicators multiple causes (ML-MIMIC) model, a univariate multilevel model, and a mixed analysis of variance (ANOVA) model. The focus is on the estimation of a cross-level interaction effect that presents the difference over time on the patient-reported outcome between two treatment groups. The ML-MIMIC model is an SEM-type model that considers the relationship between the indicators and the latent variable in a multilevel setting, whereas the univariate multilevel and mixed ANOVA model rely on sum scores to approximate the latent variable. In addition, the mixed ANOVA model uses aggregated second-level means as outcome. This study showed that the ML-MIMIC model produced unbiased cross-level interaction effect estimates when the relationships between the indicators and the latent variable being measured varied across indicators. In contrast, under similar conditions, the univariate multilevel and mixed ANOVA model underestimated the cross-level interaction effect.

show abstract

Does Interference Between Intuitive Conceptions and Scientific Concepts Produce Reliable Inter-individual Differences? A Psychometric Analysis

Edelsbrunner

2024

Sci & Educ

View full text Add to dashboard Cite

Statement-verification studies indicate the coexistence of intuitive conceptions and scientific concepts within learners. The underlying assumption is that the intuitive conceptions typically built in childhood never disappear, but are co-activated with scientific concepts when we face relevant situations. This is visible in increased reaction times and error rates when confronting statements for which intuitive conceptions and scientific concepts disagree. Studies that have tried to relate individual differences in this phenomenon to other variables such as inhibition have yielded inconclusive results. In the present study, we examine the structure of individual differences in the statement-verification paradigm by Shtulman and Valcarcel (2012). Using factor analytic model comparisons, we find that there is little stable variation in individuals’ increases of reaction times and error rates across items covering scientific concepts from different domains, as well as across topics within domains. This finding suggests that a reason for inconsistent findings is the absence of stable individual differences in statement-verification tasks. One explanation for this finding is that the answer process draws more strongly on variable cognitive resources (e.g., content knowledge) than on more stable resources (e.g., inhibition ability). The occurrence and handling of interference between intuitive conceptions and scientific concepts appear to be driven by contextual factors rather than by stable cognitive processes. We discuss alternative explanations for the lack of reliable individual differences and implications for theories and the statistical modeling of cognitive pluralism.

show abstract

A model and its fit lie in the eye of the beholder: Long live the sum score

Cited by 10 publications

References 24 publications

A Cautionary Note Regarding Multilevel Factor Score Estimates from Lavaan

A Cautionary Note Regarding Multilevel Factor Score Estimates from Lavaan

A comparison of the multilevel MIMIC model to the multilevel regression and mixed ANOVA model for the estimation and testing of a cross‐level interaction effect: A simulation study

Does Interference Between Intuitive Conceptions and Scientific Concepts Produce Reliable Inter-individual Differences? A Psychometric Analysis

Contact Info

Product

Resources

About