Interrelationships between latent state-trait theory and generalizability theory within a structural equation modeling framework.

Abstract

Over recent years, latent state-trait theory (LST) and generalizability theory (GT) have been applied to a wide variety of situations in numerous disciplines to enhance understanding of the reliability and validity of assessment data. Both methodologies involve partitioning of observed score variation into systematic and measurement error components. LST theory is focused on separating state, trait, error, and sometimes method effects, whereas generalizability theory is concerned with distinguishing universe score effects from multiple sources of measurement error. Despite these fundamental differences in focus, LST and GT share much in common. In this article, we use data from a widely used personality measure to illustrate similarities and differences between these two frameworks and show how the same data can be readily interpreted from both perspectives. We also provide comprehensive instructional online supplemental materials to demonstrate how to analyze data using the R package for all LST models and GT designs discussed.

Expanding bifactor models of psychological traits to account for multiple sources of measurement error.

“…Better fits for bifactor over single-construct models here highlighted the value of stratifying Negative Emotionality domain scores by facet representation. Further clarification of domain representativeness and gains in generalizability of scores from self-report measures might be achieved by taking directionality of item phrasing and other sources of item specificity into account (see, e.g., the present ; Vispoel, Hong, & Lee, 2022; Vispoel, Xu, & Schneider, 2021).…”

“…Uniquenesses for items within a facet can differ in the ETE model (middle diagram in Figure 1) but are set equal in the simplified ETE (S-ETE) model (bottom diagram in Figure 1). 2 S-ETE models represent the approach typically used to analyze G-theory designs within SEM frameworks in which observed score variance is partitioned solely at aggregate score levels (see, e.g., Jorgensen, 2021; Marcoulides, 1996; Raykov & Marcoulides, 2006; Vispoel et al, 2018a, 2018b, 2019; Vispoel, Hong, & Lee, 2022; Vispoel, Hong, et al, 2022; Vispoel, Xu, & Kilinc, 2021; Vispoel, Xu, & Schneider, 2021). As is the case with use of ANOVA in G-theory designs, SEMs simply provide a means for estimating variance components rather than serving as a formal basis for model testing.…”

“…The ETE designs discussed here extend S-ETE designs to allow for partitioning of general, group, and error variances at item in addition to aggregate score levels. With the corresponding single-construct models shown in Figure 2, both item loadings and uniquenesses can vary with CON relationships, only uniquenesses can vary with ETE relationships, and neither loadings nor uniquenesses can vary with S-ETE relationships (see Vispoel, Xu, & Kilinc, 2021 and Vispoel, Xu, & Schneider, 2021, for additional examples of these models)…”

“…The major shortcoming of one-occasion models is that they do not properly account for and separate the primary sources of measurement error affecting scores. For objectively scored measures of psychological traits, in which all scorers would get the same results, three primary sources of interperson measurement error are relevant: transient error (between-occasion variation), specific-factor error (enduring item-specific effects unrelated to the targeted constructs), and random-response error (within-occasion “noise”; Becker, 2000; Le et al, 2009; Reeve et al, 2005; Schmidt et al, 2003; Thorndike, 1951; Vispoel et al, 2018a, 2018b, 2018c, 2018d, 2019; Vispoel & Tao, 2013; Vispoel, Xu, & Kilinc, 2021; Vispoel, Xu, & Schneider, 2021, 2022). When single-construct or bifactor models rely on data from only one occasion, transient error is treated as systematic variance (i.e., overall trait variance in the single-construct model and general and/or group trait variance in the bifactor model) that would typically inflate estimates of reliability derived from those models.…”

“…The latter two models differ in that uniquenesses for individual items can vary in the ETE model but are constrained to be equal within facets in the S-ETE model. Allowing measurement errors to differ within the ETE model again would yield more precise partitioning of general, group, and error effects at the individual item level (see, e.g., Vispoel, Xu, & Kilinc, 2021; Vispoel, Xu, & Schneider, 2021). Paralleling one-occasion models, the two-occasion bifactor models shown in Figure 3 differ with the two-occasion single-construct models shown in Figure 4 in that they separate overall trait variance into two sources (general and group) and allow for such partitioning at both composite and subscale levels…”