Modeling Common Traits and Method Effects in Multitrait-Multimethod Analysis

Abstract

Method effects often occur when constructs are measured by different methods. In traditional multitrait-multimethod (MTMM) models method effects are regarded as residuals, which implies a mean method effect of zero and no correlation between trait and method effects. Furthermore, in some recent MTMM models, traits are modeled to be specific to a certain method. However, often we are not interested in a method-specific trait but in a trait that is common to all methods. Here we present the Method Effect model with common trait factors, which allows modeling "common" trait factors and method factors that represent method "effects" rather than residuals. The common trait factors are defined as the mean of the true-score variables of all variables measuring the same trait and the method factors are defined as differences between true-score variables and means of true-score variables. Because the model allows estimating mean method effects, correlations between method factors, and correlations between trait and method factors, new research questions may be investigated. The application of the model is demonstrated by 2 examples studying the effect of negative, as compared with positive, item wording for the measurement of mood states.

Moving beyond cognitive elements of ICT literacy: First evidence on the structure of ICT engagement

“…This gives rise to question of whether they both are actually meaningful factors or ''artifactors'' (Marsh, 1996). An alternative model keeping the notion of one self-concept factor would include one substantive (self-concept) factor across all items and one method effect factor associated with negatively worded items (Pohl & Steyer, 2010).…”

Data-Generating Mechanisms Versus Constructively Defined Latent Variables in Multitrait–Multimethod Analysis: A Comment on Castro-Schilo, Widaman, and Grimm (2013)

“…As all latent trait and method factors in this approach are functions of well-defined true score-variables, these variables as well as their associations with each other and with external variables have clear meanings and interpretations. Other CFA-MTMM approaches that also define trait and method factors constructively based on true score variables and that provide a full representation of the basic MTMM measurement model are Pohl et al’s (2008) latent difference method factor approach as well as Pohl and Steyer’s (2010) latent means model.…”

“…This idea parallels the definition of effects in analysis of variance (ANOVA) and regression analysis with code variables. For example, we distinguish between dummy coding [which uses a reference approach similar to the CT-C(M – 1) and latent difference MTMM models] and effect coding (which uses a “deviation from the grand mean” approach similar to what is done in the latent means MTMM model discussed by Pohl & Steyer, 2010). …”

“…For example, instead of using a regression approach to contrasting different TMU true score variables as in the CT-C(M – 1) model, a latent difference parameterization can also be used, in which the method factors are defined in terms of latent difference scores relative to a reference method (see Figure 2; Geiser et al, 2012; Pohl, Steyer, & Kraus, 2008; formal definitions are provided in Appendix A). Researchers interested in studying method effects as deviations from “common latent traits” rather than from “reference-method based traits” can define latent trait factors as the grand mean of the true score variables pertaining to the same trait j , leading to an effect coding scheme as discussed in Pohl and Steyer (2010; see Figure 3; formal definitions are provided in Appendix A). All these parameterizations have in common that they (1) are equivalent to the basic MTMM measurement model, (2) provide a complete decomposition of the basic MTMM true score structure, and (3) use constructive definitions of trait and method latent variables.…”

Scaling Variances of Latent Variables by Standardizing Loadings: Applications to Working Memory and the Position Effect

“…In this model the manifest variables load on two latent variables representing different sources. This model is also applied for multitrait-multimethod investigations (see Pohl & Steyer, 2010) although such applied models have not become known as bifactor models. In evaluating the results achieved by such models the question may arise as to which source exerts the stronger influence on responding and which source the weaker influence.…”