"Determining When Subscale Scores from Assessment Measures Provide Added Value"

Vispoel, Walter P

doi:10.26717/bjstr.2023.53.008457

Cited by 1 publication

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Except for the Sociability subscale within the p • × i • CON design, added value was supported for all subscales within both the original and restricted S-ETE and CON designs. When a subscale fails to reach the threshold to support added value, prophecy formulas can be used to determine the number of items and/or occasions that might be needed to support added value as further discussed in the next section (see, e.g., [16,17,70,93]).…”

Section: Subscale Added Valuementioning

confidence: 99%

Multivariate Structural Equation Modeling Techniques for Estimating Reliability, Measurement Error, and Subscale Viability When Using Both Composite and Subscale Scores in Practice

Vispoel,

Lee,

Chen

2024

Mathematics

View full text Add to dashboard Cite

We illustrate how structural equation models (SEMs) can be used to assess the reliability and generalizability of composite and subscale scores, proportions of multiple sources of measurement error, and subscale added value within multivariate designs using data from a popular inventory measuring hierarchically structured personality traits. We compare these techniques between standard SEMs representing congeneric relations between indicators and underlying factors versus SEM-based generalizability theory (GT) designs with simplified essential tau-equivalent constraints. Results strongly emphasized the importance of accounting for multiple sources of measurement error in both contexts and revealed that, in most but not all instances, congeneric designs yielded higher score accuracy, lower proportions of measurement error, greater average subscale score viability, stronger model fits, and differing magnitudes of disattenuated subscale intercorrelations. Extending the congeneric analyses to the item level further highlighted consistent weaknesses in the psychometric properties of negatively versus positively keyed items. Collectively, these findings demonstrate the practical value and advantages of applying GT-based principles to congeneric SEMs that are much more commonly encountered in the research literature and more directly linked to the specific measures being analyzed. We also provide prophecy formulas to estimate reliability and generalizability coefficients, proportions of individual sources of measurement error, and subscale added-value indices for changes made to measurement procedures and offer guidelines and examples for running all illustrated analyses using the lavaan (Version 0.6-17) and semTools (Version 0.5-6) packages in R. The methods described for the analyzed designs are applicable to any objectively or subjectively scored assessments for which both composite and subcomponent scores are reported.

show abstract

Section: Subscale Added Valuementioning

confidence: 99%