The Conceptualization Implies the Statistical Model: Implications for Measuring Domains of Teaching Quality

Abstract

Classroom observation systems are widely adopted tools for directly measuring the quality of instruction. With their growing popularity, at least four different classes of analytic models are routinely used to analyze data from observation systems: factor analysis, item-response theory, latent class analysis, and formative measurement models. These four classes of models make very different assumptions about the nature of the latent construct of teaching quality. This paper surfaces and discusses these differences, focusing on how each model conceptualizes the latent teaching quality construct and links the latent teaching quality construct to observed measurement dimensions. I argue for the importance of aligning theoretical understandings of the construct of teaching quality with the assumptions contained in statistical models to facilitate valid interpretations of the results of analyses.

Investigating the Dimensions of Mind Perception

“…Additionally, one of the assumptions of factor analysis, that of conditional independence, was not significantly addressed in our study. Though we have rarely ever seen it addressed in the psychological literature in general, there are those who emphasise its importance and assert that it is an assumption which should be addressed (White et al, 2022). A discussion of conditional independence is beyond the scope of this research, though it presents a limitation of our study and offers an opportunity to critique the extensive use of factor analysis in the psychological literature in general.…”

A model and its fit lie in the eye of the beholder: Long live the sum score

“…A second question that might be posed to evaluate whether a factor analytic model is an appropriate representation of a construct is whether the items that were developed to capture the construct are replaceable with one another. In factor analysis, exchanging or using only a subset of indicators is supposed not to alter the meaning of the construct (White et al, 2022 ). If this is not the case, for example because sleeplessness and anhedonia each provide important information about depression beyond each other, than a model representing one of the outlined alternative kinds of constructs might be more appropriate than factor analysis.…”

A Model and its Fit lie in the Eye of the Beholder: Long Live the Sum Score

“…Second, if I can affirm the question "was it good to construct each item to cover a specific aspect of my construct, such that the items are clearly not replaceable with each other?" then the assumption of factor analysis that all non-shared variance between items is error might not be reasonable, because in factor analysis exchanging indicators is supposed not to alter the meaning of the construct (White et al, 2022).…”