The Logic of Latent Variable Analysis as Validity Evidence in Psychological Measurement

Abstract

Background:Validity is the most important characteristic of tests and social science researchers have a general consensus of opinion that the trustworthiness of any substantive research depends on the validity of the instruments employed to gather the data.Objective:It is a common practice among psychologists and educationalists to provide validity evidence for their instruments by fitting a latent trait model such as exploratory and confirmatory factor analysis or the Rasch model. However, there has been little discussion on the rationale behind model fitting and its use as validity evidence. The purpose of this paper is to answer the question: why the fit of data to a latent trait model counts as validity evidence for a test?Method:To answer this question latent trait theory and validity concept as delineated by Borsboom and his colleagues in a number of publications between 2003 to 2013 is reviewed.Results:Validating psychological tests employing latent trait models rests on the assumption of conditional independence. If this assumption holds it means that there is a ‘common cause’ underlying the co-variation among the test items, which hopefully is our intended construct.Conclusion:Providing validity evidence by fitting latent trait models is logistically easy and straightforward. However, it is of paramount importance that researchers appreciate what they do and imply about their measures when they demonstrate that their data fit a model. This helps them to avoid unforeseen pitfalls and draw logical conclusions.

Analysis of the Ruff 2 & 7 Test of Attention with the Rasch Poisson Counts Model

“…The fit of data to a latent trait model, such as the Rasch model, is evidence that the covariation among the test items is caused by an underlying latent factor which could be the intended construct and is, therefore, considered a validity evidence [19,20].…”

Continuing Professional Development Strategies: A Model for the Iranian EFL Teachers’ Success

“…It is largely used to develop models in exploratory research (Ravand & Baghaei, 2016;Rönkkö & Evermann, 2013). The first phase in assessing SEM is validating the model by analyzing the measurement models (Baghaei & Tabatabaee Yazdi, 2016). To this aim, the loadings of the latent variables' items, the composite reliability score, the average variance extracted (AVE) and the discriminant validity should be measured (Ringle, Sarstedt, & Schlittgen, 2010).…”

Face Validity In ESP: A Quantitative Student Validation of an Engineering ESP Approach

“…As a course design mechanism, ESP belongs to the class of objects called educational programmes or designs, more specifically, language education programmes. As an educational programme or design, ESP, as do other educational designs, becomes subject to educational theory and practice (the curriculum) regardless of whether validations of these are through nomological networks, the theory-constituting systems of scientific laws proposed by Cronbach and Meel (1955); by the score-based interpretations, arguments and inferences proposed by Messick (1989), and supported by Kane (2001), or by the realist measurementcausal attributions approach of Borsboom, Mellenbergh, and van Heerden (2004, as critically discussed in Baghaei &Yazdi, 2016: 169-170, andin Colliver, Conlee, &Verhulst, 2012: 367-369). ESP initially sought to set itself apart from language education theory and practice as a special kind of ELT with special methodologies, but the reality of the practical constraints imposed by non-observance of curriculum principles often inhibited the effectiveness of this language teaching approach, as came to the fore in the shortcomings identified in Munby's needs analysis model of 1978.…”