Evaluating Test Validity

Validity in quantitative content analysis

“…A rigorous and systematic process of construction yields a set of indicators that are reasonable (Sheppard, 1993). However, the appropriateness of inferences made from the protocol remains hypothetical until it is demonstrated empirically.…”

“…The first section begins with the observation that QCA is a form of testing and measurement but notes that the procedures of test development codified in the psychometric literature are given meagre consideration in QCA research. The second section describes the process of constructing a coding protocol that is theoretically valid, or as Sheppard (1993) says, reasonable. This section draws on Crocker and Algina's (1986) presentation of essential steps in test construction.…”

Construct definition and validity inquiry in SLA research

“…The collective possibilities for construct validation may appear overwhelming to researchers wishing to use tests rather than endlessly investigating their construct validity. As Shepard (1993) expresses the problem, "If construct validity is seen as an exhaustive process that can be accomplished over a 50-year period, test developers may be inclined to think that any validity information is good enough in the short run" (p. 444 ). The alternative Shepard suggests is to focus on those questions essential to the consequences of testing.…”

Research on Validity Theory and Practice at ETS

“…Since the early 1990s, researchers have developed several related approaches to validation (Kane 1992(Kane , 2006(Kane , 2013aMislevy 2006Mislevy , 2009Mislevy et al 1999;Mislevy et al 2003b;Shepard 1993) that have sought to streamline models of validity and to add some more explicit guidelines for validation by stating the intended interpretation and use of the scores in the form of an argument. The argument would provide an explicit statement of the claims inherent in the proposed interpretation and use of the scores (Cronbach 1988).…”