Validating the Interpretations and Uses of Test Scores

Abstract

To validate an interpretation or use of test scores is to evaluate the plausibility of the claims based on the scores. An argument‐based approach to validation suggests that the claims based on the test scores be outlined as an argument that specifies the inferences and supporting assumptions needed to get from test responses to score‐based interpretations and uses. Validation then can be thought of as an evaluation of the coherence and completeness of this interpretation/use argument and of the plausibility of its inferences and assumptions. In outlining the argument‐based approach to validation, this paper makes eight general points. First, it is the proposed score interpretations and uses that are validated and not the test or the test scores. Second, the validity of a proposed interpretation or use depends on how well the evidence supports the claims being made. Third, more‐ambitious claims require more support than less‐ambitious claims. Fourth, more‐ambitious claims (e.g., construct interpretations) tend to be more useful than less‐ambitious claims, but they are also harder to validate. Fifth, interpretations and uses can change over time in response to new needs and new understandings leading to changes in the evidence needed for validation. Sixth, the evaluation of score uses requires an evaluation of the consequences of the proposed uses; negative consequences can render a score use unacceptable. Seventh, the rejection of a score use does not necessarily invalidate a prior, underlying score interpretation. Eighth, the validation of the score interpretation on which a score use is based does not validate the score use.

Literacia em saúde: tradução e validação de instrumento para pesquisa em promoção da saúde no Brasil

“…A estratégia metodológica proposta para análises dos dados neste estudo tem sido considerada bastante robusta para a avaliação de instrumentos de medida em diferentes áreas e amostras 22,23,24 . A sensibilidade psicométrica dos itens foi considerada adequada quando os valores absolutos de assimetria eram menores de 3 e os valores de curtose menores que 7 22,25 .…”

University entrance language tests: A matter of justice

“…IUA-based validation is the process of empirically determining whether real-world test score uses or inferences provide answers to specific problems within a specific context (Gorin, 2007;Kane, 2013). Kane demands strong empirical evidence and, when this evidence contradicts the claim made by the score user or the test developer, the validity argument cannot be maintained.…”

“…A score only becomes real when it has real-life consequences, such as access to a valued position, service, or status. For that reason, most validation theories argue that validating a test without considering its social context and consequences is inadequate (Bachman & Palmer, 2010;Kane, 2013;Kane, Kane, & Clauser, 2017). What requires validation is not only the test itself (e.g., Borsboom & Markus, 2013), but also the way in which a score is interpreted and used (Kane, 2013).…”

Validation of the Oral Interaction Strategy Scale for speakers of Chinese as the first language in elementary schools

“…Bejarano et al, 1997;Celce-Murcia, 2008;Dörnyei & Scott, 1997;Nakatani, 2006;Naughton, 2006;Tarone, 1980) and Chinese communication research (e.g., Gao & Ting-Toomey, 1998). Our analyses should be able to provide sufficient empirical evidence justifying the hypothesized appropriateness of using the OISI to profile the strategic oral communication behaviors of young L1 Chinese learners in Hong Kong primary classrooms, or evidence supporting the construct validity of the scale (Kane, 2006(Kane, , 2010(Kane, , 2013. The results of our factor analysis in Study 1 identified seven categories of interaction strategies: expressing actively; asking for opinions; expressing attitude; correcting errors; giving clarification; requesting clarification; and using non-verbal language.…”