Validity and the Consequences of Test Interpretation and Use

Hubley, Anita M.; Zumbo, Bruno D.

doi:10.1007/s11205-011-9843-4

Cited by 123 publications

(110 citation statements)

References 21 publications

Supporting

Mentioning

108

Contrasting

Unclassified

Order By: Relevance

“…Therefore, we decided that all chapters would, at least, follow the generic framework in the Standards (AERA et al 1999) wherein five sources of validity evidence were of focus: (a) content-related, (b) response processes, (c) internal structure, (d) associations with other variables, and (e) consequences. The syntheses also addressed whether recent work in validity theory was cited as informing the validation practice (e.g., Hubley and Zumbo 1996, 2011, 2013Kane 2006;Messick 1989;Zumbo 2007Zumbo , 2009.…”

Section: Prefacementioning

confidence: 99%

Validity and Validation in Social, Behavioral, and Health Sciences

Zumbo¹,

Chan²

2014

Social Indicators Research Series

View full text Add to dashboard Cite

This new series aims to provide a public forum for single treatises and collections of papers on social indicators research that are too long to be published in our journal Social Indicators Research. Like the journal, the book series deals with statistical assessments of the quality of life from a broad perspective, It welcomes the research on a wide variety of substantive areas, including health, crime, housing, education, family life, leisure activities, transportation, mobility, economics, work, religion and environmental issues. These areas of research will focus on the impact of key issues such as health on the overall quality of life and vice versa. An international review board, consisting of Ruut Veenhoven, Joachim Vogel, Ed Diener, Torbjorn Moum, Mirjam A.G. Sprangers and Wolfgang Glatzer, will ensure the high quality of the series as a whole. This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher's location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com) PrefaceIn this edited volume, 15 research syntheses of the validity evidence reported in different research areas are presented. The chapters were purposefully chosen to reflect a wide variety of disciplines, journals, or measures. Eight of the chapters focused on particular journals ranging from measurement and assessment journals like Educational and Psychological M...

show abstract

Section: Prefacementioning

confidence: 99%

Validity and Validation in Social, Behavioral, and Health Sciences

Zumbo¹,

Chan²

2014

Social Indicators Research Series

View full text Add to dashboard Cite

show abstract

“…220-221). Based on this description, we expect the degree of validity of a measure to vary by context and/or respondents characteristics [37,38]. The above evidence suggested that both versions of the CWEQ II are less valid and reliable measures when used with nurses who have high levels of SE than when used with nurses who have lower levels of SE (but not the lowest).…”

Section: Validity and Reliabilitymentioning

confidence: 97%

“…Second, unlike traditional views of validity that have described validity as a property of the measure itself, more contemporary views have described it as the extent to which empirical evidence supports the intended meaning of test scores for its proposed purpose [37,38]. In other words, measurement scholars acknowledge "validity is about whether the inference one makes is appropriate, meaningful, and useful given the individual or sample with which one is dealing, and the context in which the test user and individual/sample are working [38] (pp. 220-221).…”

Section: Validity and Reliabilitymentioning

confidence: 99%

How Well Does the CWEQ II Measure Structural Empowerment? Findings from Applying Item Response Theory

Havaei

Dahinten

2017

Administrative Sciences

View full text Add to dashboard Cite

Abstract:The main purpose of this paper is to examine the psychometric properties of the original five-point CWEQ II using Item Response Theory (IRT) methods, followed by an examination of the revised three-point CWEQ II. (1) Background: The psychometric properties of the CWEQ II have not been previously assessed using more robust techniques such as IRT. (2) Methods: This is a secondary analysis of baseline data from 1067 staff nurses whose leaders had attended a leadership development program. Data were analyzed using a polytomous IRT model. (3) Results: The two versions of CWEQ II fit the SE data equally as each had only one poor-fitting item. For the five-point CWEQ II, discriminant ability was poor for a majority of the items; one item demonstrated a disordinal step difficulty parameter; and item reliability was supported for a relatively wider range of SE levels. The discriminant ability and reliability of items for the three-point CWEQ II was better than those of the five-point CWEQ II, but for a narrower range of SE levels; and the disordinal step difficulty parameter was resolved. (4) Conclusion: The appropriate use of each version of the scale depends on the conditions of the work setting targeted.

show abstract

“…Hubley and Zumbo (2011) introduced a new conceptualization of test validity that draws a clearer attention to the inherent elements (i.e., values), external influences (i.e., social consequences and side effects), and the process of validation in validity. The framework is meant to highlight the test's purpose and use, and its concomitant values, social and personal consequences, as well as social and personal side effects.…”

Section: How Standards Are Tied To Validity Theory and A Focus On Tesmentioning

confidence: 99%

“…Very importantly, these consequences and/or side effects (either personal and/or social) may also influence test score inferences and use. Second, test-score inference and use is effected Hubley and Zumbo (2011) with permission of Social Indic. Res, Vol.…”

Section: How Standards Are Tied To Validity Theory and A Focus On Tesmentioning

confidence: 99%

Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation

Zumbo

2016

Appl. Physiol. Nutr. Metab.

Self Cite

View full text Add to dashboard Cite

A critical step in the development and use of tests of physical fitness for employment purposes (e.g., fitness for duty) is to establish 1 or more cut points, dividing the test score range into 2 or more ordered categories reflecting, for example, fail/pass decisions. Over the last 3 decades elaborated theories and methods have evolved focusing on the process of establishing 1 or more cut-scores on a test. This elaborated process is widely referred to as "standard-setting". As such, the validity of the test score interpretation hinges on the standard-setting, which embodies the purpose and rules according to which the test results are interpreted. The purpose of this paper is to provide an overview of standard-setting methodology. The essential features, key definitions and concepts, and various novel methods of informing standard-setting will be described. The focus is on foundational issues with an eye toward informing best practices with new methodology. Throughout, a case is made that in terms of best practices, establishing a test standard involves, in good part, setting a cut-score and can be conceptualized as evidence/databased policy making that is essentially tied to test validity and an evidential trail.Key words: standard-setting, physical fitness testing, cut-score, pass-score, test validity, performance standards, psychometrics.Résumé : Une étape cruciale du développement et de l'utilisation des tests de condition physique à des fins d'emploi (p. ex., aptitude au travail) est de déterminer un ou des seuils de coupure sur l'échelle des résultats pour obtenir deux catégories ordonnées ou plus d'aptitude indiquant, par exemple, une réussite ou un échec. Durant les trois dernières décennies, les théories et les méthodes se sont axées de plus en plus sur le processus de détermination d'un ou des seuils de coupure dans un test. Ce processus d'élaboration est généralement connu sous le terme de « normalisation ». À ce titre, la validité de l'interprétation du résultat au test repose sur l'établissement de la norme dont dépendent les intentions et les règles d'interprétation des résultats. Cet article a pour objectif de donner un aperçu de la méthodologie d'établissement des normes. On y décrit les caractéristiques essentielles, les définitions clés et concepts ainsi que diverses nouvelles méthodes par lesquelles on établit des normes. L'accent est placé sur les problématiques essentielles tout en gardant le cap sur les meilleures pratiques avec la nouvelle méthodologie. Tout au long, on démontre que, en matière de bonnes pratiques, l'établissement d'une norme dans un test signifie en grande partie d'établir un seuil de coupure dont la valeur peut justifier la politique décisionnelle basée essentiellement sur la validité du test et de son évidence résultante. [Traduit par la Rédaction] Mots-clés : normalisation, évaluation de la condition physique, seuil de coupure, seuil de réussite, validité du test, normes de performance, psychométrie.

show abstract

Validity and the Consequences of Test Interpretation and Use

Abstract: Consequential validity, Early development instrument, Educational achievement, Psychological assessment, Testing, Multilevel measures, Social consequences, Test interpretation, Validity, Value implications, Values,

Cited by 123 publications

References 21 publications

Validity and Validation in Social, Behavioral, and Health Sciences

Validity and Validation in Social, Behavioral, and Health Sciences

How Well Does the CWEQ II Measure Structural Empowerment? Findings from Applying Item Response Theory

Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation

Contact Info

Product

Resources

About