Validity

Messick, Samuel

doi:10.1002/j.2330-8516.1987.tb00244.x

Cited by 268 publications

(464 citation statements)

References 381 publications

(304 reference statements)

Supporting

Mentioning

448

Contrasting

Unclassified

Order By: Relevance

“…Messick (1989) discussed Toulmin and Popper at length. 20 An examination of the titles of research published in ETS's first decades clearly emphasizes predictive validity.…”

Section: Validitymentioning

confidence: 99%

“…Calls for a unitary view in the form of construct validity began early on. Messick (1989) quoted Loevinger that, "since predictive, concurrent, and content validities are all essentially ad hoc, construct validity is the whole of validity from a scientific point of view" (p. 17). Messick elaborated that idea, stating that, "almost any kind 22 Of course, generalizability theory had been under development (Rajaratnam et al 1965) during the 1960s, and it was concerned with components of observed score variability.…”

Section: Validity Theory At Etsmentioning

confidence: 99%

“…Importantly, Messick's (1980Messick's ( , 1989 unitary view explicitly extended to the consequences of test use, with implications for the use of constructed-response formats. Although the message was not well received in some quarters (Kane 2006, p. 54), it was in others.…”

Section: Validity Theory At Etsmentioning

confidence: 99%

“…Messick's (1989) work was largely dedicated to expanding the conception of validity to include not only the psychometric attributes of the test, the evidentiary aspect of validation, but also the repercussions that the use of the test could have, the consequential aspect. This broader view did not necessarily endorse the use of one format over another but provided a framework in which constructedresponse formats had a greater chance for acceptance.…”

Section: Summary and Reflectionsmentioning

confidence: 99%

See 3 more Smart Citations

A Historical Survey of Research Regarding Constructed-Response Formats

Bejar

2017

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

This chapter chronicles ETS research and development contributions related to the use of constructed-response item formats. 1 The use of constructed responses in testing dates back to imperial China, where tests were used in the selection of civil servants. However, in the United States, the multiple-choice format became dominant during the twentieth century, following its invention and use by the SAT ® examinations created by the College Board in 1926. When ETS was created in 1947, post-secondary admissions testing was largely based on tests consisting of multiplechoice items. However, from the start, there were two camps at ETS: those who believed that multiple-choice tests were sufficiently adequate for the purpose of assessing "verbal" skills and those who believed that "direct" forms of assessment requiring written responses had a role to play. For constructed-response formats to regain a foothold in American education several hurdles would need to be overcome. Research at ETS was instrumental in overcoming those hurdles.The first hurdle was that of reliability, specifically the perennial issue of low interrater agreement, which plagued the acceptance of constructed-response formats for most of the twentieth century. The second hurdle was broadening the conception of validity to encompass more than predictive considerations, a process that began with the introduction of construct validity by Cronbach and Meehl (1955). Samuel Messick at ETS played a crucial role in this process by making construct validity relevant to educational tests. An inflexion point in the process of reincorporating constructed-response formats more widely in educational tests was marked 1 Constructed responses to a prompt or question can range in scope and complexity. Perhaps the most common constructed response is the written essay. However, short written responses to questions are also considered to be constructed, as are spoken answers in response to a prompt, mathematical responses (equations, plotted functions, etc.), computer programs, and graphical responses such as architectural designs.

show abstract

“…Messick (1989) discussed Toulmin and Popper at length. 20 An examination of the titles of research published in ETS's first decades clearly emphasizes predictive validity.…”

Section: Validitymentioning

confidence: 99%

Section: Validity Theory At Etsmentioning

confidence: 99%

Section: Validity Theory At Etsmentioning

confidence: 99%

Section: Summary and Reflectionsmentioning

confidence: 99%

See 2 more Smart Citations

A Historical Survey of Research Regarding Constructed-Response Formats

Bejar

2017

Methodology of Educational Measurement and Assessment

View full text Add to dashboard Cite

show abstract

“…Impact is another link in the chain described by Crooks, Kane, and Cohen (1996) and refers not to identifiable stages of the assessment process, but "the consequential basis of validity" (Messick 1989). Threats to validity can come from assessment processes being perceived as unfair as a result of exclusion from further learning opportunities due to RVA results.…”

Section: Delivering Rva and Strengthening Professionalismmentioning

confidence: 99%

Global Perspectives on Recognising Non-formal and Informal Learning

Singh¹

2015

Technical and Vocational Education and Training: Issues, Concerns and Prospects

View full text Add to dashboard Cite

Aims and ScopeThe purpose of the Technical and Vocational Education and Training: Issues, Concerns and Prospects Book Series is to meet the needs of those interested in an in-depth analysis of current developments concerning various aspects of education for the world of work with particular reference to technical and vocational education and training. The Series examines areas that are at the 'cutting edge' of the field and are innovative in nature. It presents best and innovative practice, explores controversial topics and uses case studies as examples.The audience includes policymakers, practitioners, administrators, planners, researchers, teachers, teacher educators, students and colleagues in other fields interested in learning about TVET, in both developed and developing countries, countries in transition and countries in a post-conflict situation. The Series complements the International Handbook of Technical and Vocational Education and Training, with the elaboration of specific topics, themes and case studies in greater breadth and depth than is possible in the Handbook. Topics covered include training for the informal economy in developing countries; education of adolescents and youth for academic and vocational work; financing education for work; lifelong learning in the workplace; women and girls in technical and vocational education and training; effectively harnessing ICTs in support of TVET; planning of education systems to promote education for the world of work; recognition, evaluation and assessment; education and training of demobilized soldiers in post-conflict situations; TVET research; and school to work transition.More information about this series at

show abstract

Psychometric and edumetric validity of dimensions of geomorphological knowledge which are tapped by concept mapping

Hoz

Bowman

Chacham³

1997

J. Res. Sci. Teach.

View full text Add to dashboard Cite

The validity of several dimensions of knowledge which were inferred from concept maps was assessed for its psychometric and edumetric aspects. Data were collected from 14 students who enrolled in the university first-year introductory geomorphology course and in its prerequisite introductory geology course. They took an objective geomorphology test, the tree construction task, and the Standardized Concept Structuring Analysis Technique (SConSAT) version of concept mapping. Comparisons among these dimensions of knowledge before and after the geomorphology course yielded convergent evidence. For the psychometric perspective, the SConSAT version of concept mapping and tree construction had similar knowledge structure representations, and the cognitive map correctness was moderately positively correlated with the objective test but not with the geomorphology course test. For the edumetric perspective, the majority of the dimensions of knowledge structures from the SConSAT showed large improvements following the geomorphology course. This evidence shows that the knowledge structure dimensions have moderate to good construct validity which warrant their widespread use for evaluating learning outcomes in both experimental and classroom settings. © 1997 John Wiley & Sons, Inc. J Res Sci Teach 34: 925-947, 1997. Concept mapping is widely used in science education research and development. Most of its educational applications aim to describe features of student knowledge, assess it, and detect and characterize structural and substantive changes in this knowledge (namely, learning outcomes) and for instructional purposes (e.g., Horton, McConney, Gallo, Woods, Senn, & Hamelin, 1993;Mason, 1992;Roth & Roychoudhury, 1992, 1994Wallace & Mintzes, 1990;Wandersee, 1990). This interest may be due in part to the fact that concept mapping is consistent with a constructivist perspective of learning when it is used to examine changes in the content and organization of student knowledge by emphasizing the process of the construction and the uniqueness of the individual products (Beyerbach & Smith, 1990;Roth & Roychoudhury, 1992). In this articles we report the assessment of the psychometric and edumetric aspects of the validity of certain dimensions of knowledge derived by concept mapping in the context of a university course in geomorphology. We will review definitions of cognitive structure, outline JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 34, NO. 9, PP. 925-947 (1997) © the ideas that underlie construct validation, and describe briefly concept mapping as a probe of cognitive structure. Theoretical FrameworkIn recent decades, educational and psychological researchers came to treat learning outcomes in terms of the relations between the structure of the discipline (content area) and the individual's "cognitive structure" or "knowledge structure" (e.g., Howard, 1987;Nitko, 1989). Some of the definitions for the latter hypothetical construct are (1) a semantic space that is an organization or pattern of the internal...

show abstract

Validity

Cited by 268 publications

References 381 publications

A Historical Survey of Research Regarding Constructed-Response Formats

A Historical Survey of Research Regarding Constructed-Response Formats

Global Perspectives on Recognising Non-formal and Informal Learning

Psychometric and edumetric validity of dimensions of geomorphological knowledge which are tapped by concept mapping

Contact Info

Product

Resources

About