Characterizing the literature on validity and assessment in medical education: a bibliometric study

Abstract

IntroductionAssessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e. g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education.MethodSearches in Medline (Ovid), Web of Science, ERIC, EMBASE (Ovid), and PsycINFO (Ovid) identified articles reporting on assessment of learners in medical education published since 1999. Included articles were coded for geographic origin, journal, journal category, targeted assessment, and authors. A map of collaborations between prolific authors was generated.ResultsA total of 2,863 articles were included. The majority of articles were from the United States, with Canada producing the most articles per medical school. Most articles were published in journals with medical categorizations (73.1% of articles), but Medical Education was the most represented journal (7.4% of articles). Articles reported on a variety of assessment tools and approaches, and 89 prolific authors were identified, with a total of 228 collaborative links.DiscussionLiterature reporting on validation of assessments in medical education is heterogeneous. Literature is produced by a broad array of authors and collaborative networks, reported to a broad audience, and is primarily generated in North American and European contexts. Our findings speak to the heterogeneity of the medical education literature on assessment validation, and suggest that this heterogeneity may stem, at least in part, from differences in constructs measured, assessment purposes, or conceptualizations of validity.Electronic supplementary materialThe online version of this article (10.1007/s40037-018-0433-x) contains supplementary material, which is available to authorized users.

Validity as a social imperative: users’ and leaders’ perceptions

“…The study was conducted in Canada which has been identified as a leader in research in HPE (e.g. : number of publications per medical school), in assessment and validity in medical education specifically, 15 and in medical education more broadly.…”

El Examen de Ingreso a la Universidad Nacional Autónoma de México: Evidencias de Validez de una Prueba de Alto Impacto y Gran Escala

“…Para que los resultados de los procesos de evaluación tengan un robusto sustento y se utilicen de forma apropiada, es indispensable abordarlos desde la lente de la validez. Validez de un proceso de evaluación es el grado con el que mide lo que se supone que mide, tradicionalmente se clasificaba como las tres C: de contenido, de criterio y de constructo (Buntis, Buntis y Eggert, 2017;Sánchez-Mendiola, 2015;Young et al, 2018). En las últimas décadas el concepto ha evolucionado a una definición más amplia (AERA, 2014;Gregory, 2016;Kane, 2016;Kane y Bridgeman, 2017;Shepard, 2016).…”

How argumentation theory can inform assessment validity: A critical review

“…For instance, Cook, Hatala, and Downing have helped bring argument‐based validity frameworks, such as those of Messick and Kane, 5,6 into the HPE assessment sphere 1–3,7–14 . St. Onge and Young have helped map out the current state of our field's understanding of validity 4,15–17 . Govaerts, Schuwirth, van der Vleuten, and others have progressed our fields view of validity in the context of programmatic assessment 18–26 .…”