Towards a program of assessment for health professionals: from training into practice

Abstract

Despite multifaceted attempts to "protect the public," including the implementation of various assessment practices designed to identify individuals at all stages of training and practice who underperform, profound deficiencies in quality and safety continue to plague the healthcare system. The purpose of this reflections paper is to cast a critical lens on current assessment practices and to offer insights into ways in which they might be adapted to ensure alignment with modern conceptions of health professional education for the ultimate goal of improved healthcare. Three dominant themes will be addressed: (1) The need to redress unintended consequences of competency-based assessment; (2) The potential to design assessment systems that facilitate performance improvement; and (3) The importance of ensuring authentic linkage between assessment and practice. Several principles cut across each of these themes and represent the foundational goals we would put forward as signposts for decision making about the continued evolution of assessment practices in the health professions: (1) Increasing opportunities to promote learning rather than simply measuring performance; (2) Enabling integration across stages of training and practice; and (3) Reinforcing point-in-time assessments with continuous professional development in a way that enhances shared responsibility and accountability between practitioners, educational programs, and testing organizations. Many of the ideas generated represent suggestions for strategies to pilot test, for infrastructure to build, and for harmonization across groups to be enabled. These include novel strategies for OSCE station development, formative (diagnostic) assessment protocols tailored to shed light on the practices of individual clinicians, the use of continuous workplace-based assessment, and broadening the focus of high-stakes decision making beyond determining who passes and who fails. We conclude with reflections on systemic (i.e., cultural) barriers that may need to be overcome to move towards a more integrated, efficient, and effective system of assessment.

Targeting Assessment for Learning within Pharmacy Education

“…As experienced assessment experts within medical education, Eva and colleagues caution that a lingering desire for some raters to simply "check a box" can result in students only achieving a minimal competency to pass an assessment, as opposed to excelling and going beyond that level of ability which is good enough to pass. 32 While "box-checking" can seem quickest and easiest for some evaluators' time in that moment, pharmacy educators must overcome a minimalist "boxchecking" tendency, as it is the least helpful in providing meaningful feedback for students. This "box-checking" tendency can become a vicious cycle of increasingly minimal scoring, leading to increasingly minimal student feedback, resulting in increasingly minimal opportunities for growth in students' learning -and losing any potential for AFL.…”

“…32,35 However, deliberately harnessing formative use will often times need more thought; formative uses are not often within many educators' usual assessment paradigm. [10][11][12][13][14] One example of a combination would be administering multiple quizzes to verify students' content knowledge -including scores in final grade calculation (ie, summative), while also leveraging the iterative quizzes to provide ongoing formative feedback (eg, "How am I doing with course content at this point?…”

“…It should be regarded as a suggestion that AFL be considered for some applications rather than a blanket default to an AFL approach for all assessment activities. 35 Educational assessments should not be dichotomized into formative and summative types, 32,35 except to illustrate the overwhelming overreliance of many present practices (inside the classroom as well as programmatically) for a summative AOL approach. An AOL approach is needed at times, but should not be used exclusively.…”

An innovative approach to identifying learning needs for intrinsic CanMEDS roles in continuing professional development

“…For example, faculty members often avoid assessing intrinsic CanMEDS roles, particularly those related to collaborator, leader, health advocate, and scholar, which suggests that individuals may not understand the constructs underlying such roles [11]. Furthermore, physicians do not effectively assess their own professional development needs [14], and such difficulties in self-assessing are likely magnified when it comes to intrinsic competencies, due to poor understanding of these constructs.…”

“…While such approaches are certainly an important step in identifying CPD learning needs, the results from such surveys are limited by the extent to which physicians can accurately assess their learning needs. Given that (a) physicians are not particularly effective at self-assessing gaps in their knowledge and abilities[14], and (b) they do not fully understand the constructs relating to intrinsic competencies [10–13], CPD providers need to find alternative approaches to identify learning needs as they pertain to personal and professional competencies. Since many of physicians learning needs arise from problems arising in practice, we were interested in whether physicians’ descriptions of challenges encountered in practice could provide insight into learning needs pertaining to intrinsic competencies.…”