Assessment of the Intrinsic CanMEDS Roles in Diagnostic Radiology Residents using an Objective Structured Clinical Assessment (OSCE)

Abstract

Purpose: Non-Medical Expert or Intrinsic CanMEDS Roles, as outlined by the Royal College of Physicians and Surgeons of Canada, can be difficult to evaluate in an objective and specialty-specific manner. This study investigates an Objective Structured Clinical Examination (OSCE) evaluation tool to assess these competencies for Diagnostic Radiology Residents. Methods:A five station CanMEDS OSCE was developed for postgraduate year 3 and 4 Residents to evaluate the Communicator, Collaborator, Manager, Health Advocate, Scholar and Professional Roles. Performance was assessed by postgraduate year 5 Residents using standardized scoring rubrics. CanMEDS OSCE scores were correlated with American College of Radiology (ACR) scores and Medical Expert OSCEs.Results: Seventy Residents in three separate cohorts (n=21, 26, 23) participated in the CanMEDS OSCE. Mean station scores were consistent across cohorts. In general, one-way ANOVAs showed no effect of postgraduate year on station scores. There were no significant correlations between CanMEDS OSCE scores and ACR exam scores or CanMEDS OSCE scores and Medical Expert OSCE scores, demonstrating divergent construct validity. In turn, this indicates construct validity for the CanMEDS OSCE by demonstrating that unique competencies are being measured. In contrast, there was a correlation between ACR exam scores and Medical Expert OSCE scores, confirming that these both assess the same construct.Conclusions: An OSCE can be a useful assessment tool to assess Intrinsic CanMEDS Roles in a specialty-specific manner. Correlational analyses indicate that unique competencies are being evaluated that are not captured by other traditional assessment means.

Which Aspects of the CanMEDS Competencies are Most Valued in Radiologists? Perspectives of Trainees From Other Specialties

“…We hope that bridging the gap with EPAs in this way will support the overarching goals of training and mitigate tendencies toward a reductionist numerical approach to the completion of milestones. 13,16 We did not specifically perform respondent validation of the results, and there could be bias in the subset of those contacted who chose to respond to the survey, given the low response rate of 12%. However, these results are congruent with our preliminary interviews and pilot testing.…”

“…Challenges related to understanding and tools for evaluation of CanMEDS roles other than Medical Expert have previously been noted, 9 and have led to the development of specific assessment tools such as a CanMEDS OSCE for radiology. 13 Although 360 degree evaluations incorporate feedback from others in the radiology department, it is unclear how frequently feedback is solicited from others outside of the department after residents have completed the PGY1 rotations (during which they rotate through multiple other services) and are in the radiology department full-time. As the abstract nature of the general competencies renders them less visible and easily measured, 14 it has been proposed that aligning these training objectives in a more tangible way with clinical practice will facilitate implementation.…”

“…We hope that bridging the gap with EPAs in this way will support the overarching goals of training and mitigate tendencies toward a reductionist numerical approach to the completion of milestones. 13,16…”

Evaluación de habilidades transversales de comunicación y colaboración a través de un examen clínico objetivo estructurado de radiología de urgencias para residentes de un programa universitario de especialidad en radiología.

“…Si bien existen algunas experiencias de ECOE en radiología en la literatura educacional, estos son escasos y no están enfocados en radiología de urgencias (15,16,17,18) , por lo que esta intervención educacional es relevante y novedosa. Como fortaleza de esta experiencia educacional está la implementación del informe simulado, que se acerca a la realidad del trabajo de los residentes en las sesiones diarias a través de la redacción de informes de uno o más casos radiológicos en un tiempo determinado.…”