Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study

Thoma, Brent; Hall, Andrew K.; Clark, Kevin; Meshkat, Nazanin; Cheung, Warren J.; Desaulniers, Pierre; Ffrench, Cheryl; Meiwald, Allison; Meyers, Christine; Patocka, Catherine; Beatty, Lorri; Chan, Teresa M.

doi:10.4300/jgme-d-19-00803.1

Cited by 33 publications

(40 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If entrustment scales help mitigate the rightward shift , consideration must be given to why scores were so high, this early in the year. Finding a persistence of the rightward shift is not unique to our study 41 . One potential explanation is that EPAs, which are divided into four stages of training (see Table, Appendix ), are appropriately staged for the skill level of the corresponding PGY trainee.…”

Section: Discussionmentioning

confidence: 76%

Exploring the use of rating scales with entrustment anchors in workplace‐based assessment

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose Competency‐based medical education (CBME) has prompted widespread implementation of workplace‐based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. Method A convenience sample of 50 semi‐structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line‐by‐line coding to identify themes. Results Interviews captured interactions between 70% (26/37) of full‐time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were ‘in the room’ or not. Conclusions By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of ‘in‐the‐moment’ assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.

show abstract

Section: Discussionmentioning

confidence: 76%

Exploring the use of rating scales with entrustment anchors in workplace‐based assessment

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…40 Recent studies have also emphasized the importance of program evaluation at both a local level and across programs at a national level. 41…”

Section: Future Directionsmentioning

confidence: 99%

Improving Medical Education in Hematology and Transfusion Medicine in Canada: Standards and Limitations

Laureano¹,

Mithoowani²,

Tseng³

et al. 2021

AMEP

View full text Add to dashboard Cite

The paradigm of medical education is evolving with the introduction of competency-based medical education (CBME) and it is crucial that residency programs adapt. In this paper, we provide an overview of the current status of medical education in Hematology in Canada including models of training, assessment methods, anticipated challenges, and the effects of the COVID-19 pandemic. We will also discuss additional training that can be pursued after a Hematology residency, with a particular focus On Transfusion Medicine as it was one of the first programs to implement a competency-based curriculum. Finally, we explore the future directions of medical education in Hematology and Transfusion Medicine.

show abstract

“…[14][15][16][17][18][19][20][21][22] These limitations have prevented the application of audit and feedback techniques in faculty development. However, the number of EPA assessments completed by faculty within CBME dwarfs that of traditional assessment programs, [23][24][25] overcoming the aforementioned challenge of insufficient data. These EPA assessments represent a small part of learners' educational experience, but contain important feedback that informs competence committees and learners about their progress.…”

Section: Introductionmentioning

confidence: 99%

Developing a dashboard for faculty development in competency-based training programs: a design-based research project

et al. 2021

Self Cite

View full text Add to dashboard Cite

Background: Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires frequent assessments of entrustable professional activities (EPAs). Faculty struggle to provide helpful feedback and assign appropriate entrustment scores. CBME faculty development initiatives rarely incorporate teaching metrics. Dashboards could be used to visualize faculty assessment data to support faculty development. Methods: Using a design-based research process, we identified faculty development needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. Data was collected within the emergency medicine residency program at the University of Saskatchewan through interviews with program leaders, faculty development experts, and faculty participating in development sessions. Two investigators thematically analyzed interview transcripts to identify faculty needs that were audited by a third investigator. The needs were described using representative quotes and the dashboard elements designed to address them. Results: Between July 1, 2019 and December 11, 2020 we conducted 15 interviews with nine participants (two program leaders, three faculty development experts, and four faculty members). Three needs emerged as themes from the analysis: analysis of assessments, contextualization of assessments, and accessible reporting. We addressed these needs by designing an accessible dashboard to present contextualized quantitative and narrative assessment data for each faculty member. Conclusions: We identified faculty development needs related to EPA assessments and designed dashboard elements to meet them. The resulting dashboard was used for faculty development sessions. This work will inform the development of CBME assessment dashboards for faculty.

show abstract

Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study

Cited by 33 publications

References 40 publications

Exploring the use of rating scales with entrustment anchors in workplace‐based assessment

Exploring the use of rating scales with entrustment anchors in workplace‐based assessment

Improving Medical Education in Hematology and Transfusion Medicine in Canada: Standards and Limitations

Developing a dashboard for faculty development in competency-based training programs: a design-based research project

Contact Info

Product

Resources

About