P042: Workplace-based assessment in emergency medicine: how do physicians use entrustment anchors?

Abstract

Introduction: Competency based medical education (CBME) has triggered widespread utilization of workplace-based assessment (WBA) tools in postgraduate training programs. These WBAs predominately use rating scales with entrustment anchors, such as the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE). However, little is known about the factors that influence a supervising physician's decision to assign a particular rating on scales using entrustment anchors. This study aimed to identify the factors that influence supervisors’ ratings of trainees using WBA tools with entrustment anchors at the time of assessment and to explore the experiences with and challenges of using entrustment anchors in the emergency department (ED). Methods: A convenience sample of full-time emergency medicine (EM) faculty were recruited from two sites within a single academic Canadian EM hospital system. Fifty semi-structured interviews were conducted with EM physicians within two hours of completing a WBA for an EM trainee. Interviews were audio-recorded, transcribed verbatim, and independently analyzed by two members of the research team. Themes were stratified by trainee level, rating and task. Results: Interviews involved 73% (27/37) of all EM staff and captured assessments completed on 83% (37/50) of EM trainees. The mean WBA rating of studied samples was 4.34 ± 0.77 (2 to 5), which was similar to the mean rating of all WBAs completed during the study period. Overall, six major factors were identified that influenced staff WBA ratings: amount of guidance required, perceived competence through discussion and questioning, trainee experience, clinical context, past experience working with the trainee, and perceived confidence. The majority of staff denied struggling to assign ratings. However, when they did struggle, it involved the interpretation of WBA anchors and their application to the clinical context in the ED. Conclusion: Several factors appear to be taken into account by clinical supervisors when they make decisions regarding the particular rating that they will assign a trainee on a WBA that uses entrustment anchors. Not all of these factors are specific to that particular clinical encounter. The results from this study further our understanding on the use of entrustment anchors within the ED and may facilitate faculty development regarding WBA completion as we move forward in CBME.