Cognitive and Psychomotor Entrustable Professional Activities: Can Simulators Help Assess Competency in Trainees?

Abstract

Background An entrustable professional activity describes a professional task that postgraduate residents must master during their training. The use of simulation to assess performance of entrustable professional activities requires further investigation. Questions/purposes (1) Is simulation-based assessment of resident performance of entrustable professional activities reliable? (2) Is there evidence of important differences between Postgraduate Year (PGY)-1 and PGY-4 residents when performing simulated entrustable professional activities? Methods Three entrustable professional activities were chosen from a list of competencies: management of the patient for total knee arthroplasty (TKA); management of the patient with an intertrochanteric hip fracture; and management of the patient with an ankle fracture. Each assessment of entrustable professional activity was 40 minutes long with three components: preoperative management of a patient (history-taking, examination, image interpretation); performance of a technical procedure on a sawbones model; and postoperative management of a patient (postoperative orders, management of complications). Residents were assessed by six faculty members who used checklists based on a modified Delphi technique, an overall global rating scale as well as a previously validated global rating scale for the technical procedure component of each activity. Nine PGY-1 and nine PGY-4 residents participated in our simulated assessment. We assessed reliability by calculating the internal consistency of the mean global rating for each activity as well as the interrater reliability between the faculty assessment and The institution of one or more of the authors (TD, MN) has received, during the study period, funding from a Core Competency Innovation Grant from the OMeGA Medical Grants Association. All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research 1 editors and board members are on file with the publication and can be viewed on request. Each author certifies that his or her institution approved the human protocol for this investigation, that all investigations were conducted in conformity with ethical principles of research, and informed consent for participation in the study was obtained. This work was performed at the University of Toronto, Toronto, Ontario, Canada. Clinical Orthopaedics and Related Research ®A Publication of The Association of Bone and Joint Surgeons® blinded review of videotaped encounters. We sought evidence of a difference in performance between PGY-1 and PGY-4 residents on the overall global rating scale for each station of each entrustable professional activity. Results The reliability (Cronbach's a) for the hip fracture activity was 0.88, it was 0.89 for the ankle fracture activity, and it was 0.84 for the TKA activity. A strong correlation was seen between blinded observer video review and faculty scores (mean 0.87 [0.07], p \ 0.001). For the hip fracture entrustable professional activity, the PGY-4 group had a hi...

A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018

“…However, the implementation papers to date do give some insight into the potential of various assessment methods, which could be explored further. The role of simulation in the assessment of EPAs warrants further exploration, given its successful use in a number of the included studies . Similarly, the use of a portfolio to track trainee progress on EPAs prior to a summative entrustment decision being made shows some promise.…”

“…The role of simulation in the assessment of EPAs warrants further exploration, given its successful use in a number of the included studies. 46,65 Similarly, the use of a portfolio to track trainee progress on EPAs prior to a summative entrustment decision being made shows some promise. The portfolio could potentially be accessed by supervisors at critical moments to provide an empirical basis for entrusting a learner with a task in practice.…”

Prior Generic Arthroscopic Volume Correlates with Hip Arthroscopic Proficiency

“…In accordance with educational theory around psychomotor skill development through "sustained deliberate practice", simulation models may play an important role in both the development and maintenance of expertise [75][76][77][78][79] . Global changes in surgical training have used such educational principles to push for technical skills training and assessment on simulators.…”

Objective measures of operating room wire navigation performance

“…Competency has been labeled as the ability to perform the task without supervision. 6 This metric is a means of evaluating independent practice in the less experimentally controllable OR environment. The link to the actual OR setting is critical in the success and development of simulation based medical education and is some of the first research in orthopedics validating performance transfer to the operating room.…”

“…9 Fixation of intertrochanteric hip fractures has been identified as a critical task to be mastered prior to graduation from residency. 6 The task chosen, wire navigation, is used in a variety of orthopedic procedures including hip fracture fixation. Wire navigation plays a crucial role in the success of fixation of intertrochanteric hip fractures with a plate and screw fixation device such as a Dynamic Hip Screw (DHS) or Telescoping Screw Plate (TSP).…”