Narrative Performance Level Assignments at Initial Entrustment and Graduation: Integrating EPAs and Milestones to Improve Learner Assessment

Abstract

Purpose To determine which narrative performance level for each general pediatrics entrustable professional activity (EPA) reflects the minimum level clinical competency committees (CCCs) felt should be associated with graduation as well as initial entrustment and compare expected narrative performance levels (ENPLs) for each EPA with actual narrative performance levels (ANPLs) assigned to residents at initial entrustment. Method A series of 5 narratives, corresponding to the 5 milestone performance levels, were developed for each of the 17 general pediatrics EPAs. In academic year (AY) 2015–2016, the CCCs at 22 Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network member sites reported ENPLs for initial entrustment and at time of graduation. From AYs 2015–2016 to 2017–2018, programs reported ANPLs for initial entrustment decisions. ENPLs and ANPLs were compared using a logistic mixed effects model. Results ENPLs for graduation and entrustment were most often level 3 (competent) followed by level 4 (proficient). For 8 EPAs, the ENPLs for graduation and entrustment were the same. For the remaining 9, some programs would entrust residents before graduation or graduate them before entrusting them. There were 4,266 supervision level reports for initial entrustment for which an ANPL was provided. ANPLs that were lower than the ENPLs were significantly more likely to be assigned to the medical home–well child (OR = 0.39; 95% CI: 0.26–0.57), transition to adult care (OR = 0.43; 95% CI: 0.19–0.95), behavioral or mental health (OR = 0.36; 95% CI: 0.18–0.71), make referrals (OR = 0.31; 95% CI: 0.17–0.55), lead a team (OR = 0.34; 95% CI: 0.22–0.52), and handovers (OR = 0.18; 95% CI: 0.09–0.36) EPAs. Conclusions CCCs reported lower ENPLs for graduation than for entrustment for 5 EPAs, possibly indicating curricular gaps that milestones and EPAs could help identify.

Making prospective entrustment decisions: Knowing limits, seeking help and defaulting

“…Interviews were conducted with 12 individuals representing paediatric residency CCCs, eight individuals representing UME entrustment committees (representing psychiatry, internal medicine, emergency medicine, family medicine and paediatrics) and three individuals representing internal medicine residency CCCs for theoretical sampling purposes described below. Participants were drawn from paediatric residency CCC members engaged in a previous national study of EPA‐based assessment that was conducted by members of our team 26,27 . This pool of potential participants ensured those recruited had experience with EPAs and entrustment decision making.…”

“…We conducted semi-structured interviews, ranging 17 based assessment that was conducted by members of our team. 26,27 This pool of potential participants ensured those recruited had experience with EPAs and entrustment decision making. Some of these participants spoke to their UME experiences as well.…”

Competency-based anesthesiology teaching: comparison of programs in Brazil, Canada and the United States

“…There are currently, several medical residency programs in the US, not only in Anesthesiology, studying and trying to implement EPA as the evaluation method of physicians undergoing specialization, such as Pathology, Radiology and Pediatriacs. 8 , 21 , 22 Breckwoldt J et al describe EPA as the progress of the competency-based curriculum concept in medical education, presenting a more holistic vision of physicians being trained. 23 One core evaluation point in EPA is professionalism of physician trainees in regard to patient, family, society and work colleagues, beyond practical skills for procedures.…”

Twelve tips for essential aspects in the elaboration of Global Medical Curricula in a context of Medical Competencies, EPAs, and Milestones

“…As of 2016, publications increased and began to indicate different ways to assess more safely, and the results are Pedra Dias E, Duarte Ferreira M MedEdPublish https://doi.org/10.15694/mep.2021.000039.1 promising (Schumacher et al, 2020;Henry and West, 2019;Chan et al, 2020). However, the high and low performance approach are not being presented, and the application of in the workplace based assessment (WPBA) does not seem to ensure the detection and management of low performance (Brightwell and Grant, 2013).…”

“…Several recent studies emphasize the importance of integrating CBME and EPAs concepts, and their impact on learning, teaching and assessment (Sohrmann et al, 2020;Schumacher et al, 2020).…”