The Role of Generic Competencies in the Entrustment of Professional Activities: A Nationwide Competency-Based Curriculum Assessed

Loon, Karsten A. van; Teunissen, Pim W.; Driessen, Erik; Scheele, Fedde

doi:10.4300/jgme-d-15-00321.1

Cited by 25 publications

(17 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Anchor-based, milestone end-of-shift ratings are commonly used to inform summative evaluation by educators. However, recent literature suggests that the goal of "entrustment" in medical education is not reached by the attainment of generic anchors (van Loon, Teunissen, Driessen, & Scheele, 2016). In fact, an overestimation of resident proficiency and an inability to discriminate between residents of different levels of training have been described when utilizing milestone-based end-of-shift evaluations (Dehon, Jones, Puskarich, Sandifer, & Sikes, 2015).…”

Section: Discussionmentioning

confidence: 99%

Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance

et al. 2018

View full text Add to dashboard Cite

Anchor-based, end-of-shift ratings are commonly used to conduct performance assessments of resident physicians. These performance evaluations often include narrative assessments, such as solicited or “free-text” commentary. Although narrative commentary can help to create a more detailed and specific assessment of performance, there are limited data describing the effects of narrative commentary on the global assessment process. This single-group, observational study examined the effect of narrative comments on global performance assessments. A subgroup of the clinical competency committee, blinded to resident identity, assigned a single, consensus-based performance score (1–6) to each resident based solely on end-of-shift milestone scores. De-identified narrative comments from end-of-shift evaluations were then included and the process was repeated. We compared milestone-only scores to milestone plus narrative commentary scores using a nonparametric sign test. During the study period, 953 end-of-shift evaluations were submitted on 41 residents. Of these, 535 evaluations included free-text narrative comments. In 17 of the 41 observations, performance scores changed after the addition of narrative comments. In two cases, scores decreased with the addition of free-text commentary. In 15 cases, scores increased. The frequency of net positive change was significant ( p = .0023). The addition of narrative commentary to anchor-based ratings significantly influenced the global performance assessment of Emergency Medicine residents by a committee of educators. Descriptive commentary collected at the end of shift may inform more meaningful appraisal of a resident’s progress in a milestone-based paradigm. The authors recommend clinical training programs collect unstructured narrative impressions of residents’ performance from supervising faculty.

show abstract

Section: Discussionmentioning

confidence: 99%

Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance

et al. 2018

View full text Add to dashboard Cite

show abstract

“…The same survey was performed three months after the implementation of the new system. To facilitate analysis, each level of agreement was assigned a numerical value (1)(2)(3)(4)(5). The results were aggregated, and were analyzed using t-tests, assuming unequal variances.…”

Section: Methodsmentioning

confidence: 99%

“…1 Previous research has shown that ad hoc entrustment decisions in the clinical setting are made by clinical supervisors primarily based on each resident's experience and exposure to a particular task, rather than generic cross-context competencies such as communication skills. 2 EPAs integrate competencies, which are not observable in isolation, into observable professional tasks, allowing direct assessment of the resident's work performance rather than attempting to infer competence based on underlying generic competencies.…”

Section: Introductionmentioning

confidence: 99%

Impact of an entrustable professional activities-based assessment system

Lord

Murchison

Lawrence

et al. 2018

Proc Obstet Gynecol

View full text Add to dashboard Cite

Background: Beginning in 2014, all Accreditation Counsel of Graduate Medical Education (ACGME) accredited residency programs were required to move to a Milestones-based system for biannual resident assessment. The resident assessment system for the Virginia Tech-Carilion Obstetrics and Gynecology (OB/GYN) residency program was redesigned to meet this requirement in July, 2014. The ACGME Milestones based assessment tool was identified on multiple faculty surveys as an area for improvement. To address this issue, an entrustable professional activities (EPA) based assessment system was designed and implemented for assessment of all OB/GYN rotations.

show abstract

“…We compared our label scheme with labels identified in earlier studies. 9,14 This led to a new structure, in which initial labels were combined into categories. Discussion with all authors (2 educational scientists and 2 faculty members) resulted in a final set of categories based on coded quotes.…”

Section: Bottom Linementioning

confidence: 99%

Formal Versus Informal Judgments: Faculty Experiences With Entrustment in Graduate Medical Education

Loon

Teunissen

Driessen

et al. 2018

Journal of Graduate Medical Education

Self Cite

View full text Add to dashboard Cite

Background Entrustment of residents has been formalized in many competency-based graduate medical education programs, but its relationship with informal decisions to entrust residents with clinical tasks is unclear. In addition, the effects of formal entrustment on training practice are still unknown.Objective Our objective was to learn from faculty members in training programs with extensive experience in formal entrustment how formal entrustment relates to informal entrustment decisions.Methods A questionnaire was e-mailed to all Dutch obstetrics and gynecology program directors to gather information on how faculty entrusts residents with clinical independence. We also interviewed faculty members to explore the relationship between formal entrustment and informal entrustment. Interviews were analyzed with conventional content analysis. ResultsOf 92 programs, 54 program directors completed the questionnaire (59% response rate). Results showed that formal entrustment was seen as valuable for generating formative feedback and giving insight into residents' progress in technical competencies. Interviewed faculty members (n ¼ 12) used both formal and informal entrustment to determine the level of resident independence. Faculty reported they tended to favor informal entrustment because it can be reconsidered. In contrast, formal entrustment was reported to feel like a fixed state.Conclusions In a graduate medical education program where formal entrustment has been used for more than a decade, faculty used a combination of formal and informal entrustment. Informal entrustment is key in deciding if a resident can work independently. Faculty members reported being unsure how to optimally use formal entrustment in practice next to their informal decisions.

show abstract

The Role of Generic Competencies in the Entrustment of Professional Activities: A Nationwide Competency-Based Curriculum Assessed

Abstract: Background Entrustable professional activities (EPAs) seek to translate essential physician competencies into clinical practice. Until now, it is not known whether EPA-based curricula offer enhanced assessment and feedback to trainees.

Cited by 25 publications

References 36 publications

Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance

Examining the Effects of Narrative Commentary on Evaluators’ Summative Assessments of Resident Performance

Impact of an entrustable professional activities-based assessment system

Formal Versus Informal Judgments: Faculty Experiences With Entrustment in Graduate Medical Education

Contact Info

Product

Resources

About