Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons

Young, John Q.; Hasser, Caitlin; Hung, Erick; Kusz, Martin; O’Sullivan, Patricia; Stewart, Colin J.R.; Weiss, Andrea; Williams, Nancy I.

doi:10.1097/acm.0000000000002058

Cited by 42 publications

(41 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Most adopted the Delphi approach or a modified version of it, and some sought expert discussions . A few studies shortlisted important EPAs through online surveys or focus groups . EPAs were mostly mapped to their respective competencies or curricula's milestones.…”

Section: Resultsmentioning

confidence: 99%

Entrustable professional activities in health care education: a scoping review

et al. 2019

View full text Add to dashboard Cite

CONTEXT The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care.RESULTS Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight subthemes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialtyspecific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs).CONCLUSIONS Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidencebased research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.

show abstract

Section: Resultsmentioning

confidence: 99%

Entrustable professional activities in health care education: a scoping review

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Methods used to develop EPAs included Delphi groups, literature reviews and focus groups. [47][48][49][50][51][52][54][55][56][57][58][60][61][62][63] Although no apparent 'best' method could be determined, this systematic review is nonetheless an important overview of the various processes that can contribute to EPA development in the future. The use of standardised guidelines may also be a means of improving the development process.…”

Section: Methodological Qualitymentioning

confidence: 99%

A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018

et al. 2019

View full text Add to dashboard Cite

Purpose This review aimed to synthesise some of the extant work on the use of entrustable professional activities (EPAs) for postgraduate physicians, to assess the quality of the work and provide direction for future research and practice. Method Systematic searches were conducted within five electronic databases (Medline, Scopus, Web of Science, PsycINFO and CINAHL) in September 2018. Reference lists, Google Scholar and Google were also searched. Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD). Results In total, 49 studies were included, classified as Development of EPAs (n = 37; 76% of total included), Implementation and/or assessment of EPAs (n = 10; 20%), or both (n = 2; 4%). EPAs were described for numerous specialties, including internal medicine (n = 14; 36%), paediatrics (n = 8; 21%) and psychiatry (n = 4; 10%). Of the development studies, 92% utilised more than one method to generate EPAs. The two most commonly used methods were developing initial EPAs in a working group, (n = 27; 69%) and revising through deliberation (n = 21; 54%). Development papers were of variable quality (mean QATSDD score = 20, range 6–41). Implementation and assessment studies utilised methods that included observing trainee performance (n = 6; 50%) and enrolling trainees in competency‐based curricula, which included EPAs (n = 4; 33%). The methodological quality of these implementation studies varied (mean QATSDD score = 19.5, range = 6–32). Conclusions This review highlighted a need for: (i) consideration of best practice guidelines for EPA development; (ii) focus on the methodological quality of research on EPA development and of EPAs, and (iii) further work investigating the implementation of EPAs in the curriculum.

show abstract

“…The activity that the P-SCO assesses is considered a primary activity of a psychiatrist and has recently been identified as a core end-of-training entrustable professional activity. 28…”

Section: Designmentioning

confidence: 99%

Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation

Young

Sugarman

Holmboe

et al. 2019

Journal of Graduate Medical Education

Self Cite

View full text Add to dashboard Cite

Background While prior research has focused on the validity of quantitative ratings generated by direct observation tools, much less is known about the written comments. Objective This study examines the quality of written comments and their relationship with checklist scores generated by a direct observation tool, the Psychopharmacotherapy-Structured Clinical Observation (P-SCO). Methods From 2008 to 2012, faculty in a postgraduate year 3 psychiatry outpatient clinic completed 601 P-SCOs. Twenty-five percent were randomly selected from each year; the sample included 8 faculty and 57 residents. To assess quality, comments were coded for valence (reinforcing or corrective), behavioral specificity, and content. To assess the relationship between comments and scores, the authors calculated the correlation between comment and checklist score valence and examined the degree to which comments and checklist scores addressed the same content. Results Ninety-one percent of the comments were behaviorally specific. Sixty percent were reinforcing, and 40% were corrective. Eight themes were identified, including 2 constructs not adequately represented by the checklist. Comment and checklist score valence was moderately correlated (Spearman's rho = 0.57, P < .001). Sixty-seven percent of high and low checklist scores were associated with a comment of the same valence and content. Only 50% of overall comments were associated with a checklist score of the same valence and content. Conclusions A direct observation tool such as the P-SCO can generate high-quality written comments. Narrative comments both explain checklist scores and convey unique content. Thematic coding of comments can improve the content validity of a checklist.

show abstract

Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons

Cited by 42 publications

References 61 publications

Entrustable professional activities in health care education: a scoping review

Entrustable professional activities in health care education: a scoping review

A systematic review of 7 years of research on entrustable professional activities in graduate medical education, 2011–2018

Advancing Our Understanding of Narrative Comments Generated by Direct Observation Tools: Lessons From the Psychopharmacotherapy-Structured Clinical Observation

Contact Info

Product

Resources

About