A unique approach to faculty development using an Observed Structured Teaching Encounter (OSTE)

Abstract

We have challenges with poor patient satisfaction scores (Hospital Consumer Assessment of Healthcare Providers and Systems [HCAHPS]) and internal medicine resident (IMR) evaluations of voluntary attending physicians. Using an Observed Structured Teaching Encounter (OSTE), we designed a faculty development project that focused on attendings’ teaching and feedback skills. To assess attending communication with interns and improve attending teaching and feedback skills. All IM attendings on the Long Island Jewish Forest Hills (LIJFH) Emergency Department (ED) call schedule participated. OSTE simulation sessions included two clinical scenarios, standardized patients (SPs), fourth-year medical students trained as ‘interns,’ OSTE checklists, and debriefing. We analyzed ‘intern’ ratings of communication with attendings and attending self-assessment during the OSTE, and attending HCAHPS scores and IMR evaluations of attendings pre- and post-OSTE. Twenty-nine of 29 attendings completed the OSTE. Although an increase was demonstrated pre- to post- for ‘intern’ OSTE ratings of attendings and LIJFH attending self-assessment ratings, there was no statistically significant difference. Mean HCAHPS scores and resident evaluations of attendings also increased from pre- (22% and 3.59) to post-OSTE (30% and 3.87) but did not reach statistical significance. A statistically significant difference for both cases was demonstrated when comparing mean attending self-assessment ratings with ‘intern’ evaluation of attendings. Attending teaching/feedback skills improved between cases, attending self-ratings were higher than ‘intern’ ratings of attendings. HCAHPS and IMR evaluations of attendings improved post-OSTE. Regular intervention utilizing an OSTE may provide a sustained benefit for enhancing attendings’ skills, patient satisfaction, and resident training.

A 360-degree assessment of teaching effectiveness using a structured-videorecorded observed teaching exercise for faculty development

“…We think that the use of video is a major benefit to the program and actually yields different results than other programs that have compared self-assessment to learner assessment. For example, a recent study of an Observed Structured Teaching Encounter aimed at improving attending physician communication skills, attending physicians actually rated their skills higher than learners, contrary to what we observed [ 24 ] – we think that a major difference is the use of video for self-assessment. By providing the variety of perspectives, self, peer and trainee, the VOTE method offers a balanced view of faculty teaching.…”

A Qualitative Preceptor Development Needs Assessment to Inform Program Design and Effectiveness

“…OSTEs-similar to Objective Structured Clinical Examinations (OSCEs)-use a standardized encounter where a preceptor interacting with a learner is observed, evaluated, and offered feedback to improve teaching skills. 33 Available evidence on OSTEs in medical A J P E A c c e p t e d D r a f t education is generally favorable, but more research is needed on the design, rating instruments, and outcomes. [33][34][35] Collaboration with faculty who organize OSCEs may support OSTE design, and clear frameworks of competencies to be assessed should be considered for evaluation.…”

“…33 Available evidence on OSTEs in medical A J P E A c c e p t e d D r a f t education is generally favorable, but more research is needed on the design, rating instruments, and outcomes. [33][34][35] Collaboration with faculty who organize OSCEs may support OSTE design, and clear frameworks of competencies to be assessed should be considered for evaluation. 32,36 Literature on peer assessment of experiential teaching suggests that peer observation with formative feedback may be beneficial to preceptors.…”

Review of Cognitive Biases in ACGME Milestones Training Assessments in Post-graduate Medical Education Programs

“…One recent study even proposed a six-point actionable framework for integrating implicit bias recognition and management into health professions' education [14]. Other approaches to deal with bias have included an observed structured teaching encounter [15][16].…”