Emergency Medicine Residents’ Self‐assessments Play a Critical Role When Receiving Feedback

Abstract

Objectives: Emergency medicine (EM) faculty often aim to improve resident performance by enhancing the quality and delivery of feedback. The acceptance and integration of external feedback is influenced by multiple factors. However, it is interpreted through the "lens" of the learner's own self-assessment. Ideally, following an educational activity with feedback, a learner should be able to generate and act upon specific learning goals to improve performance. Examining the source of generated learning goals, whether from one's self-assessment or from external feedback, might shed light on the factors that lead to improvement and guide educational initiatives. Using a standard oral board scenario, the objective of this study was to determine the effects that residents' self-assessment and specific feedback from faculty have on not only the generation of learning goals but also the execution of these goals for performance improvement.Methods: In this cross-sectional educational study at four academic programs, 72 senior EM residents participated in a standardized oral board scenario. Following the scenario, residents completed a self-assessment form. Next, examiners used a standardized checklist to provide both positive and negative feedback. Subsequently, residents were asked to generate "SMART" learning goals (specific, measurable, attainable, realistic, and time-bound). The investigators categorized the learning goals as stemming from the residents' self-assessments, feedback, or both. Within 4 weeks, the residents were asked to recall their learning goals and describe any actions taken to achieve those goals. These were grouped into similar categories. Descriptive statistics were used to summarize the data.Results: A total of 226 learning goals were initially generated (mean AE SD = 3.1 AE 1.3 per resident). Forty-seven percent of the learning goals were generated by the residents' self-assessments only, while 27% were generated by the feedback given alone. Residents who performed poorly on the case incorporated feedback more often than high performers when generating learning goals. Follow-up data collection showed that 62 residents recalled 89 learning goals, of which 52 were acted upon. On followup, the numbers of learning goals from self-assessment and feedback were equal (25% each, 13 of 52), while the greatest number of reportedly executed learning goals came from self-assessments and feedback in agreement (40%). Conclusions:Following feedback on an oral board scenario, residents generated the majority of their learning goals from their own self-assessments. Conversely, at the follow-up period, they recalled an increased number of learning goals stemming from feedback, while the largest proportion of learning goals acted upon stemmed from both feedback and self-assessments in

Supporting Learners' Agentic Engagement With Feedback: A Systematic Review and a Taxonomy of Recipience Processes

“…Quality, in this case, should be more important than quantity, even though in at least one study no substantial relation was found between medical residents' ratings of the quality of their feedback and the extent to which they implemented it in their learning goals (Bounds et al, 2013). This particular finding notwithstanding, Nicol and Macfarlane-Dick (2006) theorized that high-quality feedback-through clarifying what good performance entails and providing opportunities to close the gap between current and desired levels of performance-influences learners' ability to self-regulate, which we noted earlier as a crucial determinant of feedback use.…”

“…In contrast, lower achievers reported that they tended to simply read the feedback several times and did not typically use it for self-assessment or to plan for future work. A study by Bounds et al (2013), however, found conflicting results. In that study, medical students were prompted to generate written learning goals after receiving feedback.…”

“…One reason is that the effectiveness of feedback is typically assessed by measuring changes in learners' grades or satisfaction, rather than changes in their behavior (Bounds et al, 2013). This is an important issue because the relation between feedback and learners' achievement is necessarily mediated by the more proximal factor of their engagement with that feedback.…”

“…There is increasing consensus that a critical determinant of feedback effectiveness is the quality of learners' engagement with, and use of, the feedback they receive. However, studies investigating this engagement are underrepresented in academic research (Bounds et al, 2013), which leaves a "blind spot" in our understanding (Burke, 2009). With this blind spot in mind, the present work sets out to systematically map the research literature concerning learners' proactive recipience of feedback.…”

Learner-Driven Synthesis of Assessment Data: Engaging and Motivating Residents in Their Milestone-Based Assessments

“…However, there is potential concern about whether self-assessment using milestones will help to inform resident progress or to facilitate self-directed learning skills. Research over the past decade has shown that selfassessment may encourage some residents to improve their awareness of deficiencies, create individualized learning plans, and bolster specific knowledge areas [4][5][6][7][8][9]. Unfortunately, self-assessment is fraught with inaccuracy and varies depending on the setting, content, and learner perspective [10][11][12].…”