Redesigning evaluation forms improves didactic conference assessment

Nelson, Adin; Traba, Christin

doi:10.1111/medu.13847

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is very little published on the subject of generating feedback for residency didactics; 6 most examples of such feedback were collected specifically to measure a new type of educational session. 7 When discussed, "survey fatigue" has been cited as an impediment.…”

Section: Backg Rou N Dmentioning

confidence: 99%

Educational autopsy: An innovative structured debrief for residency didactic teaching

Griffith

Brown

Haas

et al. 2021

AEM Education and Training

View full text Add to dashboard Cite

Background: Educational autopsy (EA) is an innovative technique designed to improve the quality of feedback provided to conference presenters. In response to survey fatigue and suboptimal feedback from online evaluations, this postlecture group debrief was adapted to emergency medicine residency didactics, with a goal of collecting timely, specific, and balanced feedback for presenters. Other aims include encouraging participants to think critically about educational methods and providing presenters with formal feedback for a portfolio or promotion packet. It was hypothesized that EA provides more specific and actionable feedback than traditional online evaluations deployed individually to conference attendees. Methods:The authors analyzed 4 months of evaluations pre-and postimplementation of EA. Rate of completion, presence of comments, and types of comments were compared. Comments were coded as specific, nonspecific, and unrelated/unclear. Specific comments were further categorized as about audiovisual presentation design, speaker presentation style, and educational methods of the session.Results: A total of 46 of 65 (71%) preimplementation presentations eligible for evaluation received comments through traditional online evaluations. A total of 44 of 75 (59%) eligible postimplementation presentations generated comments via EA. Among presentations that received comments, none received nonspecific comments via EA, compared to 46% of lectures through traditional evaluations. EA generated specific comments for more presentations regarding presentation design (91% vs. 63%), presentation style (66% vs. 24%), and educational methods (48% vs. 28%). EA produced no unclear comments; traditional evaluations resulted in unclear comments for 15% of lectures.Conclusions: EA generated more specific feedback for residency conference presenters, although there were a number of sessions not evaluated by EA. Although this limited analysis suggested that EA produced higher-quality presenter feedback, it also showed a drop-off in the proportion of didactic sessions that received narrative feedback. NEED FOR INNOVATIONFeedback is essential for the development of health professions educators. Just as the master clinician does not achieve mastery overnight, the expert educator owes their expertise to years of practice and formative feedback.The Accreditation Council for Graduate Medical Education (ACGME) requires structured didactics as part of the emergency medicine (EM) residency training experience. While there are no formal requirements for didactic feedback, the ACGME does require "ongoing feedback that can be used by residents to improve their learning in the context of provision of patient care or other educational opportunities." 1 Historically, our residency conference audience received an online feedback form for each of several weekly presentations (Data Supplement S1, Table S1, available as supporting information in the

show abstract

Section: Backg Rou N Dmentioning

confidence: 99%

Educational autopsy: An innovative structured debrief for residency didactic teaching

Griffith

Brown

Haas

et al. 2021

AEM Education and Training

View full text Add to dashboard Cite

show abstract

“…Given the unique instructional setting of graduate medical education, with its advanced learners and complex knowledge to be acquired, a tool should be developed specifically for use in this setting and aimed at the goals of EM didactic instruction. Thoughtful design of didactic evaluation tools has been shown to enhance the quality and quantity of feedback obtained 13 . The development of a didactic lecture evaluation tool rooted in cognitive load theory can generate valuable feedback for lecturers of all levels, facilitate assessment and teaching of didactic skills, and enhance learning outcomes by encouraging effective presentation of content.…”

Section: Introductionmentioning

confidence: 99%

“…Thoughtful design of didactic evaluation tools has been shown to enhance the quality and quantity of feedback obtained. 13 The development of a didactic lecture evaluation tool rooted in cognitive load theory can generate valuable feedback for lecturers of all levels, facilitate assessment and teaching of didactic skills, and enhance learning outcomes by encouraging effective presentation of content. The objective of this study was to develop an evaluation tool, rooted in cognitive load theory, to assess the quality of didactic lectures in graduate medical education including content, presentation, and delivery.…”

Section: Introductionmentioning

confidence: 99%

Development of a lecture evaluation tool rooted in cognitive load theory: A modified Delphi study

Jordan

Haas

Hickam

et al. 2023

AEM Education and Training

View full text Add to dashboard Cite

Background: Didactics play a key role in medical education. There is no standardized didactic evaluation tool to assess quality and provide feedback to instructors.Cognitive load theory provides a framework for lecture evaluations. We sought to develop an evaluation tool, rooted in cognitive load theory, to assess quality of didactic lectures. Methods:We used a modified Delphi method to achieve expert consensus for items in a lecture evaluation tool. Nine emergency medicine educators with expertise in cognitive load participated in three modified Delphi rounds. In the first two rounds, experts rated the importance of including each item in the evaluation rubric on a 1 to 9 Likert scale with 1 labeled as "not at all important" and 9 labeled as "extremely important." In the third round, experts were asked to make a binary choice of whether the item should be included in the final evaluation tool. In each round, the experts were invited to provide written comments, edits, and suggested additional items.Modifications were made between rounds based on item scores and expert feedback.We calculated descriptive statistics for item scores. Results:We completed three Delphi rounds, each with 100% response rate. After Round 1, we removed one item, made major changes to two items, made minor wording changes to nine items, and modified the scale of one item. Following Round 2, we eliminated three items, made major wording changes to one item, and made minor wording changes to one item. After the third round, we made minor wording changes to two items. We also reordered and categorized items for ease of use. The final evaluation tool consisted of nine items. Conclusions:We developed a lecture assessment tool rooted in cognitive load theory specific to medical education. This tool can be applied to assess quality of instruction and provide important feedback to speakers.

show abstract

Redesigning evaluation forms improves didactic conference assessment

Cited by 2 publications

References 0 publications

Educational autopsy: An innovative structured debrief for residency didactic teaching

Educational autopsy: An innovative structured debrief for residency didactic teaching

Development of a lecture evaluation tool rooted in cognitive load theory: A modified Delphi study

Contact Info

Product

Resources

About