Standardized Assessment of Simulation Based ECMO Educational Courses: A Pilot Study

Abstract

Background: The Extracorporeal Life Support Organization (ELSO) education taskforce (ELSOed) recently identified seven extracorporeal membrane oxygenation (ECMO) educational domains that would benefit from international collaborative efforts among which standardized assessments of ECMO courses was prioritized. We aimed to develop a standardized online assessment tool to evaluate the effectiveness of a comprehensive didactic and simulation-based ECMO course on participants’ gain in confidence, knowledge, and simulation-based skills.Methods: We performed a prospective multicenter observational study of five US adult ECMO courses. Standardized online assessment forms were developed and administered before and after courses, covering demographics, self-assessment regarding ECMO management, and knowledge exam (15 simple-recall multiple choice questions), while psychomotor skill assessment was performed during the course (time to complete pre-specified critical actions during simulation scenarios). Self-assessment covered cognitive, behavioral, and technical aspects of ECMO care.Results: Out of 211 participants, 107 completed both pre- and post-course forms (97 completed both pre and post-course knowledge forms). Physician-intensivists were the largest group (53%) and the majority practiced at academic hospitals (51%) and had less than 1-year of ECMO experience (50%). Post-course, participants reported significant increases in confidence across all domains (cognitive, technical, and behavioral; p<0.0001 CI:1.2-1.5, p<0.0001 CI:2.2-2.6, and p=0.002 CI:1.7-2.1, respectively) as well as an increase in knowledge scores (p<0.001, CI: 1.4-2.5). These findings were consistent for all specialties and prior ECMO experience. There were also significant reductions in the times to critical actions in 3 of the 4 scored simulation scenarios.Conclusions: We successfully developed and tested a comprehensive standardized ECMO course assessment tool, demonstrating participants’ self-reported benefit as well as improvement in both knowledge and psychomotor skill acquisition . Standardized course evaluation is feasible and potentially provides important information to improve ECMO courses. Future steps include national implementation, addition of questions targeting clinical decision making to further assess knowledge gain, and multi-language translation for implementation at international courses.