Impact of Simulation-based Mastery Learning on Resident Skill Managing Mechanical Ventilators

Abstract

Background: Caring for patients requiring mechanical ventilation is complex, and residents may lack adequate skill for managing these patients. Simulation-based mastery learning (SBML) is an educational model that trains clinicians to a high standard and can reduce complications. The mastery learning model has not been applied to ventilator management. Objective: The purpose of this study was to determine whether SBML, as compared with traditional training, is an effective strategy for teaching residents the skills necessary to manage patients requiring mechanical ventilation. Methods: We developed an SBML curriculum and a 47-item skills checklist to test ventilator management for patients with normal, restricted, and obstructed lung physiology. A minimum passing standard (MPS) on the checklist was set using the Mastery Angoff method. Residents rotating through the medical intensive care unit in Academic Year 2017–2018 were assigned to SBML or traditional training based on their medical intensive care unit team. The SBML group was pretested on a ventilator simulator using the skills checklist. They then received a 1.5-hour session (45 min didactic and 45 min deliberate practice on the simulator with feedback). At rotation completion, they were posttested on the simulator using the checklist until the MPS was met. Both SBML-trained and traditionally trained groups received teaching during daily bedside rounds and twice weekly didactic lectures. At rotation completion, traditionally trained residents were tested using the same skills checklist on the simulator. We compared pretest and posttest performance among SBML-trained residents and end of the rotation test performances between the SBML-trained and traditionally trained residents. Results: The MPS was set at 87% on the checklist. Fifty-seven residents were assigned to the SBML-trained group and 49 were assigned to the traditionally trained group. Mean checklist scores for SBML-trained residents improved from 51.4% (standard deviation [SD] = 17.5%) at pretest to 86.1% (SD = 7.6%) at initial posttest and 92.5% (SD = 3.7%) at final (mastery) posttest (both P < 0.001). Forty-two percent of residents required more than one attempt at the posttest to meet or exceed the MPS. At rotation completion, the traditionally trained residents had a mean test score of 60.9% (SD = 13.3%). Conclusion: SBML is an effective strategy to train residents on mechanical ventilator management. An SBML curriculum may augment traditional training methods to further equip residents to safely manage ventilated patients.

Simulation-based Assessment to Measure Proficiency in Mechanical Ventilation among Residents

“…We chose to apply the SBA at the beginning of rotation because one of our aims was to identify the most common knowledge gaps, to address them during the rotation. Our finding is similar to that of a recent study ( 30 ) in which investigators used simulation to assess baseline skills, as a teaching tool in the form of deliberate practice, and to assess end-of-rotation skills. They found that at the beginning of ICU rotation the median score was 52%, and it increased to 86% at the end of rotation.…”

“…Although written assessments such as the MCE can be used to assess learning, SBAs require that learners apply knowledge when they operate a mechanical ventilator in a scenario that mimics real clinical scenarios, providing a more complete assessment of competence. Previous studies showed that simulation in MV can provide lasting knowledge and prepare residents to safely manage ventilated patients ( 30 , 39 , 40 ), and SBAs have been successfully used to identify learning gaps and provide remediation ( 30 ).…”

“…Moreover, SBA may not be easily generalizable to other settings, as it depends on the availability of the simulator and a ventilator and requires time availability from both the learner and raters. However, simulation in MV has been increasingly used as it becomes part of the teaching tools in critical care ( 30 , 40 ). Fourth, we did not formally survey participants regarding their experience and perceived advantages and disadvantages of the SBA, compared with traditional evaluations; in addition, we did not provide feedback to participants because the assessment was not validated yet, and we wanted to prevent residents from discussing the scenarios with one another on the day of the evaluation.…”

Assessing Mechanical Ventilation Management Skills: More Tools for the Toolbox

“…Simply relying on experiential learning could lead to patient harm if learners are called upon to provide MV management despite a paucity in background knowledge and training. Prior studies of MV education have shown simulation-based practical examinations to be superior to traditional assessments ( 10 , 13 , 14 ). In this study, the authors make an important contribution by developing an MV curriculum in an objective structured clinical examination format that incorporates core competencies required to care for critically ill patients receiving MV ( 15 ).…”

“…Therefore, the participants’ learned knowledge and skill during their ICU rotation were not assessed by the SBA. Use of a pretest paired with deliberate practice and feedback before a posttest has been shown to successfully get learners to a mastery standard for MV management and other common ICU procedures ( 10 , 11 ). The learners in this study were not provided feedback on their SBA, nor were they allotted the opportunity for deliberate practice.…”