High-fidelity simulation is associated with good discriminability in emergency medicine residents’ in-training examinations

Abstract

In-training examinations (ITEs), arranged during residency training, evaluate the residents’ performances periodically. There is limited literature focusing on the effectiveness of resident ITEs in the format of simulation-based examinations, as compared to traditional oral or written tests. Our primary objective is to investigate the effectiveness and discriminative ability of high-fidelity simulation compared with other measurement formats in emergency medicine (EM) residency training program. This is a retrospective cohort study. During the 5-year study period, 8 ITEs were administered to 68 EM residents, and 253 ITE measurements were collected. Different ITE scores were calculated and presented as mean and standard deviation. The ITEs were categorized into written, oral, or high-fidelity simulation test forms. Discrimination of ITE scores between different training years of residency was examined using a one-way analysis of variance test. The high-fidelity simulation scores correlated to the progression of EM training, and residents in their fourth training year (R4) had the highest scores consistently, followed by R3, R2, and then R1. The oral test scores had similar results but not as consistent as the high-fidelity simulation tests. The written test scores distribution failed to discriminate the residents’ seniority. The high-fidelity simulation test had the best discriminative ability and better correlation between different EM residency training years comparing to other forms. High-fidelity simulation tests had the good discriminative ability and were well correlated to the EM training year. We suggest high-fidelity simulation should be a part of ITE in training programs associated with critical or emergency patient cares.

Role of Simulation in Italian Emergency Medicine training programs: data from a national survey

“…A retrospective cohort study examined differential outcomes between oral and written High delity simulation test forms for EM residents. They supported the idea that simulation should be an integral part of the examination modalities, especially for residents dealing with critical areas and emergency patient care (23).…”

Multimodal In-training Examination in an Emergency Medicine Residency Training Program: A Longitudinal Observational Study

“…Traditional written and oral examinations used in ITEs may not accurately assess resident competency on their own. Simulation-based examinations and HFS have been demonstrated to accurately assess resident competencies across multiple domains ( 8 , 28 , 29 ). Integrating HFS into ITEs can improve the accuracy and efficiency of competency assessments and make them more comprehensive.…”

“…However, oral and written test performance may not directly reflect residents' clinical experience and multitasking abilities, especially relevant for clinical competency in a busy and rushed clinical environment, such as the emergency department (E.D.) (6)(7)(8).…”