Teamwork evaluation during emergency medicine residents’ high-fidelity simulation

Abstract

BackgroundTeamwork training has been included in several emergency medicine (EM) curricula; the aim of this study was to compare different scales’ performance in teamwork evaluation during simulation for EM residents.MethodsIn the period October 2013–June 2014, we performed bimonthly high-fidelity simulation sessions, with novice (I–III year, group 1 (G1)) and senior (IV–V year, group 2 (G2)) EM residents; scenarios were designed to simulate management of critical patients. Videos were assessed by three independent raters with the following scales: Emergency Team Dynamics (ETD), Clinical Teamwork Scale (CTS) and Team Emergency Assessment Measure (TEAM). In the period March–June, after each scenario, participants completed the CTS and ETD.ResultsThe analysis based on 18 sessions showed good internal consistency and good to fair inter-rater reliability for the three scales (TEAM, CTS, ETD: Cronbach's α 0.954, 0.954, 0.921; Intraclass Correlation Coefficients (ICC), 0.921, 0.917, 0.608). Single CTS items achieved highly significant ICC results, with 12 of the total 13 comparisons achieving ICC results ≥0.70; a similar result was confirmed for 4 of the total 11 TEAM items and 1 of the 8 total ETD items. Spearman's r was 0.585 between ETD and CTS, 0.694 between ETD and TEAM, and 0.634 between TEAM and CTS (scales converted to percentages, all p<0.0001). Participants gave themselves a better evaluation compared with external raters (CTS: 101±9 vs 90±9; ETD: 25±3 vs 20±5, all p<0.0001).ConclusionsAll examined scales demonstrated good internal consistency, with a slightly better inter-rater reliability for CTS compared with the other tools.

Review article: Use of the Team Emergency Assessment Measure in the rating of emergency teams' non‐technical skills: A mapping review

“…These comparisons verify that NTS teamwork performance can be enhanced by practice. We also noted that longitudinal studies that involved recurring medical emergencies demonstrated performance improvement trends in professional disciplines 20,23 and significant NTS improvements in medical resident teams 11,25 …”

“…We also noted that longitudinal studies that involved recurring medical emergencies demonstrated performance improvement trends in professional disciplines 20,23 and significant NTS improvements in medical resident teams. 11,25 Training impact, evaluated using Kirkpatrick Model…”

The Effects of Leadership Curricula With and Without Implicit Bias Training on Graduate Medical Education: A Multicenter Randomized Trial

“…The primary outcome was the leadership performance score on the Clinical Teamwork Scale (CTS), which is a validated tool commonly used in simulation, education, and clinical arenas. 16–22 The leadership performance score has a range of 0–10, with 0 being “unacceptable,” 1–3 being “poor,” 4–6 being “average,” 7–9 being “good,” and 10 being “perfect” leadership. We also conducted a detailed leadership evaluation (DLE), which was adapted for this study from the previously validated Leadership Behavior Description Questionnaire (LBDQ) and included 9 specific leadership components scored on a 5-point Likert-type scale.…”

SC15 In-situ simulation in the emergency department: our experience, development and outcome

“…The efficacy, feasibility and acceptability of in-situ simulation is well known, 1,2 as is its effectiveness at reviewing patient safety issues, 3 improving confidence, clinical and non-technical skills 4 and fostering teamwork. 5 Each simulation runs in real time utilising standard clinical staff and environment. Human simulated or high-fidelity simulated patients are treated like any other patient with staff expected to conduct bedside tests and interventions and prescribe and administer drugs as usual.…”