Measuring situation awareness and team effectiveness in pediatric acute care by using the situation global assessment technique

Abstract

Situation awareness (SA) is an important human factor and necessary for effective teamwork and patient safety. Human patient simulation (HPS) with video feedback allows for a safe environment where health care professionals can develop both technical and teamwork skills. It is, however, very difficult to observe and measure SA directly. The Situation Global Assessment Technique (SAGAT) was developed by Endsley to measure SA during real-time simulation. Our objective was to measure SA among team members during simulation of acute pediatric care scenarios on the medical ward and its relationship with team effectiveness. Twenty-four pediatric teams, consisting of two nurses, one resident, and one consultant, participated in three acute care scenarios, using high-fidelity simulation. Individual SAGAT scores contained shared and complimentary knowledge questions on different levels of SA. Within each scenario, two “freezes” were incorporated to assess SA of each team members’ clinical assessment and decision-making. SA overlap within the team (team SA) was computed and compared to indicators of team effectiveness (time to goal achievement, consensus on primary problem, diagnosis, task prioritization, leadership, and teamwork satisfaction). In 13 scenarios (18%), the team failed to reach the primary goals within the prescribed time of 1200 s. There was no significant difference in failure of goal completion between the scripted scenarios; however, there was a significant difference between scenario 3 and the other scenarios in time to goal completion. In all three scenarios, SA overlap level 2 (consensus on primary problem during the first freeze and consensus on diagnosis during the second freeze) leads to significantly faster achievement of the predefined goals. There was a strong relationship between team SA on the primary problem and diagnosis and team SA on task prioritization. Consensus on leadership within the team was low. Teamwork satisfaction was more influenced by knowledge about the importance of the assigned task than outcome of the scenario. Conclusion : The use of SAGAT enables us to measure SA of team members during real-time simulation of acute care scenarios. Although there is no direct connection between team SA and goal achievement, SAGAT provides insight in differences in SA among team members, and the process of shared mental model formation. By measuring SA, issues that may improve team effectiveness (prioritizing tasks, enhancing shared mental models, and providing leadership) can be trained and assessed during medical team simulation, enhancing teamwork in health care settings. What is known? • Teamwork skills such as communication, leadership, and situational awareness have become increasingly recognized as essential for good performance in pediatric resuscitation. However, the assessment of pediatric team performance in these clinical situations has been traditionally difficult. ...

Relationship between Executive Functions, Mindfulness, Stress, and Performance in Pediatric Emergency Simulations

“…In our study, we observed a relationship between SA and the students' technical skills, including achieving the goals of the scenario. In contrast to our results, similar pediatric simulations have shown no relationship between SA as assessed by the SAGAT method and achievement of the goal of the scenario [24]. In another study, this skill correlated with the team's clinical performance but did not correlate with the team's perception of shared understanding, team leader effectiveness, or team experience-similar to our results [25].…”

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

“…Although the exact methods for assessing SA in teams varied among these studies, a few general statements can be made. Shared SA as measured by SAGAT was shown to be predictive of overall team performance (Bonney, Davis-Sramek, & Cadotte, 2016;Coolen, Draaisma, & Loeffen, 2019;Rosenman et al, 2018), as well as when it was measured by SPAM (Cooke, Kiekel, & Helm, 2001). Combining SA scores into an overall Team SA score generated mixed results, with five studies showing it to be predictive of overall team performance (Cooke et al, 2001;Crozier et al, 2015;Gardner, Kosemund, & Martinez, 2017;Parush et al, 2017;Prince, Ellis, Brannick, & Salas, 2007), and three that did not (Brooks, Switzer, & Gugerty, 2003;P.…”

Non-technical skills of medical students in the course of pediatric emergency simulations are associated with their mindfulness: an observational cohort study

“…greater capacity to regularly reassess patient's condition and situation, resulted in a higher score on the technical skills scale. In similar conditions of pediatric simulations no direct relationship was found between situation awareness and goal achievement [14]. In that study, consensus on the primary problem (shared mental model) led to faster achievement of predefined goals.…”