Exploring objective measures for assessing team performance in healthcare: an interview study

BACKGROUND Mixed Reality (MR) simulation training is emerging in paramedical education for practicing stress-intensive scenarios like mass casualty incidents in a safe and controlled environment. Current training platforms, however, lack real-time stress and human performance monitoring tools. OBJECTIVE The study aims to enhance mixed reality training for medical first responders through real-time evaluation of performance and stress levels, leveraging bio-signal monitoring and advanced analytics to allow instructors to tailor feedback and maintain optimal challenge and safety levels. METHODS The study includes a structured, multi-phase approach including initial requirement gathering (structured interviews and co-creation workshops), an online design survey, iterative prototype development and a field trial (including training observations and interviews). Data was collected from five end-user consortium members across Europe. Quantitative data from checklists were analyzed using frequencies and percentages to understand feature usage and event occurrences. Qualitative data from semi-structured interviews and co-creation workshops were transcribed, coded and subjected to thematic analysis to identify patterns and insights into the usability and effectiveness of the enhanced features on the MR training. RESULTS The study identified a wide range of requirements Medical First Responders (MFRs) have towards a MR training system that go beyond currently available solutions. A total of 80 performance metrics were initially identified and refined to a set of 54, which were categorized into KPI groups such as scene safety, triage performance and communication. Requirements for smart wearables to monitor stress levels are provided and highlight the importance of a user-centered design process to provide users with effective tools that fit their needs. Stress visualization preferences are described in form of a dashboard as well as in virtual environments surrounding the avatar. Using an iterative design process and user feedback, a training system was developed, integrating real-time performance tracking and stress monitoring. The field trial provided insights into practical use of these features during a real training exercise, showed interaction preferences between trainer and trainees and highlighted further improvement opportunities. CONCLUSIONS This research enhances MR training for paramedics by integrating real-time performance metrics and stress indicators, based on a human-centered design approach that aligns with end-user needs, thereby setting a foundation for developing more effective and immersive training solutions for high-stress professions. CLINICALTRIAL

show abstract

No Pain, No Gain—Giving Real-Time Emotional Feedback in a Virtual Mirror Improves Collaboration in Virtual Teamwork

Schneider,

Vazquez,

Gloor

2024

Applied Sciences

View full text Add to dashboard Cite

This study investigates the impact of real-time emotional feedback on the quality of teamwork conducted over videoconferencing. We developed a framework that provides real-time feedback through a virtual mirror based on facial and voice emotion recognition. In an experiment with 28 teams (84 participants), teams collaborated over Zoom (version 5.16.6) to set up a virtual Mars station using custom simulation software (Mars Star City, version 4.0). Participants were divided into 14 experimental teams, which were shown the virtual mirror, and 14 control teams without it. Team performance was measured by the improvement in the Mars simulation output quality. Our analysis using correlation, multi-level regression, and machine learning revealed that fewer interruptions but an increasing number over time correlated with higher performance. Higher vocal arousal and happiness also enhanced performance. We confirmed that female presence in teams boosts performance. SHAP values indicated that high variability in happiness, head movement, and positive facial valence—an “emotional rollercoaster”—positively predicted team performance. The experimental group outperformed the control group, suggesting that virtual mirroring improves virtual teamwork and that interrupting each other more while speaking less, leads to better results.

show abstract

Exploring objective measures for assessing team performance in healthcare: an interview study

Cited by 4 publications

References 68 publications

Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

No Pain, No Gain—Giving Real-Time Emotional Feedback in a Virtual Mirror Improves Collaboration in Virtual Teamwork

Contact Info

Product

Resources

About