Enhancing Mixed Reality Simulation Training Technology with Real-Time Performance Visualization: Mixed Methods Study with Medical First Responders (Preprint)

Abstract

BACKGROUND Mixed Reality (MR) simulation training is emerging in paramedical education for practicing stress-intensive scenarios like mass casualty incidents in a safe and controlled environment. Current training platforms, however, lack real-time stress and human performance monitoring tools. OBJECTIVE The study aims to enhance mixed reality training for medical first responders through real-time evaluation of performance and stress levels, leveraging bio-signal monitoring and advanced analytics to allow instructors to tailor feedback and maintain optimal challenge and safety levels. METHODS The study includes a structured, multi-phase approach including initial requirement gathering (structured interviews and co-creation workshops), an online design survey, iterative prototype development and a field trial (including training observations and interviews). Data was collected from five end-user consortium members across Europe. Quantitative data from checklists were analyzed using frequencies and percentages to understand feature usage and event occurrences. Qualitative data from semi-structured interviews and co-creation workshops were transcribed, coded and subjected to thematic analysis to identify patterns and insights into the usability and effectiveness of the enhanced features on the MR training. RESULTS The study identified a wide range of requirements Medical First Responders (MFRs) have towards a MR training system that go beyond currently available solutions. A total of 80 performance metrics were initially identified and refined to a set of 54, which were categorized into KPI groups such as scene safety, triage performance and communication. Requirements for smart wearables to monitor stress levels are provided and highlight the importance of a user-centered design process to provide users with effective tools that fit their needs. Stress visualization preferences are described in form of a dashboard as well as in virtual environments surrounding the avatar. Using an iterative design process and user feedback, a training system was developed, integrating real-time performance tracking and stress monitoring. The field trial provided insights into practical use of these features during a real training exercise, showed interaction preferences between trainer and trainees and highlighted further improvement opportunities. CONCLUSIONS This research enhances MR training for paramedics by integrating real-time performance metrics and stress indicators, based on a human-centered design approach that aligns with end-user needs, thereby setting a foundation for developing more effective and immersive training solutions for high-stress professions. CLINICALTRIAL