A comprehensive method to design and assess mixed reality simulations

Abstract

The scientific literature highlights how Mixed Reality (MR) simulations allow obtaining several benefits in healthcare education. Simulation-based training, boosted by MR, offers an exciting and immersive learning experience that helps health professionals to acquire knowledge and skills, without exposing patients to unnecessary risks. High engagement, informational overload, and unfamiliarity with virtual elements could expose students to cognitive overload and acute stress. The implementation of effective simulation design strategies able to preserve the psychological safety of learners and the investigation of the impacts and effects of simulations are two open challenges to be faced. In this context, the present study proposes a method to design a medical simulation and evaluate its effectiveness, with the final aim to achieve the learning outcomes and do not compromise the students' psychological safety. The method has been applied in the design and development of an MR application to simulate the rachicentesis procedure for diagnostic purposes in adults. The MR application has been tested by involving twenty students of the 6th year of Medicine and Surgery of Università Politecnica delle Marche. Multiple measurement techniques such as self-report, physiological indices, and observer ratings of performance, cognitive and emotional states of learners have been implemented to improve the rigour of the study. Also, a user-experience analysis has been accomplished to discriminate between two different devices: Vox Gear Plus® and Microsoft Hololens®. To compare the results with a reference, students performed the simulation also without using the MR application. The use of MR resulted in increased stress measured by physiological parameters without a high increase in perceived workload. It satisfies the objective to enhance the realism of the simulation without generating cognitive overload, which favours productive learning. The user experience (UX) has found greater benefits in involvement, immersion, and realism; however, it has emphasized the technological limitations of devices such as obstruction, loss of depth (Vox Gear Plus), and narrow FOV (Microsoft Hololens).

Measuring cognitive load in augmented reality with physiological methods: A systematic review

“…Therefore, experimental design and analytical procedures need to be tactical. Some researchers are advocating the exhaustive approach incorporating self‐rating, performance, and multi‐sensor physiological methods all together (Brunzini et al, 2022), and most of the reviewed studies took a multi‐method approach at some levels. Statistical use of parameters collectively can increase the accuracy of the measurement (Brunzini et al, 2022; Ogunseiju et al, 2021; Ruberto et al, 2021; Seeling, 2017), but the simplicity of measurement also matters for the practical use and applicability of them.…”

“…While AR use in construction and physics education embraces a mixture of techniques including contiguity, spatial operation and concretisation of the abstracts, medical education largely explored reality and immersion (Antoniou et al, 2020; Brunzini et al, 2022). This indicates that GSR/EDA and ECG are chosen as preferred methods to evaluate reality and immersion while eye‐tracking was applied to examine a wider range of AR techniques.…”

“…Ruberto et al and Brunzini et al explored multiple sensor data from ECG and GSR/EDA with statistics and machine learning approaches (Brunzini et al, 2022; Ruberto et al, 2021). The analysis by Ruberto et al shows that the mean RR intervals and ultra‐low frequency band (ULF) of ECG signals are the most influential parameters to separate experts from novices (Ross et al, 2019) though their sample size is very small.…”

“…For example, the reliability of devices such as Empatica E4 and Muse, which were used in the reviewed studies (Antoniou et al, 2020;Brunzini et al, 2022), is questionable (Borrego et al, 2019;Karwowski et al, 2022).…”

“…One of the advantages of physiological measures is the availability of a continuous data stream that allows researchers to access direct and temporal information on cognitive load although self‐rating may still excel in simplicity and sensitivity to more subtle differences (Bläsing & Bornewasser, 2021; Mayer, 2019; Sweller et al, 2019). It seems that the increasing penetration and improving accuracy of non‐invasive and low‐cost biometric measurement devices that can be applied more widely are also behind this trend (Brunzini et al, 2022; Wolf et al, 2019).…”