No abstract

The dynamic COVID-19 pandemic has destabilized education and forced academic centers to explore non-traditional teaching modalities. A key challenge this creates is in reconciling the fact that hands-on time in lab settings has been shown to increase student understanding and peak their interests. Traditional visualization methods are already limited and topics such as 3D molecular structures remain difficult to understand. This is where advances in Information and Communication Technologies (ICT), including remote meetings, Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Extended Reality (XR, so-called Metaverse) offer vast potential to revolutionize the education landscape. Specifically, how MR merges real and virtual life in a uniquely promising way and offers opportunities for entirely new educational applications. In this paper, we briefly overview and report our initial experience using MR to teach medical and pharmacy students. We also explore the future usefulness of MR in pharmacy education. MR mimics real-world experiences both in distance education and traditional laboratory classes. We also propose ICT-based systems designed to run on the Microsoft HoloLens2 MR goggles and can be successfully applied in medical and pharmacy coursework. The models were developed and implemented in Autodesk Maya and exported to Unity. Our findings demonstrate that MR-based solutions can be an excellent alternative to traditional classes, notably in medicine, anatomy, organic chemistry, and biochemistry (especially 3D molecular structures), in both remote and traditional in-person teaching modalities. MR therefore has the potential to become an integral part of medical education in both remote learning and in-person study.

Information and communication technologies combined with extended reality improve diagnostics, medical treatment, and surgical operations courses. Thus, the new generation of devices, which enable displaying of holographic objects, allows visualizing a patient’s internal organs, both anatomical and pathological structures, as interactive 3D objects, based on retrospective 2D images, namely computer tomography (CT) or magnetic resonance imaging (MRI). Remote users can observe ongoing surgery with additional features like standing in front of the operation table, walking around in, and changing the user’s perspective without disrupting the operating doctors. Moreover, the operation can be recorded, then played with additional functionalities—remote pointing and painting, which is important in medical education. The operating surgeon may also ask remotely more experienced operators for consultation, and as a consequence, the number of complications and failed procedures can decrease. It can be also applied to robot surgeries as a first step to remote surgery.