Applications of Mixed Reality Technology in Orthopedics Surgery: A Pilot Study

Abstract

Objective: The aim of this study is to explore the potential of mixed reality (MR) technology in the visualization of orthopedic surgery.Methods: The visualization system with MR technology is widely used in orthopedic surgery. The system is composed of a 3D imaging workstation, a cloud platform, and an MR space station. An intelligent segmentation algorithm is adopted on the 3D imaging workstation to create a 3D anatomical model with zooming and rotation effects. This model is then exploited for efficient 3D reconstruction of data for computerized tomography (CT) and magnetic resonance imaging (MRI). Additionally, the model can be uploaded to the cloud platform for physical parameter tuning, model positioning, rendering and high-dimensional display. Using Microsoft’s HoloLens glasses in combination with the MR system, we project and view 3D holograms in real time under different clinical scenarios. After each procedure, nine surgeons completed a Likert-scale questionnaire on communication and understanding, spatial awareness and effectiveness of MR technology use. In addition to that, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) is also used to evaluate the workload of MR hologram support.Results: 1) MR holograms can clearly show the 3D structures of bone fractures, which improves the understanding of different fracture types and the design of treatment plans; 2) Holograms with three-dimensional lifelike dynamic features provide an intuitive communication tool among doctors and also between doctors and patients; 3) During surgeries, a full lesion hologram can be obtained and blended in real time with a patient’s virtual 3D digital model in order to give surgeons superior visual guidance through novel high-dimensional “perspectives” of the surgical area; 4) Hologram-based magnetic navigation improves the accuracy and safety of the screw placement in orthopaedics surgeries; 5) The combination of mixed reality cloud platform and telemedicine system based on 5G provides a new technology platform for telesurgery collaboration. Results of qualitative study encourage the usage of MR technology for orthopaedics surgery. Analysis of the Likert-scale questionnaire shows that MR adds significant value to understanding and communication, spatial awareness, learning and effectiveness. Based on the NASA TLX-scale questionnaire results, mixed reality scored significantly lower under the “mental,” “temporal,” “performance,” and “frustration” categories compared to usual 2D.Conclusion: The integration of MR technology in orthopaedic surgery reduces the dependence on surgeons’ experience and provides personalized 3D visualization models for accurate diagnosis and treatment of orthopaedic abnormalities. This integration is clearly one of the prominent future development directions in medical surgery.

Use of Mixed Reality in Neuro-Oncology: A Single Centre Experience

“…The neurosurgical community has been fortunate. Despite the slow and pendulous interest in adopting new technology, parallel developments in the mixed reality (MR) realms have allowed for the rekindling of interest in adapting new MR gadgets [ 1 , 5 , 19 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. HoloLens 2 has brought MR back to our theatres, providing a significant leap forward and overcoming the disadvantages of the previous VR and AR systems.…”

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

“…Within orthopedic surgery, Lu et al [ 7 ] describe a collection of cases in which MR primarily improved preoperative doctor-patient communication and patients’ understanding of complex pathology.…”

Advances and Innovations in Ablative Head and Neck Oncologic Surgery Using Mixed Reality Technologies in Personalized Medicine

“…It provides radiologists and surgeons with a multimodal interactive user interface for data processing and an efficient way to navigate through tomographic data, enabling surgical planning tailored to the patient [ 34 , 35 , 36 ]. Combined with mixed reality technology, digital patient-specific models can be created with high precision, enabling individualized treatment planning [ 37 ]. After the patient was scheduled for complex tumor resection, 3D digital reconstruction was performed using preoperative cross-sectional imaging.…”