Next step trauma and orthopaedic surgery: integration of augmented reality for reduction and nail implantation of tibial fractures

Abstract

Introduction There is a tremendous scope of hardware and software development going on in augmented reality (AR), also in trauma and orthopaedic surgery. However, there are only a few systems available for intra-operative 3D imaging and guidance, most of them rely on peri- and intra-operative X-ray imaging. Especially in complex situations such as pelvic surgery or multifragmentary multilevel fractures, intra-operative 3D imaging and implant tracking systems have proven to be of great advantage for the outcome of the surgery and can help reduce X-ray exposure, at least for the surgical team (Ochs et al. in Injury 41:1297 1305, 2010). Yet, the current systems do not provide the ability to have a dynamic live view from the perspective of the surgeon. Our study describes a prototype AR-based system for live tracking which does not rely on X-rays. Materials and methods A protype live-view intra-operative guidance system using an AR head-mounted device (HMD) was developed and tested on the implantation of a medullary nail in a tibia fracture model. Software algorithms that allow live view and tracking of the implant, fracture fragments and soft tissue without the intra-operative use of X-rays were derived. Results The implantation of a medullar tibia nail is possible while only relying on AR-guidance and live view without the intra-operative use of X-rays. Conclusions The current paper describes a feasibility study with a prototype of an intra-operative dynamic live tracking and imaging system that does not require intra-operative use of X-rays and dynamically adjust to the perspective of the surgeons due to an AR HMD. To our knowledge, the current literature does not describe any similar systems. This could be the next step in surgical imaging and education and a promising way to improve patient care.

Towards an augmented reality system supporting nail implantation for tibial fractures

“…An AR overlay of the bony fragments at the correct anatomical position as well as a visualization of the nail to be inserted has the potential to reduce radiation exposure and intervention time. In [1] we presented a first prototype of a pre-clinical system demonstrating the feasibility of such an approach (see Figure 1) in a phantom study. In this paper, we will present how real clinical data sets can be prepared to be used in the system supporting nail insertion on tibial fractures.…”

“…3D Slicer 1 is an open-source software platform used for visualizing, segmenting, and analyzing medical images. We used 3D Slicer to create the initial manual gold standard segmentation.…”

“…The segmented data sets are converted to surface meshes and the number of polygons was reduced to allow for smooth interaction with the scene as a stand-alone application on the hololens. At the current stage of the project, UI and tracking of bones and instruments are used as described in [1].…”

The 3D operating room with unlimited perspective change and remote support

“…Improved 3D imaging would provide both, an increase in the efficiency of the medical procedure, and reduce the time of exposure to X-rays of the medical team [12,21]. Thus, the extended reality has a huge potential to become an intraoperative support for surgeons by visualization and superimposition of CT or MRI image scans collected before surgery on the patient, thus improving navigation during surgery [22]. The intraoperatively 3D model/objects of the desired anatomical structures can be displayed in HDMs [23].…”