The Challenge of Augmented Reality in Surgery

Abstract

Imaging has revolutionized surgery over the last 50 years. Diagnostic imaging is a key tool for deciding to perform surgery during disease management; intraoperative imaging is one of the primary drivers for minimally invasive surgery (MIS), and postoperative imaging enables effective follow up and patient monitoring. However, notably there is still relatively little interchange of information or imaging modality fusion between these different clinical pathway stages. This book chapter provides a critique of existing augmented reality (AR) methods or application studies described in the literature using relevant examples. The aim is not to provide a comprehensive review, but rather to give an indication of the clinical areas in which AR has been proposed, to begin to explain the lack of clinical systems and to provide some clear guidelines to those intending pursue research in this area.

SERV-CT: A disparity dataset from cone-beam CT for validation of endoscopic 3D reconstruction

“…This is a significant research field in itself, with AR abdominal laparoscopic applications having been proposed in various fields, including surgery of the liver ( Hansen, Wieferich, Ritter, Rieder, Peitgen, 2010 , Thompson, Schneider, Bosi, Gurusamy, Ourselin, Davidson, Hawkes, Clarkson, 2018 , Quero, Lapergola, Soler, Shahbaz, Hostettler, Collins, Marescaux, Mutter, Diana, Pessaux, 2019 ), uterus ( Bourdel et al., 2017 ), kidney and prostate ( van Oosterom, van der Poel, Navab, van de Velde, van Leeuwen, 2018 , Bertolo, Hung, Porpiglia, Bove, Schleicher, Dasgupta, 2020 , Hughes-Hallett, Mayer, Marcus, Cundy, Pratt, Darzi, Vale, 2014 ). The application-specific literature in all these areas as well as more general reviews of AR surgery ( Bernhardt, Nicolau, Soler, Doignon, 2017 , Edwards, Chand, Birlo, Stoyanov, 2021 ) all conclude that the technology shows great promise, but also note that alignment accuracy, workflow integration and perceptual issues have limited clinical uptake.…”

Implementation of 3D printed superior mesenteric vascular models for surgical planning and/or navigation in right colectomy with extended D3 mesenterectomy: comparison of virtual and physical models to the anatomy found at surgery

“…Lately, there is an increasing interest in mixed reality navigation systems for surgical applications, which promise facilitating complex dissections during surgery. Still, measurable clinical benefits remain to be seen [79]. From a clinical perspective, the study design focused on assisting the surgeon to perform a more radical dissection approach for right-sided colon cancer at our institution by correctly placing 3D vascular models, specific to the patient, as an overlay image over the patient.…”

Advances in surgical treatment for atlantoaxial instability focusing on rheumatoid arthritis: Analysis of a series of 67 patients

“…This microscope-based AR system is an optical see-through AR, in which the overlaid information is projected into the optical view by a half-silvered mirror of the microscope lens. 37 The advantage of optical see-through AR is that it provides a natural view and the clearest surgical view from the real world. 38 AR-IGS permits visualization of navigation information and the preset trajectory in the same surgical view.…”