Image Overlay Surgery Based on Augmented Reality: A Systematic Review

Abstract

Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. ARbased image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient's position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient's body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients' recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach allows to easily develop mobile AR applications, thus facilitating the implementation of AR-based image overlay surgery in clinical practice.

Effect of marker position and size on the registration accuracy of HoloLens in a non-clinical setting with implications for high-precision surgical tasks

“…Mean vertex and centroid position errors were up to 5 mm (Table 1 ). These findings are in line with the results of a systematic review which explored AR-guided open surgery [ 20 ] and with previous studies using holographic headsets [ 28 – 32 ]. A non-negligible percentage of cases (24–30% approximately) presented positional errors over 2 mm (Table 2 ).…”

“…Some studies using image-pattern-recognition have achieved submillimetre accuracy [ 15 – 19 ]. Few studies have systematically analysed the accuracy that can be achieved with the combined use of image-pattern-recognition algorithms and holographic headsets [ 20 ]. These studies did not measure the effect of image marker position and size on registration accuracy [ 1 ].…”

Smartphone-assisted augmented reality technology for preoperative planning in spine surgery

“…После совмещения объектов эти данные в формате приложения переносятся в исходное программное обеспечение, работающее на смартфоне (рис. 1.4, 1.5) [12,13].…”

Augmented Reality to Assist Skin Paddle Harvesting in Osteomyocutaneous Fibular Flap Reconstructive Surgery: A Pilot Evaluation on a 3D-Printed Leg Phantom

“…However, the provided registration accuracy between the real and the virtual content, that will affect the reliability of the virtual planning overlaying the patient anatomy, requires to be systematically evaluated before widely promoting this AR protocol in the clinical practice. Indeed, virtual-to-real scene registration, patient position tracking and projection of the digital content onto the targeted anatomical structures are crucial steps of AR-guided navigation systems ( 8 , 25 , 26 ).…”