Novel navigation system by augmented reality technology using a tablet PC for hepatobiliary and pancreatic surgery

Abstract

We successfully developed the tablet method and tested it in a clinical setting. This system may contribute to surgical efficacy and improve the educational effects.

Development of recognised position‐guided navigation system

“…This system has been clinically applied since 2006 for hepatobiliary and pancreatic surgery. [1][2][3][4][5][6][7] Initially, the internal structure of open hepatectomy was overlaid; however, this technique proved inferior to conventional ultrasonography affected by organ deformation due to progressive convolution and hepatic dissection. 5 Nevertheless, the system has proved effective especially for laparoscopic hepatectomy and pancreatic surgery where organs are fixed and less likely to deform or less likely to move.…”

“…Because the IG‐NS displays preoperative simulation images on the operative field directly, the operator is able to locate dissection sites and lesions in a more intuitive and sensory manner. It also allows users to share information via tablets and monitors, and the availability of annotation functions is helpful in educating comedical personnel and students 6,15,16 . However, these are simply display functions, and similar to indocyanine green (ICG) fluorescence endoscopy that has become popular in recent years, the system's effects on surgical operations are limited to the skills of individual operators (on how to utilise the displayed information).…”

“…The new system was tested under the same surgical environment as used for the previously reported IG-NS. 6,7 The positional recognition functions of the system tracked positional information using a pointing device fitted with optical position sensor (Optotrak: Northern Digital Inc.), which was attached to the surgical apparatus (either Pean forceps or scissors) (Figure 1). Then, data were measured with an optical location sensor installed in the operating room.…”

“…Previously, we developed an image‐guided navigation system (IG‐NS) incorporating augmented reality (AR) technology as a novel surgical navigation system and reported its effectiveness and problems in hepatobiliary and pancreatic surgery 1‐7 . The IG‐NS was capable of overlaying the images on the operative field in real time, allowing the operator to intuitively recognise where to dissect and locate lesions.…”

Research on Accuracy of Augmented Reality Surgical Navigation System Based on Multi-View Virtual and Real Registration Technology

“…This method greatly reduces the pain of the patient and improves the success rate of the surgery. Jungo Yasuda et al developed a new navigation system by using augmented reality in some surgical procedures similar to hepatobiliary soft tissue [12]. It can perform surgery on the liver and pancreas using a tablet computer, and capture the three-dimensional (3D) model in real time by the camera of the tablet computer to display the 3D model superimposed on the surgical area.…”