Easy-to-Use Augmented Reality Neuronavigation Using a Wireless Tablet PC

Abstract

Background/Aims: Augmented reality (AR) technology solves the problem of view switching in traditional image-guided neurosurgery systems by integrating computer-generated objects into the actual scene. However, the state-of-the-art AR solution using head-mounted displays has not been widely accepted in clinical applications because it causes some inconvenience for the surgeon during surgery. Methods: In this paper, we present a Tablet-AR system that transmits navigation information to a movable tablet PC via a wireless local area network and overlays this information on the tablet screen, which simultaneously displays the actual scene captured by its back-facing camera. With this system, the surgeon can directly observe the intracranial anatomical structure of the patient with the overlaid virtual projection images to guide the surgery. Results: The alignment errors in the skull specimen study and clinical experiment were 4.6 pixels (approx. 1.6 mm) and 6 pixels (approx. 2.1 mm), respectively. The system was also used for navigation in 2 actual clinical cases of neurosurgery, which demonstrated its feasibility in a clinical application. Conclusions: The easy-to-use Tablet-AR system presented in this study is accurate and feasible in clinical applications and has the potential to become a routine device in AR neuronavigation.

Augmented reality in neurosurgery: a systematic review

“…More recently, a tablet based-AR system[8] was applied to neurosurgery. It consists of a navigational tablet that superimposes a virtual 3D model on the surgical field as it is recorded by the tablet’s posterior camera.…”

“…In fact, optical tracking is very practical because it does not require wires to connect the tracked object. Additionally, it is promptly available because it can rely on widely available cameras included in smartphones, tablets,[8] digital recording cameras. [14] The main shortcoming is that tracked objects have to be in line of sight of the tracking system.…”

“…[17,22,11,29,23,14,32,4,3,15,8,5] We present a literature review aiming to: describe and evaluate the advantages and shortcomings of each of the different AR setups tested in vivo in humans; to understand the efficacy of AR in the treatment of neurosurgical diseases; and, to define potential future research directions.…”

The role of technology in minimally invasive surgery: state of the art, recent developments and future directions

“…This approach can alleviate the problems arising from the loss of depth perception and obstructed vision encountered in MIS, by showing anatomical information of targets that would otherwise be hidden to the eye 51 52 86. Current setups have the possibility of offering ‘on-demand’ augmented reality overlaid directly on the video feed of the operative screen, using a transparent screen or a separate device such as a tablet PC 87 88. Current research is focusing on relieving the surgeon from the pressure of deciding when visual input is needed, employing machine learning algorithms and analysis of surgical workflows to automate and improve the augmented reality systems 89…”

Towards estimating fiducial localization error of point-based registration in image-guided neurosurgery

“…For example, in image-guided surgery, this technique can be used to track the navigation tools, register images with different modalities, register the patient space to the image space and serve as an initialization step in surface based patient-to-image registration [1][2][3][4][5]. The point-based registration problem can be solved analytically [6], but the two coordinate systems cannot be registered perfectly because of the imperfectness of the correspondence.…”