Fusion and visualization of intraoperative cortical images with preoperative models for epilepsy surgical planning and guidance

Abstract

Objective: During epilepsy surgery it is important for the surgeon to correlate the preoperative cortical morphology (from preoperative images) with the intraoperative environment. Augmented Reality (AR) provides a solution for combining the real environment with virtual models. However, AR usually requires the use of specialized displays, and its effectiveness in the surgery still needs to be evaluated. The objective of this research was to develop an alternative approach to provide enhanced visualization by fusing a direct (photographic) view of the surgical field with the 3D patient model during image guided epilepsy surgery. Materials and Methods: We correlated the preoperative plan with the intraoperative surgical scene, first by a manual landmark-based registration and then by an intensity-based perspective 3D-2D registration for camera pose estimation. The 2D photographic image was then texture-mapped onto the 3D preoperative model using the solved camera pose. In the proposed method, we employ direct volume rendering to obtain a perspective view of the brain image using GPUaccelerated ray-casting. The algorithm was validated by a phantom study and also in the clinical environment with a neuronavigation system. Results: In the phantom experiment, the 3D Mean Registration Error (MRE) was 2.43 AE 0.32 mm with a success rate of 100%. In the clinical experiment, the 3D MRE was 5.15 AE 0.49 mm with 2D in-plane error of 3.30 AE 1.41 mm. A clinical application of our fusion method for enhanced and augmented visualization for integrated image and functional guidance during neurosurgery is also presented. Conclusions: This paper presents an alternative approach to a sophisticated AR environment for assisting in epilepsy surgery, whereby a real intraoperative scene is mapped onto the surface model of the brain. In contrast to the AR approach, this method needs no specialized display equipment. Moreover, it requires minimal changes to existing systems and workflow, and is therefore well suited to the OR environment. In the phantom and in vivo clinical experiments, we demonstrate that the fusion method can achieve a level of accuracy sufficient for the requirements of epilepsy surgery.

An integrated approach to endoscopic instrument tracking for augmented reality applications in surgical simulation training

“…The experimental values are very close to the reference ones, although there is some jitter that increases with distance, as expected. To obtain a better understanding about the significance of this deviation, in Figure 12 we present the absolute errors of Z (in mm), as well as the corresponding marker width error calculated from the experimental and reference values of the width of the tip (d theor = f  L/Z th , where L is the physical width of the marker; see equation (16)). For working distances close to the camera (< 17 cm), the average error was found: Z err = 2.5 AE 2.1 mm (AE SD), whereas for longer distances, Z err = 6.2 AE 4.5 mm (AE SD, 17 cm < Z < 26 cm).…”

3D Preoperative Planning in the ER with OsiriX®: When There is No Time for Neuronavigation

“…The possibility that we may have an X-ray vision, enabling the neurosurgeon to see and interact with the precise location of the hematoma through a 3D model of the skull, represents a breakthrough that cannot be overlooked or underutilized [36–39]. Furthermore, the incorporation of this technology allows a precision that enables the performance of minimally invasive procedures with similar success rates to established procedures [1,4].…”

Functional Magnetic Resonance Imaging for Language Mapping in Temporal Lobe Epilepsy

“…This in turn degrades the fidelity of registration between ESM and fMRI. Figure 5 demonstrates our approach, which employs automatic 2D to 3D projective registration to fuse an interoperative cortical photographic image onto an MR brain image that incorporates a preoperative fMRI activation map [32]. In, Figure 5(a) the label “F” is a speech arrest site elicited by ESM that corresponds well to the peak of fMRI activation in the frontal lobe as shown in Figure 5(b).…”