Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

Abstract

The challenge is to accurately guide the surgical tool within the three-dimensional (3D) surgical field for robotically-assisted operations such as tumor margin removal from a debulked brain tumor cavity. The proposed technique is 3D image-guided surgical navigation based on matching intraoperative video frames to a 3D virtual model of the surgical field. A small laser-scanning endoscopic camera was attached to a mock minimally-invasive surgical tool that was manipulated toward a region of interest (residual tumor) within a phantom of a debulked brain tumor. Video frames from the endoscope provided features that were matched to the 3D virtual model, which were reconstructed earlier by raster scanning over the surgical field. Camera pose (position and orientation) is recovered by implementing a constrained bundle adjustment algorithm. Navigational error during the approach to fluorescence target (residual tumor) is determined by comparing the calculated camera pose to the measured camera pose using a micro-positioning stage. From these preliminary results, computation efficiency of the algorithm in MATLAB code is near real-time (2.5 sec for each estimation of pose), which can be improved by implementation in C++. Error analysis produced 3-mm distance error and 2.5 degree of orientation error on average. The sources of these errors come from 1) inaccuracy of the 3D virtual model, generated on a calibrated RAVEN robotic platform with stereo tracking; 2) inaccuracy of endoscope intrinsic parameters, such as focal length; and 3) any endoscopic image distortion from scanning irregularities. This work demonstrates feasibility of micro-camera 3D guidance of a robotic surgical tool.

Path planning for semi-automated simulated robotic neurosurgery

“…Furthermore, the stereo-vision may lose the tracking ability during the ablation inside a deep cavity. In this case, the pose estimation of surgical tool tip from the SFE image [29] will play a key role for robot motion guidance.…”

Bound constrained bundle adjustment for reliable 3D reconstruction

“…These advantages also can be utilized to provide reliable estimation of camera poses for 3D image-guided surgery [31] and MIS. Not limited to biomedical research, the BCBA algorithm may be utilized in other fields, such as a common application of the 3D reconstruction of Google street view [34].…”

“…2(b) and 2(c). This reconstruction of the surgical field aims to guide the autonomous surgical robot by obtaining the accurate 3D coordinates of residual tumor within the surgical field [31]. In this experiment, the SFE was attached on a micro-positioning stage, where the position of the camera was obtained.…”

A cable‐driven soft robotic end‐effector actuator for probe‐based confocal laser endomicroscopy: Development and preclinical validation

“…focus on general surgical, ear nose and throat and urological procedures where they have seen widespread adoption [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15]. Neurosurgery is yet to fully embrace robotic surgery for precision tumour resection with a growing interest in paediatrics and epilepsy surgery, but the earliest attempts leveraged the PUMA 560 for brain tumour biopsies [16][17][18][19][20][21][22]. To overcome challenges in achieving gross total resection and excision margin delineation affecting tumour recurrence and poor prognosis, other real-time in-situ optical and endoscopic methods have been proposed such as two-photon microscopy and Raman spectroscopy [19,20,23,24].…”

“…Neurosurgery is yet to fully embrace robotic surgery for precision tumour resection with a growing interest in paediatrics and epilepsy surgery, but the earliest attempts leveraged the PUMA 560 for brain tumour biopsies [16][17][18][19][20][21][22]. To overcome challenges in achieving gross total resection and excision margin delineation affecting tumour recurrence and poor prognosis, other real-time in-situ optical and endoscopic methods have been proposed such as two-photon microscopy and Raman spectroscopy [19,20,23,24]. Probe-based confocal laser endomicroscopy (p-CLE) for in-situ histology is such a method that can be used to differentiate cancer from normal tissue and guide precision surgery [25][26][27][28][29][30][31][32][33].…”