Accurate three-dimensional virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot

Abstract

Abstract. Brain tumor margin removal is challenging because diseased tissue is often visually indistinguishable from healthy tissue. Leaving residual tumor leads to decreased survival, and removing normal tissue causes lifelong neurological deficits. Thus, a surgical robotics system with a high degree of dexterity, accurate navigation, and highly precise resection is an ideal candidate for image-guided removal of fluorescently labeled brain tumor cells. To image, we developed a scanning fiber endoscope (SFE) which acquires concurrent reflectance and fluorescence wide-field images at a high resolution. This miniature flexible endoscope was affixed to the arm of a RAVEN II surgical robot providing programmable motion with feedback control using stereo-pair surveillance cameras. To verify the accuracy of the three-dimensional (3-D) reconstructed surgical field, a multimodal physical-sized model of debulked brain tumor was used to obtain the 3-D locations of residual tumor for robotic path planning to remove fluorescent cells. Such reconstruction is repeated intraoperatively during margin clean-up so the algorithm efficiency and accuracy are important to the robotically assisted surgery. Experimental results indicate that the time for creating this 3-D surface can be reduced to one-third by using known trajectories of a robot arm, and the error from the reconstructed phantom is within 0.67 mm in average compared to the model design.

Semi-autonomous simulated brain tumor ablation with RAVENII Surgical Robot using behavior tree

“…Gong et al [27] showed the feasibility and quality of 3D target reconstruction in a curved phantom using the SFE mechanically scanned by same platform in this study (Figure 2). …”

Bound constrained bundle adjustment for reliable 3D reconstruction

“…As the extension work of our previous research [25,30], these advantages of BCBA can be utilized for intraoperatively reconstructing accurate 3D virtual models of the surgical field to extract the tumor coordinates, which requires highly efficient processing. These advantages also can be utilized to provide reliable estimation of camera poses for 3D image-guided surgery [31] and MIS.…”

“…To validate and compare two 3D reconstruction algorithms, an object with pre-known geometry was required. In this study, a 3D model was designed and 3D-printed out to mimic the surgical field [30], see Figs. 2(b) and 2(c).…”

Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model

“…By robotically moving the tiny SFE or micro-camera, three-dimensional (3D) virtual reconstructions can be made for minimally-invasive disease surveillance 4 , guiding biopsy 2 , or planning surgical procedures 3 . By attaching the SFE to a calibrated robotic arm and scanning across a phantom of a cavity from debulked brain tumor with fluorescence targets representing residual tumor tissue, accurate 3D virtual reconstructions of the surgical field can be generated 5. …”

“…Without the surgeon in the loop, the RAVEN II currently does not provide accurate closed-loop servo control using the mechanical linkages and sensors within the robot arm, requiring an external measurement system for tracking the tip of the robot arm. To provide 0.5-mm accuracy of the scope/probe set during RAVEN procedures with a phantom of the debulked tumor cavity, an external stereo vision tracking system has been added 5 , but limitations restrict its use to more open surgical procedures by requiring constant environment lighting and direct line-of sight to the tip of the scope/tool set. Thus a new navigational system is required when the scope/tool set enters the debulked surgical cavity for residual tumor removal.…”