Abstract-Despite a wide range of feature detectors developed in the computer vision community over the years, direct application of these techniques to surgical navigation has shown significant difficulties due to the paucity of reliable salient features coupled with free-form tissue deformation and changing visual appearance of surgical scenes. The aim of this paper is to propose a novel probabilistic framework to track affine-invariant anisotropic regions under contrastingly different visual appearances during Minimally Invasive Surgery (MIS). The theoretical background of the affine-invariant anisotropic feature detector is presented and a real-time implementation exploiting the computational power of the GPU is proposed. An Extended Kalman Filter (EKF) parameterisation scheme is used to adaptively adjust the optimal templates of the detected regions, enabling accurate identification and matching of the tracked features. For effective tracking verification, spatial context and region similarity have also been incorporated. They are used to boost the prediction of the EKF and recover potential tracking failure due to drift or false positives. The proposed framework is compared to the existing methods and their respective performance is evaluated with in vivo video sequences recorded from robotic assisted MIS procedures, as well as real-world scenes.

The proposed method shows a novel architecture to perform real-time monocular depth estimation without losing patient specificity in bronchoscopy. Future work will include integration within SLAM systems and collection of in vivo datasets.

The use of physically-based models combined with image constraints for intraoperative guidance is important for surgical procedures that involve large-scale tissue deformation. A biomechanical model of tissue deformation is described in which surface positional constraints and internally generated forces are derived from endoscopic images and preoperative 4D CT data, respectively. Considering cardiac motion, a novel technique is presented which minimises the average registration error over one or more complete cycles. Features tracked in the stereo video stream provide surface constraints, and an inverse finite element simulation is presented which allows internal forces to be recovered from known preoperative displacements. The accuracy of surface texture, segmented mesh and volumetrically rendered overlays is evaluated with detailed phantom experiments. Results indicate that by combining preoperative and intraoperative images in this manner, accurate intraoperative tissue deformation modelling can be achieved.

No abstract