Image tracking of laparoscopic instrument using spiking neural networks

Abstract

Minimally Invasive Surgery (MIS) has become more and more popular in recent years. An endoscopic image tracking system will assist surgeons to adjust the field of view autonomously in MIS. In this paper, we propose a novel image tracking algorithm based on natural features of surgical instruments. We suggest to use texture and geometric features in laparoscopic instrument imagery and to adopt a spiking neural network approach for object detection; considering color will be affected by lighting and the white balance conditions in the endoscope imagery. To enhance tracking performance, we further design a Kalman filter to combine with the neuro-based tracker. The instrument can be detected more robustly despite of deformation of the instrument image during surgery. A laparoscopic video has been tested to verify the developed methods. Experimental results show that two instrnments can be distinguished and tracked simultaneously in the surgical video.

CRF-Based Model for Instrument Detection and Pose Estimation in Retinal Microsurgery

“…Much research has been done to address the problem of detecting and tracking medical instruments including color-based [4, 5] and geometry-based [6–8] approaches. A recent work of Roodaki et al [1] proposed to estimate the instrument tip depth to retina surface by building their method on top of instrument tracking algorithms.…”