Hybrid Tracking Module for Real-Time Tool Tracking for an Autonomous Exoscope

Abstract

Exoscopes have emerged as a promising visual solution within the field of microneurosurgery. However, manual repositioning poses a challenge causing interruptions that disrupt the surgical flow. Thus, the need for hands-free exoscope control arises. This paper introduces a position-based visual-servoing control approach, comprising a detection module, a hybrid tracking module, and a control module that adjusts a robotic camera holder to follow a surgical tool. The hybrid module was integrated to track and predict the surgical tool's future position to minimize system latency. The proposed system is composed of a 7 Degree-of-Freedom robotic manipulator with an eye-in-hand stereo camera. A comparative analysis with three alternative approaches (Convolutional Neural Network -CNN, Particle Filter -PF, Optical Flow -OF) was assessed using Tracking Error and Center Error metrics. Results showed improved tracking accuracy with an average error of 9.84 ± 0.08 mm for slow movements (2.5 cm/s) and 13.11±0.39 mm for rapid movements (4 cm/s). Finally, a User Study was conducted to investigate whether the proposed system effectively reduced the users' workload compared to the manual repositioning of the camera.