A Comparative Study of Spatio-Temporal U-Nets for Tissue Segmentation in Surgical Robotics

Abstract

In surgical robotics, the ability to achieve high levels of autonomy is often limited by the complexity of the surgical scene. Autonomous interaction with soft tissues requires machines able to examine and understand the endoscopic video streams in real-time and identify the features of interest. In this work, we show the first example of spatio-temporal neural networks, based on the U-Net, aimed at segmenting soft tissues in endoscopic images. The networks, equipped with Long Short-Term Memory and Attention Gate cells, can extract the correlation between consecutive frames in an endoscopic video stream, thus enhancing the segmentation's accuracy with respect to the standard U-Net. Initially, three configurations of the spatiotemporal layers are compared to select the best architecture. Afterwards, the parameters of the network are optimised and finally the results are compared with the standard U-Net. An accuracy of 83.77% ± 2.18% and a precision of 78.42% ± 7.38% are achieved by implementing both Long Short Term Memory (LSTM) convolutional layers and Attention Gate blocks. The results, although originated in the context of surgical tissue retraction, could benefit many autonomous tasks such as ablation, suturing and debridement.

Robot-Assisted Minimally Invasive Surgery—Surgical Robotics in the Data Age

“…Surgeons' vision may also be enhanced via semantic information, and therefore, active research is underway in the community on surgical tool identification and tracking, which may be an efficient tool not only for compensating robot inaccuracies or performing surgical skill assessment but also a necessary safety feature during autonomous task execution, replicating the surgeon's visual feedback loop [72], [73]. Computer vision-based methods are known for identifying the anatomy and providing situation awareness in the environment [74] or to provide accurate surgical phase and workflow reconstruction [75], [76].…”

Deep Learning Based Image Processing for Robot Assisted Surgery: A Systematic Literature Survey

“…The tissue segmentation appears to be the third largest studied task in this survey comprising around 12% of the total articles (see Table 6). This section comprises all the studies performed on tissues including vessel segmentation, edge detection, healthy and cancerous tissue classification, uncertainty inference segmentation, and tissue retraction [164]- [175].…”

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

“…A wide range of papers have reported algorithms aimed at detecting, segmenting, and tracking important elements in the surgical scene, such as surgical instruments [172], [182], [183], [186], [188], [189], [191], [192], [199], [219], [256], tissues [258], suturing needles [201], and threads [200]. Detecting and tracking image structures or instruments is important for various applications, for example, avoiding instrument interaction with certain tissues or automating surgical subtasks.…”