Deep Residual Learning for Instrument Segmentation in Robotic Surgery

Pakhomov, Daniil; Premachandran, Vittal; Allan, Max; Azizian, Mahdi; Navab, Nassir

doi:10.1007/978-3-030-32692-0_65

Cited by 89 publications

(80 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the binary segmentation task the best results is achieved by TernausNet-16 providing IoU = 0.836 and Dice = 0.901. These values are the best reported in the literature up to now [7, 15]. Next, we consider multi-class segmentation of different parts of instruments.…”

Section: Resultsmentioning

confidence: 88%

“…In the domain of medical imaging, convolutional neural networks (CNN) have been successfully used, for example, for breast cancer histology image analysis [17], bone disease prediction [21] and age assessment [10], and other problems [5]. Previous deep learning-based applications to robotic instrument segmentation have demonstrated competitive performance in binary segmentation [1, 7] and promising results in multi-class segmentation [15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

Shvets

Rakhlin

Kalinin

et al. 2018

Preprint

View full text Add to dashboard Cite

Semantic segmentation of robotic instruments is an important problem for the robot-assisted surgery. One of the main challenges is to correctly detect an instrument's position for the tracking and pose estimation in the vicinity of surgical scenes. Accurate pixel-wise instrument segmentation is needed to address this challenge. In this paper we describe our deep learning-based approach for robotic instrument segmentation. Our approach demonstrates an improvement over the state-of-the-art results using several novel deep neural network architectures. It addressed the binary segmentation problem, where every pixel in an image is labeled as an instrument or background from the surgery video feed. In addition, we solve a multi-class segmentation problem, in which we distinguish between different instruments or different parts of an instrument from the background. In this setting, our approach outperforms other methods for automatic instrument segmentation thereby providing state-of-the-art results for these problems. The source code for our solution is made publicly available.

show abstract

Section: Resultsmentioning

confidence: 88%

Section: Introductionmentioning

confidence: 99%

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

Shvets

Rakhlin

Kalinin

et al. 2018

Preprint

View full text Add to dashboard Cite

show abstract

“…However both surgical robotics and robotic imaging will play increasingly crucial roles in the years to come. Machine learning is demonstrating convincing results in real-time tool tracking [118], [172]- [174]. This for example enables automatic positioning of intra-operative OCT imaging planes within surgical microscopy for ophthalmic surgery [119], [175].…”

Section: Discussionmentioning

confidence: 96%

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

et al. 2020

Self Cite

View full text Add to dashboard Cite

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.Index Terms-Artificial intelligence, computer assisted interventions, interventional workflow, intra-operative imaging, surgical planning, data fusion, surgical scene understanding, contextaware user interface, machine and deep learning, surgical data science

show abstract

“…As specified in the guidelines, we preformed a cross-validation by leaving one surgery out of the training data. The segmentation result was compared to generic methods as well as algorithms that were explicitly published for this task [14,7,18]. Garcia P. Herrera et al [7] proposed a Fully convolutional network for segmentation in minimally invasive surgery.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…To achieve real-time performance, they applied the network only on every couple of frames and propagated the information with optical flow (FCN+OF). DLR [18] represents a deep residual network with dilated convolutions. Laina and Rieke et al [14] suggested a unified deep learning approach for simultaneous segmentation and 2D pose estimation using Fully Convolutional Residual Network with skip connections.…”

Section: Experimental Evaluationmentioning

confidence: 99%

CFCM: Segmentation via Coarse to Fine Context Memory

Milletarì

Rieke

Baust

et al. 2018

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

Self Cite

View full text Add to dashboard Cite

Recent neural-network-based architectures for image segmentation make extensive usage of feature forwarding mechanisms to integrate information from multiple scales. Although yielding good results, even deeper architectures and alternative methods for feature fusion at different resolutions have been scarcely investigated for medical applications. In this work we propose to implement segmentation via an encoderdecoder architecture which differs from any other previously published method since (i) it employs a very deep architecture based on residual learning and (ii) combines features via a convolutional Long Short Term Memory (LSTM), instead of concatenation or summation. The intuition is that the memory mechanism implemented by LSTMs can better integrate features from different scales through a coarse-to-fine strategy; hence the name Coarse-to-Fine Context Memory (CFCM). We demonstrate the remarkable advantages of this approach on two datasets: the Montgomery county lung segmentation dataset, and the EndoVis 2015 challenge dataset for surgical instrument segmentation. * Maximilian Baust is now working for

show abstract

Deep Residual Learning for Instrument Segmentation in Robotic Surgery

Cited by 89 publications

References 18 publications

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions

CFCM: Segmentation via Coarse to Fine Context Memory

Contact Info

Product

Resources

About