Segmentation of surgical instruments in laparoscopic videos: training dataset generation and deep-learning-based framework

Lee, Eung Joo; Plishker, William; Liu, Xinyang; Kane, Timothy D.; Bhattacharyya, Shuvra S.; Shekhar, Raj

doi:10.1117/12.2512994

Cited by 12 publications

(7 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparative tests were also conducted on other models using identical pre-processing and augmentation techniques. Lee et al (2019b) presented a "Two-phase Deep learning Segmentation for Laparoscopic Images" (TDSLI) model and tested it on the EndoVis 2017 dataset and an additional dataset of four retrospectively collected laparoscopic image sequences in different animal surgeries. The LinkNet-34 network was used in a convolutional encoder-decoder architecture, with a pre-trained ResNet-34 network used for the encoder.…”

Section: Tool Segmentation Researchmentioning

confidence: 99%

Surgical Tool Datasets for Machine Learning Research: A Survey

2022

View full text Add to dashboard Cite

This paper is a comprehensive survey of datasets for surgical tool detection and related surgical data science and machine learning techniques and algorithms. The survey offers a high level perspective of current research in this area, analyses the taxonomy of approaches adopted by researchers using surgical tool datasets, and addresses key areas of research, such as the datasets used, evaluation metrics applied and deep learning techniques utilised. Our presentation and taxonomy provides a framework that facilitates greater understanding of current work, and highlights the challenges and opportunities for further innovative and useful research.

show abstract

Section: Tool Segmentation Researchmentioning

confidence: 99%

Surgical Tool Datasets for Machine Learning Research: A Survey

2022

View full text Add to dashboard Cite

show abstract

“…While the task of binary instrument segmentation received a lot of attention over the last couple of years such as [12,13,14,15,16], literature on multi-instance segmentation in applications for minimally invasive surgeries is extremely sparse. To our knowledge, the only peer-reviewed work published independently of the ROBUST-MIS challenge [3] (which this work is based on), was published by Shvets et al [13].…”

Section: Multi-instance Segmentationmentioning

confidence: 99%

How can we learn (more) from challenges? A statistical approach to driving future algorithm development

Ross¹,

Bruno²,

Reinke³

et al. 2021

Preprint

View full text Add to dashboard Cite

Challenges have become the state-of-the-art approach to benchmark image analysis algorithms in a comparative manner. While the validation on identical data sets was a great step forward, results analysis is often restricted to pure ranking tables, leaving relevant questions unanswered. Specifically, little effort has been put into the systematic investigation on what characterizes images in which state-of-the-art algorithms fail. To address this gap in the literature, we (1) present a statistical framework for learning from challenges and (2) instantiate it for the specific task of instrument instance segmentation in laparoscopic videos. Our framework relies on the semantic meta data annotation of images, which serves as foundation for a General Linear Mixed Models (GLMM) analysis. Based on 51,542 meta data annotations performed on 2,728 images, we applied our approach to the results of the Robust Medical Instrument Segmentation Challenge (ROBUST-MIS) challenge 2019 and revealed underexposure, motion and occlusion of instruments as well as the presence of smoke or other objects in the background as major sources of algorithm failure. Our subsequent method development, tailored to the specific remaining issues, yielded a deep learning model with state-of-the-art overall performance and specific strengths in the processing of images in which previous methods tended to fail. Due to the objectivity and generic applicability of our approach, it could become a valuable tool for validation in the field of medical image analysis and beyond. and segmentation of small, crossing, moving and transparent instrument(s) (parts). Keywords surgical data science • image characteristics driven algorithm development • minimally invasive surgery • endoscopic vision • grand challenges • biomedical image analysis challenges • generalized linear mixed models • instrument segmentation • deep learning • artificial intelligence

show abstract

“…Ceron et al [16] recently proposed an attention-based segmentation for MIS instruments on the same datasets that achieved state-of-the-art performance at nearly 45 frames-per-second. In addition to these challenges, there are works on segmentation [17] and identification of surgical instruments in laparoscopy [18]. We aim to segment surgical instruments from laparoscopy in real-time with wellestablished deep learning methods in this work.…”

Section: Related Workmentioning

confidence: 99%

Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy

Jha¹,

Ali²,

Tomar³

et al. 2021

Preprint

View full text Add to dashboard Cite

Minimally Invasive Surgery (MIS) is a surgical intervention used to examine the organs inside the abdomen and has been widely used due to its effectiveness over open surgery. Due to the hardware improvements such as high definition cameras, this procedure has significantly improved and new software methods have demonstrated potential for computer-assisted procedures. However, there exists challenges and requirement to improve detection and tracking of the position of the instruments during these surgical procedures. To this end, we evaluate and compare some popular deep learning methods that can be explored for the automated segmentation of surgical instruments in laparoscopy, an important step towards tool tracking. Our experimental results exhibit that the Dual decoder attention network (DDANet) produces a superior result compared to other recent deep learning methods. DDANet yields a Dice coefficient of 0.8739 and mean intersection-over-union of 0.8183 for the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge 2019 dataset, at a real-time speed of 101.36 framesper-second that is critical for such procedures.

show abstract

Segmentation of surgical instruments in laparoscopic videos: training dataset generation and deep-learning-based framework

Cited by 12 publications

References 6 publications

Surgical Tool Datasets for Machine Learning Research: A Survey

Surgical Tool Datasets for Machine Learning Research: A Survey

How can we learn (more) from challenges? A statistical approach to driving future algorithm development

Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy

Contact Info

Product

Resources

About