Autonomous Neurosurgical Instrument Segmentation Using End-To-End Learning

Kalavakonda, Niveditha; Hannaford, Blake; Zeeshan, Qazi; Sekhar, Laligam N.

doi:10.1109/cvprw.2019.00076

Cited by 9 publications

(15 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous surgical video analysis studies using deep learning and pixel-wise instrument segmentations were possible, achieving a fair level of accuracy with a neurosurgical microscope (Kalavakonda, 2019) or laparoscopic datasets (Kamrul Hasan and Linte, 2019). In recent studies, for surgical instrument segmentation, the mean Dice score was ∼0.769-0.9 (Kalavakonda, 2019;Kamrul Hasan and Linte, 2019). However, studies on anatomical structure segmentation using neurosurgical microscope operational video are rare (Jiang et al, 2021).…”

Section: Backgrounds Of Cerebral Blood Vessel Segmentationmentioning

confidence: 99%

“…Deep learning algorithms have been successfully applied to medical images, such as in an MRI analysis (Doke et al, 2020 ; Wang et al, 2020 ). In previous surgical video analysis studies using deep learning and pixel-wise instrument segmentations were possible, achieving a fair level of accuracy with a neurosurgical microscope (Kalavakonda, 2019 ) or laparoscopic datasets (Kamrul Hasan and Linte, 2019 ). In recent studies, for surgical instrument segmentation, the mean Dice score was ~0.769–0.9 (Kalavakonda, 2019 ; Kamrul Hasan and Linte, 2019 ).…”

Section: Introductionmentioning

confidence: 99%

“…Numerous difficulties with surgical video instruments and anatomical feature segmentation have been described in the literature, including anatomical variability, dynamical changes in three-dimensional viewpoints, few differences in texture, limited resolutions, occlusions, high tissue deformations, shadows, and reflections (Kalavakonda, 2019 ; Bamba et al, 2021 ; Bano et al, 2021 ). As a result, datasets of surgical video features are likely much more difficult to analyze than the previously investigated datasets with more constant vessels, for example, retinal blood vessels (Khanal and Estrada, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

“…Almost all previous operative anatomy segmentation studies depend on manual ground truth markings (Kalavakonda, 2019 ; Madad Zadeh et al, 2020 ; Bamba et al, 2021 ; Bano et al, 2021 ; Pangal et al, 2021 ). However, achieving the anatomical structure ground truth through manual markings is time-consuming, expensive, and potentially inaccurate.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography

et al. 2022

View full text Add to dashboard Cite

There have been few anatomical structure segmentation studies using deep learning. Numbers of training and ground truth images applied were small and the accuracies of which were low or inconsistent. For a surgical video anatomy analysis, various obstacles, including a variable fast-changing view, large deformations, occlusions, low illumination, and inadequate focus occur. In addition, it is difficult and costly to obtain a large and accurate dataset on operational video anatomical structures, including arteries. In this study, we investigated cerebral artery segmentation using an automatic ground-truth generation method. Indocyanine green (ICG) fluorescence intraoperative cerebral videoangiography was used to create a ground-truth dataset mainly for cerebral arteries and partly for cerebral blood vessels, including veins. Four different neural network models were trained using the dataset and compared. Before augmentation, 35,975 training images and 11,266 validation images were used. After augmentation, 260,499 training and 90,129 validation images were used. A Dice score of 79% for cerebral artery segmentation was achieved using the DeepLabv3+ model trained using an automatically generated dataset. Strict validation in different patient groups was conducted. Arteries were also discerned from the veins using the ICG videoangiography phase. We achieved fair accuracy, which demonstrated the appropriateness of the methodology. This study proved the feasibility of operating field view of the cerebral artery segmentation using deep learning, and the effectiveness of the automatic blood vessel ground truth generation method using ICG fluorescence videoangiography. Using this method, computer vision can discern blood vessels and arteries from veins in a neurosurgical microscope field of view. Thus, this technique is essential for neurosurgical field vessel anatomy-based navigation. In addition, surgical assistance, safety, and autonomous surgery neurorobotics that can detect or manipulate cerebral vessels would require computer vision to identify blood vessels and arteries.

show abstract

Section: Backgrounds Of Cerebral Blood Vessel Segmentationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The challenge in this project has been to decipher how the surgeon interacts with the assistant during an operation. The team has been recording the surgeon's voice and tool movement under the microscope and deciphering them using convolutional neural networks 56,57 and Python speech application program interfaces, respectively. The instruments in a surgical field are visually identified based on their type and tracked across the frames at a pixel level 58 (Figure 10).…”

Section: Literature Reviewmentioning

confidence: 99%

The Future of Skull Base Surgery: A View Through Tinted Glasses

et al. 2020

Self Cite

View full text Add to dashboard Cite

Elsevier has created a COVID-19 resource centre with free information in English and Mandarin on the novel coronavirus COVID-19. The COVID-19 resource centre is hosted on Elsevier Connect, the company's public news and information website. Elsevier hereby grants permission to make all its COVID-19-related research that is available on the COVID-19 resource centre-including this research content-immediately available in PubMed Central and other publicly funded repositories, such as the WHO COVID database with rights for unrestricted research re-use and analyses in any form or by any means with acknowledgement of the original source. These permissions are granted for free by Elsevier for as long as the COVID-19 resource centre remains active.

show abstract

Surgical Tool Datasets for Machine Learning Research: A Survey

2022

View full text Add to dashboard Cite

This paper is a comprehensive survey of datasets for surgical tool detection and related surgical data science and machine learning techniques and algorithms. The survey offers a high level perspective of current research in this area, analyses the taxonomy of approaches adopted by researchers using surgical tool datasets, and addresses key areas of research, such as the datasets used, evaluation metrics applied and deep learning techniques utilised. Our presentation and taxonomy provides a framework that facilitates greater understanding of current work, and highlights the challenges and opportunities for further innovative and useful research.

show abstract

Autonomous Neurosurgical Instrument Segmentation Using End-To-End Learning

Cited by 9 publications

References 4 publications

Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography

Deep-Learning-Based Cerebral Artery Semantic Segmentation in Neurosurgical Operating Microscope Vision Using Indocyanine Green Fluorescence Videoangiography

The Future of Skull Base Surgery: A View Through Tinted Glasses

Surgical Tool Datasets for Machine Learning Research: A Survey

Contact Info

Product

Resources

About