Supervised CNN Strategies for Optical Image Segmentation and Classification in Interventional Medicine

Abstract

The analysis of interventional images is a topic of high interest for the medical-image analysis community. Such an analysis may provide interventionalmedicine professionals with both decision support and context awareness, with the final goal of improving patient safety. The aim of this chapter is to give an overview of some of the most recent approaches (up to 2018) in the field, with a focus on Convolutional Neural Networks (CNNs) for both segmentation and classification tasks. For each approach, summary tables are presented reporting the used dataset, involved anatomical region and achieved performance. Benefits and disadvantages of each approach are highlighted and discussed. Available datasets for algorithm training and testing and commonly used performance metrics are summarized to offer a source of information for researchers that are approaching the field of interventionalimage analysis. The advancements in deep learning for medical-image analysis are involving more and more the interventional-medicine field. However, these advancements are undeniably slower than in other fields (e.g. preoperative-image analysis) and considerable work still needs to be done in order to provide clinicians with all possible support during interventional-medicine procedures.

Real-time vessel segmentation and reconstruction for virtual fixtures for an active handheld microneurosurgical instrument

“…The FSCNN [13] is a FCN that combines two-branch FCN architectures such as those used in [12,16] with the encoderdecoder framework having skip connections, widely used in intra-operative image segmentation [10]. Two-branch architectures employ an encoder module with two branches: a deep branch that reduces resolution of input to learn global context and a shallow branch using full resolution input to learn the boundaries.…”

“…For intra-operative tissue segmentation, fully convolutional networks (FCNs) have gained much attention, allowing fast and accurate segmentation even in the presence of challenging vascular architectures and high noise level. Though a wealth of literature exists for segmentation of gastrointestinal lesions [3] and surgical tools [2], the use of a FCN for intraoperative segmentation of brain vasculature is investigated only in [9] due to a lack of large datasets, high inter-patient variability in vascular structure, sensor noise, inconsistent illumination, occlusion by surgical tools and low visibility in images acquired during surgery [10]. We rely on FCN-based segmentation algorithms, which are capable of dealing with such noise and of automatically learning complex features through training.…”

Deep learning using Havrda-Charvat entropy for classification of pulmonary endomicroscopy

“…Amongst them, the LeNet architecture [6] is the first successful CNN used for classify digits; AlexNet [35] was the first CNN applied to computer vision and was submitted to the ImageNet ILSVRC challenge in 2012; ZFNet is an improvement of AlexNet proposed in [36]. Many applications of deep learning for classification of optical images have already been made in [37,38,39].…”

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

“…In addition to dexterous instrumentation, CAI can enable better intervention planning and surgical navigation alongside with real-time surgical AI assistance [4]. Modules for surgical image understanding tasks such as the segmentation and tracking of surgical instruments are at the core of CAI capabilities, and recent deep learning-based approaches have shown rapid advances [5].…”