Weakly-Supervised Learning-Based Feature Localization for Confocal Laser Endomicroscopy Glioma Images

Abstract

Confocal Laser Endomicroscopy (CLE) is novel handheld fluorescence imaging technology that has shown promise for rapid intraoperative diagnosis of brain tumor tissue. Currently CLE is capable of image display only and lacks an automatic system to aid the surgeon in diagnostically analyzing the images. The goal of this project was to develop a computer-aided diagnostic approach for CLE imaging of human glioma with feature localization function. Despite the tremendous progress in object detection and image segmentation methods in recent years, most of such methods require large annotated datasets for training. However, manual annotation of thousands of histopathology images by physicians is costly and time consuming. To overcome this problem, we constructed a Weakly-Supervised Learning (WSL)-based model for feature localization that trains on image-level annotations, and then localizes incidences of a class-of-interest in the test image. We developed a novel convolutional neural network for diagnostic features localization from CLE images by employing a novel multiscale activation map that is laterally inhibited and collaterally integrated. To validate our method, we compared the model output to the manual annotation performed by four neurosurgeons on test images. The model achieved 88% mean accuracy and 86% mean intersection over union on intermediate features and 87% mean accuracy and 88% mean intersection over union on restrictive fine features, while outperforming other state of the art methods tested. This system can improve accuracy and efficiency in characterization of CLE images of glioma tissue during surgery, and may augment intraoperative decision-making regarding the tumor margin and improve brain tumor resection.

Full convolutional network based multiple side‐output fusion architecture for the segmentation of rectal tumors in magnetic resonance images: A multi‐vendor study

“…The results also showed that, compared to the models with single side-output module, the proposed model performed better, thus verifying the effectiveness of side-output fusion strategy. This conclusion was consistent with that of Izadyyazdanabadi et al 25 They proposed a novel weakly supervised learning model to localize the diagnostic features of gliomas in confocal laser endomicroscopy images. The multiscale feature learning strategy was also adopted in their study to obtain better model performance.…”

Image computing for fibre-bundle endomicroscopy: A review

“…Yet, there has been very limited work on the semantic segmentation and subsequent pathology detection and quantification for FBEµ frames and mosaics. (IIzadyyazdanabadi et al, 2018) for example proposed a weakly supervised CNN architecture for localising brain tumours in eCLE images.…”

Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract

“…artifacttainted) image parts, however, can be performed by a CLE expert. This sub-image classification can be done supervised (as proposed by Stoeve et al for motion artifacts 20 ), or also weakly supervised (as proposed by Izadyyazdanabadi et al) 21 Even for the case of malignancy classification, however, a sub-image classification would be interesting for the observer, as it helps interpretation of the image and the classification result.…”