Multilabel multiclass classification of OCT images augmented with age, gender and visual acuity data

Mehta, Parmita; Lee, Anna; C, Lee; Balazinska, Magdalena; Rokem, Ariel

doi:10.1101/316349

Cited by 10 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, all weights were unfrozen and the network was retrained for 20 epochs with the same callbacks and learning rate. As suggested in the literature, 22 training from scratch on OCT images may be preferable as many of the low-level filters in networks pretrained on natural images are tuned to colors and OCT images are monochromatic. However, retraining the entire network preinitialized on ImageNet provided us with better performance than training from uninitialized weights, likely due to our significantly smaller dataset.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…Class activation maps (CAMs) 20 are a common method where a heat map is generated by projecting the class specific weights of the output classification layer back to the feature maps of the last convolutional layer, thereby highlighting important regions for predicting a particular class. This method has been used in ophthalmic application previously to confirm CNN decision was based off the anterior chamber angle in categorizing angle closure, 21 areas of OCT B-scans associated with various diagnoses 22 , 23 and areas of segmentation error, 24 and area of OCT enface images associated with the diagnosis of glaucoma. 25 There exists several variants of this method that build off of the original CAM paper, 20 including: Grad-Cam, 26 Guided Grad-Cam, 26 Guided Grad-Cam++, 27 and GAIN.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography

Heisler

Karst

et al. 2020

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

To evaluate the role of ensemble learning techniques with deep learning in classifying diabetic retinopathy (DR) in optical coherence tomography angiography (OCTA) images and their corresponding co-registered structural images. Methods: A total of 463 volumes from 380 eyes were acquired using the 3 × 3-mm OCTA protocol on the Zeiss Plex Elite system. Enface images of the superficial and deep capillary plexus were exported from both the optical coherence tomography and OCTA data. Component neural networks were constructed using single data-types and fine-tuned using VGG19, ResNet50, and DenseNet architectures pretrained on ImageNet weights. These networks were then ensembled using majority soft voting and stacking techniques. Results were compared with a classifier using manually engineered features. Class activation maps (CAMs) were created using the original CAM algorithm and Grad-CAM. Results: The networks trained with the VGG19 architecture outperformed the networks trained on deeper architectures. Ensemble networks constructed using the four finetuned VGG19 architectures achieved accuracies of 0.92 and 0.90 for the majority soft voting and stacking methods respectively. Both ensemble methods outperformed the highest single data-type network and the network trained on hand-crafted features. Grad-CAM was shown to more accurately highlight areas of disease. Conclusions: Ensemble learning increases the predictive accuracy of CNNs for classifying referable DR on OCTA datasets. Translational Relevance: Because the diagnostic accuracy of OCTA images is shown to be greater than the manually extracted features currently used in the literature, the proposed methods may be beneficial toward developing clinically valuable solutions for DR diagnoses.

show abstract

Section: Experimental Settingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography

Heisler

Karst

et al. 2020

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

show abstract

“…Marzieh Mokhtari et al [20] calculate local cup to disc ratio by fusing fundus images and OCT B-scans to get the symmetry of two eyes, which can detect early ocular diseases better. Mehta et al [19] proposed a OCT images system for multi-class, multi-label classification, which augments data by using patient information, such as age, gender and visual acuity data.…”

Section: Oct Images Recognitionmentioning

confidence: 99%

A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection

et al. 2021

Preprint

View full text Add to dashboard Cite

In ophthalmology, early fundus screening is an economic and effective way to prevent blindness caused by ophthalmic diseases. Clinically, due to the lack of medical resources, manual diagnosis is timeconsuming and may delay the condition. With the development of deep learning, some researches on ophthalmic diseases have achieved good results, however, most of them are just based on one disease. During fundus screening, ophthalmologists usually give diagnoses of multi-disease on binocular fundus image, so we release a dataset with 8 diseases to meet the real medical scene, which contains 10,000 fundus images from both eyes of 5,000 patients. We did some benchmark experiments on it through some state-of-the-art deep neural networks. We found simply increasing the scale of network cannot bring good results for multi-disease classification, and a well-structured feature fusion method combines characteristics of multi-disease is needed. Through this work, we hope to advance the research of related fields.

show abstract

“…Since the choice for the label is from a set of multiple descriptors (classes), the term multiclass is used [10]. Oftentimes, multilabels are treated as subsets of multiclass labeling [3], [4], [5].…”

Section: Related Workmentioning

confidence: 99%

“…In the context of videos, our multilabel vector represents distinct spatial regions (i.e. regions of interest) of an image frame, unlike common uses of the multilabel vector as a representation of the presence or absence of multiple descriptors of interest [3], [4], [5]. Each region is assigned one of three possible levels of activity to train and validate our deep learning model.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous and Spatiotemporal Detection of Different Levels of Activity in Multidimensional Data

et al. 2020

View full text Add to dashboard Cite

In this work, we present a novel and promising approach to autonomously detect different levels of simultaneous and spatiotemporal activity in multidimensional data. We introduce a new multilabeling technique, which assigns different labels to different regions of interest in the data, and thus, incorporates the spatial aspect. Each label is built to describe the level of activity/motion to be monitored in the spatial location that it represents, in contrast to existing approaches providing only a binary result as the presence or absence of activity. This novel Spatially and Motion-Level Descriptive (SMLD) labeling schema is combined with a Convolutional Long Short Term Memory-based network for classification to capture different levels of activity both spatially and temporally without the use of any foreground or object detection. The proposed approach can be applied to various types of spatiotemporal data captured for completely different application domains. In this paper, it was evaluated on video data as well as respiratory sound data. Metrics commonly associated with multilabeling, namely Hamming Loss and Subset Accuracy, as well as confusion matrix-based measurements are used to evaluate performance. Promising testing results are achieved with an overall Hamming Loss for video datasets close to 0.05, Subset Accuracy close to 80% and confusion matrix-based metrics above 0.9. In addition, our proposed approach's ability in detecting frequent motion patterns based on predicted spatiotemporal activity levels is discussed. Encouraging results have been obtained on the respiratory sound dataset as well, while detecting abnormalities in different parts of the lungs. The experimental results demonstrate that the proposed approach can be applied to various types of spatiotemporal data captured for different application domains.

show abstract

Multilabel multiclass classification of OCT images augmented with age, gender and visual acuity data

Cited by 10 publications

References 8 publications

Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography

Ensemble Deep Learning for Diabetic Retinopathy Detection Using Optical Coherence Tomography Angiography

A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection

Simultaneous and Spatiotemporal Detection of Different Levels of Activity in Multidimensional Data

Contact Info

Product

Resources

About