Gradient-Based Severity Labeling for Biomarker Classification in Oct

Abstract

In this paper, we propose a novel selection strategy for contrastive learning for medical images. On natural images, contrastive learning uses augmentations to select positive and negative pairs for the contrastive loss. However, in the medical domain, arbitrary augmentations have the potential to distort small localized regions that contain the biomarkers we are interested in detecting. A more intuitive approach is to select samples with similar disease severity characteristics, since these samples are more likely to have similar structures related to the progression of a disease. To enable this, we introduce a method that generates disease severity labels for unlabeled OCT scans on the basis of gradient responses from an anomaly detection algorithm. These labels are used to train a supervised contrastive learning setup to improve biomarker classification accuracy by as much as 6% above self-supervised baselines for key indicators of Diabetic Retinopathy.

Deep learning for retina structural biomarker classification using OCT images

“…It is applied using different data argument methods to manipulate the image, feeding into the feature extraction network, which outputs the feature map. Moreover, contrast loss 19 between image features is calculated to train the feature extraction network for downstream classification tasks. In the phase of semi-supervised training, we utilize the trained network to predict the classification results of the test set and the results with prediction confidence greater than a certain threshold as pseudo-labels, which were added to training in fine-tuned parameters and a lower learning rate.…”

Contrastive Explanations In Neural Networks

“…Recently, a number of works have used gradients as features to characterize data as a function of network weights. This characterizations has shown promising results in a number of disparate applications including novelty [18], anomaly [17], and adversarial image detection [35], severity detection [20], image quality assessment [16], and human visual saliency detection [47]. A number of theories as to their efficacy has been put forward including neurobiological [48], behavioral [29], and reasoningbased [30].…”

“…The complexity of visual tasks tackled by neural networks range from microscopic textures [2] to earth's subsurface [3]. They are used in sensitive tasks like detecting cardiovascular [4] and diabetic [5] risk factors in humans through retinal images. Hence, the utility of neural networks in such wide-ranging and sensitive applications call for explainability regarding their decisions.…”

“…Gradients provide an intelligent and intuitive search direction to stochastically update the network parameters towards the minima. Recently, loss gradients have been utilized at inference to characterize data [16]- [20]. Specific to explanations, a number of gradient-based methods have been proposed including Grad-CAM [6], GradCAM++ [21], Guided Backpropagation [9], and SmoothGrad [10] among others.…”

Exploiting the Distortion-Semantic Interaction in Fisheye Data

“…While the data space reflects the distortion characteristic of the data, standard methods do not integrate distortion into the training paradigm and rely on just semantic labels Y C with an associated semantic-based loss L C . However, previous work [14], [15], [16] has shown that a model lacks generalization capability when the learnt representation does not reflect the underlying distribution of the data space. We argue in this work that the underlying distribution of fisheye data reflects not only semantic context or distortion alone, but a complex interaction between both.…”