Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation

Abstract

Deep learning (DL) networks have recently been shown to outperform other segmentation methods on various public, medical-image challenge datasets [3,11,16], especially for large pathologies. However, in the context of diseases such as Multiple Sclerosis (MS), monitoring all the focal lesions visible on MRI sequences, even very small ones, is essential for disease staging, prognosis, and evaluating treatment efficacy. Moreover, producing deterministic outputs hinders DL adoption into clinical routines. Uncertainty estimates for the predictions would permit subsequent revision by clinicians. We present the first exploration of multiple uncertainty estimates based on Monte Carlo (MC) dropout [4] in the context of deep networks for lesion detection and segmentation in medical images. Specifically, we develop a 3D MS lesion segmentation CNN, augmented to provide four different voxel-based uncertainty measures based on MC dropout. We train the network on a proprietary, large-scale, multisite, multi-scanner, clinical MS dataset, and compute lesion-wise uncertainties by accumulating evidence from voxel-wise uncertainties within detected lesions. We analyze the performance of voxel-based segmentation and lesion-level detection by choosing operating points based on the uncertainty. Empirical evidence suggests that uncertainty measures consistently allow us to choose superior operating points compared only using the network's sigmoid output as a probability.

Deep Residual Learning for Neuroimaging: An application to Predict Progression to Alzheimer’s Disease

“…Peak activations of the identified brain regions which are most discriminative of progression of MCI to AD were localized by estimating occlusion sensitivity using the network occlusion approach (Zeiler and Fergus 2014). We pursued this probability-based approach to estimate and quantify the relevance of the different brain regions in the classification decisions, although few other popular approaches (Nair, Precup, Arnold, & Arbel, 2018;Zintgraf, Cohen, Adel, & Welling, 2017) could be adapted too. In this approach, brain networks in correspondence with the automated anatomical labelling (AAL) brain atlas were occluded one at a time, class probabilities re-evaluated, and the relevance of each brain region was estimated proportional to the decrease in target class probabilities when that specific region was occluded.…”

DeepCEST 3T: Robust MRI parameter determination and uncertainty quantification with neural networks—application to CEST imaging of the human brain at 3T

“…Consequently, it is highly desirable to make deep learning models probabilistic in a way that allows for uncertainty quantification, indicating how confident the NN is about predictions for a given input. Steps towards such uncertainty quantification in deep learning have been taken in the context of computer vision [19][20][21][22] as well as in medical imaging applications (e.g., in MS lesion detection, 23 melanoma detection 24 and generation of synthetic CT from MRI 25 ).…”

Automated Muscle Segmentation from Clinical CT Using Bayesian U-Net for Personalized Musculoskeletal Modeling

“…The uncertainty metrics were recently investigated by Eaton-Rosen et al [6] in a binary segmentation problem of the brain tumor, specifically for quantifying the uncertainty in volume measurement. Nair et al [22] also explored uncertainty in binary segmentation for lesion detection in multiple sclerosis. Our present work is distinct from these previous works in that we demonstrated correlation between the structure-wise uncertainty metric, namely MC sample variance, and the Dice coefficient of each structure.…”

“…It measures the degree of difference of each test sample from the training data set, originated from the deficiency of training data, namely epistemic uncertainty [15]. This method has been applied to brain lesion segmentation [6], [22] and surgical tool segmentation [10]. Two example applications of the uncertainty metric explored in this study are; 1) prediction of segmentation accuracy without using the ground truth similar to the goal of Valindria et al [35] and, 2) the active-learning framework [19], [39] for the reduction of manual annotation costs.…”