Unsupervised lesion detection via image restoration with a normative prior

Chen, Xiaoran; You, Suhang; Tezcan, Kerem Can; Konukoğlu, Ender

doi:10.1016/j.media.2020.101713

Cited by 86 publications

(66 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This demonstrates the effectiveness of our approach. Comparing our results on the BraTS 2019 data set with other works that utilize additional image information, e.g., T2-weighted data [8,22], highlights the advantage of additional image information. Similar, we observe immediate performance improvement for our methods when evaluated on T1ceweighted data, despite the domain adaption from T1, see Table 2.…”

Section: Discussionsupporting

confidence: 59%

Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI

et al. 2021

View full text Add to dashboard Cite

Purpose Brain Magnetic Resonance Images (MRIs) are essential for the diagnosis of neurological diseases. Recently, deep learning methods for unsupervised anomaly detection (UAD) have been proposed for the analysis of brain MRI. These methods rely on healthy brain MRIs and eliminate the requirement of pixel-wise annotated data compared to supervised deep learning. While a wide range of methods for UAD have been proposed, these methods are mostly 2D and only learn from MRI slices, disregarding that brain lesions are inherently 3D and the spatial context of MRI volumes remains unexploited. Methods We investigate whether using increased spatial context by using MRI volumes combined with spatial erasing leads to improved unsupervised anomaly segmentation performance compared to learning from slices. We evaluate and compare 2D variational autoencoder (VAE) to their 3D counterpart, propose 3D input erasing, and systemically study the impact of the data set size on the performance. Results Using two publicly available segmentation data sets for evaluation, 3D VAEs outperform their 2D counterpart, highlighting the advantage of volumetric context. Also, our 3D erasing methods allow for further performance improvements. Our best performing 3D VAE with input erasing leads to an average DICE score of 31.40% compared to 25.76% for the 2D VAE. Conclusions We propose 3D deep learning methods for UAD in brain MRI combined with 3D erasing and demonstrate that 3D methods clearly outperform their 2D counterpart for anomaly segmentation. Also, our spatial erasing method allows for further performance improvements and reduces the requirement for large data sets.

show abstract

Section: Discussionsupporting

confidence: 59%

Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The latest advances in deep learning, mostly GANs [ 8 ] and VAEs [ 42 ], have allowed for the accurate estimation of the high-dimensional healthy distributions. Except for discriminative boundary-based approaches including [ 43 ], almost all unsupervised medical anomaly detection studies have leveraged reconstruction: as pioneering research, Schlegl et al proposed AnoGAN to detect outliers in the learned feature space of the GAN [ 44 ]; then, the same authors presented fast AnoGAN that can efficiently map query images onto the latent space [ 14 ]; since the reconstruction-based models often suffer from many false positives, Chen et al penalized large deviations between original/reconstructed images in gliomas and stroke lesion detection on brain MRI [ 45 ]. However, to the best of our knowledge, all previous studies are based on 2D/3D single image reconstruction, without considering continuity between multiple adjacent slices.…”

Section: Related Workmentioning

confidence: 99%

MADGAN: unsupervised medical anomaly detection GAN using multiple adjacent brain MRI slice reconstruction

Han¹,

et al. 2021

View full text Add to dashboard Cite

Background Unsupervised learning can discover various unseen abnormalities, relying on large-scale unannotated medical images of healthy subjects. Towards this, unsupervised methods reconstruct a 2D/3D single medical image to detect outliers either in the learned feature space or from high reconstruction loss. However, without considering continuity between multiple adjacent slices, they cannot directly discriminate diseases composed of the accumulation of subtle anatomical anomalies, such as Alzheimer’s disease (AD). Moreover, no study has shown how unsupervised anomaly detection is associated with either disease stages, various (i.e., more than two types of) diseases, or multi-sequence magnetic resonance imaging (MRI) scans. Results We propose unsupervised medical anomaly detection generative adversarial network (MADGAN), a novel two-step method using GAN-based multiple adjacent brain MRI slice reconstruction to detect brain anomalies at different stages on multi-sequence structural MRI: (Reconstruction) Wasserstein loss with Gradient Penalty + 100 $$\ell _1$$ ℓ 1 loss—trained on 3 healthy brain axial MRI slices to reconstruct the next 3 ones—reconstructs unseen healthy/abnormal scans; (Diagnosis) Average $$\ell _2$$ ℓ 2 loss per scan discriminates them, comparing the ground truth/reconstructed slices. For training, we use two different datasets composed of 1133 healthy T1-weighted (T1) and 135 healthy contrast-enhanced T1 (T1c) brain MRI scans for detecting AD and brain metastases/various diseases, respectively. Our self-attention MADGAN can detect AD on T1 scans at a very early stage, mild cognitive impairment (MCI), with area under the curve (AUC) 0.727, and AD at a late stage with AUC 0.894, while detecting brain metastases on T1c scans with AUC 0.921. Conclusions Similar to physicians’ way of performing a diagnosis, using massive healthy training data, our first multiple MRI slice reconstruction approach, MADGAN, can reliably predict the next 3 slices from the previous 3 ones only for unseen healthy images. As the first unsupervised various disease diagnosis, MADGAN can reliably detect the accumulation of subtle anatomical anomalies and hyper-intense enhancing lesions, such as (especially late-stage) AD and brain metastases on multi-sequence MRI scans.

show abstract

“…The authors also applied the same methodology to brain MRI, with similar results. An AAE is employed in ( Chen et al , 2020 ) to learn the distribution of healthy subject brain MRI images, and is then applied to test images of brain MRI containing lesions. The difference images between input and AAE reconstruction successfully localize lesions in the test images.…”

Section: Understanding Model Structure and Functionmentioning

confidence: 99%

Interpretation and visualization techniques for deep learning models in medical imaging

2021

View full text Add to dashboard Cite

Deep learning approaches to medical image analysis tasks have recently become popular; however, they suffer from a lack of human interpretability critical for both increasing understanding of the methods’ operation and enabling clinical translation. This review summarizes currently available methods for performing image model interpretation and critically evaluates published uses of these methods for medical imaging applications. We divide model interpretation in two categories: (1) understanding model structure and function and (2) understanding model output. Understanding model structure and function summarizes ways to inspect the learned features of the model and how those features act on an image. We discuss techniques for reducing the dimensionality of high-dimensional data and cover autoencoders, both of which can also be leveraged for model interpretation. Understanding model output covers attribution-based methods, such as saliency maps and class activation maps, which produce heatmaps describing the importance of different parts of an image to the model prediction. We describe the mathematics behind these methods, give examples of their use in medical imaging, and compare them against one another. We summarize several published toolkits for model interpretation specific to medical imaging applications, cover limitations of current model interpretation methods, provide recommendations for deep learning practitioners looking to incorporate model interpretation into their task, and offer general discussion on the importance of model interpretation in medical imaging contexts.

show abstract

Unsupervised lesion detection via image restoration with a normative prior

Cited by 86 publications

References 25 publications

Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI

Three-dimensional deep learning with spatial erasing for unsupervised anomaly segmentation in brain MRI

MADGAN: unsupervised medical anomaly detection GAN using multiple adjacent brain MRI slice reconstruction

Interpretation and visualization techniques for deep learning models in medical imaging

Contact Info

Product

Resources

About