MultiSDGAN: Translation of OCT Images to Superresolved Segmentation Labels Using Multi-Discriminators in Multi-Stages

Abstract

Optical coherence tomography (OCT) has been identified as a non-invasive and inexpensive imaging modality to discover potential biomarkers for Alzheimer's diagnosis and progress determination. Current hypotheses presume the thickness of the retinal layers, which are analyzable within OCT scans, as an effective biomarker for the presence of Alzheimer's. As a logical first step, this work concentrates on the accurate segmentation of retinal layers to isolate the layers for further analysis. This paper proposes a generative adversarial network (GAN) that concurrently learns to increase the image resolution for higher clarity and then segment the retinal layers. We propose a multi-stage & multi-discriminatory generative adversarial network (MultiSDGAN) specifically for superresolution and segmentation of OCT scans of the retinal layer. The resulting generator is adversarially trained against multiple discriminator networks at multiple stages. We aim to avoid early saturation of generator model training leading to poor segmentation accuracies and enhance the process of OCT domain translation by satisfying all the discriminators in multiple scales. We also investigated incorporating the Dice loss and Structured Similarity Index Measure (SSIM) as additional loss functions to specifically target and improve our proposed GAN architecture's segmentation and superresolution performance, respectively. The ablation study results conducted on our data set suggest that the proposed MultiSDGAN with ten-fold cross-validation (10-CV) provides a reduced equal error rate with 44.24% and 34.09% relative improvements, respectively (p-values of the improvement level tests <.01). Furthermore, our experimental results also demonstrate that the addition of the new terms to the loss function improves the segmentation results significantly by relative improvements of 31.33% (p-value<.01).

Superresolution and Segmentation of OCT Scans Using Multi-Stage Adversarial Guided Attention Training

“…Similar to the MultiSDGAN model [1], during the training, the proposed network weights are updated based on the Dice, SSIM and L-1 losses that are obtained via the following formulas. y is our ground truth and G(x) is the generator output.…”

“…Conditional GANs depict good performance translating data from one domain to another [9], [10], thus it is appropriate for semantic segmentation. In our previous works, we proposed a GAN-based domain translation and superresolution architecture that learns to increase the medical image resolution from low to high and learn to segment the retinal layers at the same time [11,12,1]. This particular type of GAN considers multiple stages of output from different layers of the network.…”

“…This particular type of GAN considers multiple stages of output from different layers of the network. Each intermediary output from the multi-stage is subjected to various discriminators [1].…”

“…In this paper, we aim to investigate the impact of incorporating attention mechanism in the MultiSDGAN framework [1]. We aim to evaluate and compare various combinations of channel and spatial attention to capture rich contextual relationships to extract more powerful feature maps and improve segmentation outcome.…”

“…This work aims to segment the OCT images automatically; however, it is a challenging task due to various issues such as the speckle noise, small target region, and unfavorable imaging conditions. In our previous work, we have proposed the multi-stage & multi-discriminatory generative adversarial network (MultiSDGAN) [1] to translate OCT scans in highresolution segmentation labels. In this investigation, we aim to evaluate and compare various combinations of channel and spatial attention to the MultiSDGAN architecture to extract more powerful feature maps by capturing rich contextual relationships to improve segmentation performance.…”