Accurate tissue interface segmentation via adversarial pre-segmentation of anterior segment OCT images

Abstract

Optical Coherence Tomography (OCT) is an imaging modality that has been widely adopted for visualizing corneal, retinal and limbal tissue structure with micron resolution. It can be used to diagnose pathological conditions of the eye, and for developing pre-operative surgical plans. In contrast to the posterior retina, imaging the anterior tissue structures, such as the limbus and cornea, results in B-scans that exhibit increased speckle noise patterns and imaging artifacts. These artifacts, such as shadowing and specularity, pose a challenge during the analysis of the acquired volumes as they substantially obfuscate the location of tissue interfaces. To deal with the artifacts and speckle noise patterns and accurately segment the shallowest tissue interface, we propose a cascaded neural network framework, which comprises of a conditional Generative Adversarial Network (cGAN) and a Tissue Interface Segmentation Network (TISN). The cGAN pre-segments OCT B-scans by removing undesired specular artifacts and speckle noise patterns just above the shallowest tissue interface, and the TISN combines the original OCT image with the pre-segmentation to segment the shallowest interface. We show the applicability of the cascaded framework to corneal datasets, demonstrate that it precisely segments the shallowest corneal interface, and also show its generalization capacity to limbal datasets. We also propose a hybrid framework, wherein the cGAN pre-segmentation is passed to a traditional image analysis-based segmentation algorithm, and describe the improved segmentation performance. To the best of our knowledge, this is the first approach to remove severe specular artifacts and speckle noise patterns (prior to the shallowest interface) that affects the interpretation of anterior segment OCT datasets, thereby resulting in the accurate segmentation of the shallowest tissue interface. To the best of our knowledge, this is the first work to show the potential of incorporating a cGAN into larger deep learning frameworks for improved corneal and limbal OCT image segmentation. Our cGAN design directly improves the visualization of corneal and limbal OCT images from OCT scanners, and improves the performance of current OCT segmentation algorithms.

Dissecting the Profile of Corneal Thickness With Keratoconus Progression Based on Anterior Segment Optical Coherence Tomography

“…The main obstacle to accurate evaluation of the thickness is precise outlining of the corneal tissue boundaries. Besides the manual segmentation or semi-automated traditional methods (Chen et al, 2012;Corre-Perez et al, 2012;Li et al, 2012;Xu et al, 2016;Ang et al, 2018;Morishige et al, 2019;Yang et al, 2020;Toprak et al, 2021), deep learning-based methods have been proposed for corneal tissue interface segmentation (Mathai et al, 2019;Ouyang et al, 2019;Santos et al, 2019). In our previous studies, we compared these methods with our proposed hierarchyconstrained segmentation network (Liu et al, 2020) and validated the effectiveness of our network architecture and boundary constraint.…”

“…Whereas manual labeling is time-consuming and has poor repeatability, the traditional image processing methods are less robust to deal with pathological corneas (Larocca et al, 2011;Williams et al, 2015;Ang et al, 2018;Elsawy et al, 2019). Recent studies have explored the feasibility of using deep learning-based methods for corneal tissue segmentation with AS-OCT images (Mathai et al, 2019;Ouyang et al, 2019;Santos et al, 2019). We have also proposed a hierarchy-constrained network, which robustly improves the segmentation performance of the corneal tissue interfaces in both normal and KC eyes (Liu et al, 2020).…”

Inpainting for Saturation Artifacts in Optical Coherence Tomography Using Dictionary-Based Sparse Representation

“…An important application of this work is to better the accuracy of segmentation. In clinic diagnosis, accurate estimation of tissue boundaries is crucial to determine the quantitative parameter for treatment [ 24 ]. With the existence of saturation pixels, the pixel information needed to estimate boundaries is blurry.…”

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

“…The method improved the image quality metrics for OCT Mahapatra et al [ 55 ] PGGAN with a conditional design Fundus photography Super-resolution Image super-resolution using multi-stage PGGAN outperforms competing methods and baseline GANs. The super-resolved images can be used for landmark and pathology detection Huang et al [ 49 ] Conditional GAN Retinal OCT Super-resolution and removing noise The GAN model effectively suppressed speckle noise and super-resolved OCT images at different scales Ouyang et al [ 51 ] Conditional GAN Anterior Segment OCT Removing speckle noise The model removed undesired specular artifacts and speckle-noise patterns to improve the visualization of corneal and limbal OCT images Yoo et al [ 53 ] CycleGAN Fundus photography Removing artifacts and noise The GAN model removed the artifacts automatically in a fundus photograph without matching paired images Cheong et al [ 16 ] DeshadowGAN (modified conditional GAN with perceptual loss) Peripapillary retinal OCT (spectral domain) Removing vessel shadow artifacts The GAN model using manually masked artifact images and perceptual loss function removed blood vessel shadow artifacts from OCT images of the optic nerve head Chen et al [ 50 ] Conditional GAN Peripapillary retinal OCT (spectral domain) Removing speckle noise The GAN model was designed for speckle noise reduction in OCT images and preserved the textural details found in OCT Das et al [ 52 ] CycleGAN Retinal OCT Super-resolution and removing noise To achieve denoising and super-resolution, adversarial learning with cycle consistency was used without requiring aligned low–high resolution pairs …”

“…Cheong et al built DeShadowGAN using manually masked artifact images and conditional GAN with perceptual loss and demonstrated the effectiveness of the model in removing shadow artifacts [ 16 ]. Similarly, conditional GAN has also been applied to remove speckle noise in peripapillary retinal OCT [ 50 ] and anterior segment OCT [ 51 ]. However, image denoising methods using conditional GAN can match low- and high-quality image pairs; however, these data are typically unavailable in the medical field.…”