Leveraging Regular Fundus Images for Training UWF Fundus Diagnosis Models via Adversarial Learning and Pseudo-Labeling

Ju, Lie; Wang, Xin; Zhao, Xin; Bonnington, C. Paul; Drummond, Tom; Ge, Zongyuan

doi:10.1109/tmi.2021.3056395

Cited by 41 publications

(21 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has not been confirmed whether data augmentation through GAN can increase the diagnostic accuracy of AI systems for ocular diseases in real clinics. A GAN can be used to bridge the domain gap between training and external data from different sources [ 64 ]. If difficult access to reliable annotated data from multiple data sources remains problematic, domain adaptation can be considered to address the generalization issue [ 87 ].…”

Section: Discussionmentioning

confidence: 99%

“…Domain adaptation via the domain transfer function of a GAN may provide a chance to use a machine learning system in different settings. For example, retinal images taken with ultra-widefield fundus photography can be analyzed by an AI system developed with FP via domain transfer using GAN [ 63 , 64 ]. Although GAN has several shortcomings, its ability to adapt to domains and expand data by generating realistic images can increase generalizability and may help to increase the use of machine learning algorithms in ophthalmology image domains.…”

Section: Discussionmentioning

confidence: 99%

“…Autoencoder was used to synthesize new retinal vessel images apart from training of GAN Zhao et al [ 59 ] Conditional GAN Vessel image → Fundus photography Retinal image synthesis can be effectively learned in a data-driven fashion from a relatively small sample size using a conditional GAN architecture Yu et al [ 60 ] Pix2pix (with ResU-net generator) (conditional GAN) Vessel image → Fundus photography To enlarge training datasets for facilitating medical image analysis, the multiple-channels-multiple-landmarks (MCML) was developed to synthesize color fundus images from a combination of vessel and optic disc masked images Wu et al [ 61 ] Conditional GAN Volumetric retinal OCT → Fundus autofluorescence The en-face OCT images were synthesized from volumetric retinal OCT by restricted summed voxel projection. The fundus autofluorescence images were generated from en-face OCT images using GAN to identify the geographic atrophy region Tavakkoli et al [ 62 ] Conditional GAN Fundus photography → Fluorescein angiography The proposed GAN produced anatomically accurate fluorescein angiography images that were indistinguishable from real angiograms Yoo et al [ 63 ] CycleGAN Ultra-widefield fundus photography → Fundus photography Ultra-widefield images were successfully translated into traditional fundus photography-style images by CycleGAN, and the main structural information of the retina and optic nerve was retained Ju et al [ 64 ] CycleGAN Fundus photography → Ultra-widefield fundus photography The CycleGAN model transferred the color fundus photographs to ultra-widefield images to introduce additional data for existing limited ultra-widefield images. The proposed method was adopted for diabetic retinopathy grading and lesion detection Lazaridis et al [ 91 , 108 ] Wasserstein GAN + perceptual loss (conditional GAN) Time-domain OCT → spectral-domain OCT Time-domain OCT was converted to synthetic spectral-domain OCT using GAN.…”

Section: Reviewmentioning

confidence: 99%

“…Based on CycleGAN, ultra-widefield fundus photography can be transformed into classic color fundus photography to integrate retinal imaging domains [ 63 ]. In contrast, a study converting classic fundus photography to ultra-widefield fundus photography via CycleGAN was also reported [ 64 ]. Larzaridis et al demonstrated that time-domain OCT could be converted to spectral-domain OCT using conditional GAN with Wasserstein distance and perceptual loss, showing that the integrated dataset fused by the GAN improved the statistical power of the OCT measurements.…”

Section: Reviewmentioning

confidence: 99%

See 3 more Smart Citations

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

et al. 2022

View full text Add to dashboard Cite

Background Recent advances in deep learning techniques have led to improved diagnostic abilities in ophthalmology. A generative adversarial network (GAN), which consists of two competing types of deep neural networks, including a generator and a discriminator, has demonstrated remarkable performance in image synthesis and image-to-image translation. The adoption of GAN for medical imaging is increasing for image generation and translation, but it is not familiar to researchers in the field of ophthalmology. In this work, we present a literature review on the application of GAN in ophthalmology image domains to discuss important contributions and to identify potential future research directions. Methods We performed a survey on studies using GAN published before June 2021 only, and we introduced various applications of GAN in ophthalmology image domains. The search identified 48 peer-reviewed papers in the final review. The type of GAN used in the analysis, task, imaging domain, and the outcome were collected to verify the usefulness of the GAN. Results In ophthalmology image domains, GAN can perform segmentation, data augmentation, denoising, domain transfer, super-resolution, post-intervention prediction, and feature extraction. GAN techniques have established an extension of datasets and modalities in ophthalmology. GAN has several limitations, such as mode collapse, spatial deformities, unintended changes, and the generation of high-frequency noises and artifacts of checkerboard patterns. Conclusions The use of GAN has benefited the various tasks in ophthalmology image domains. Based on our observations, the adoption of GAN in ophthalmology is still in a very early stage of clinical validation compared with deep learning classification techniques because several problems need to be overcome for practical use. However, the proper selection of the GAN technique and statistical modeling of ocular imaging will greatly improve the performance of each image analysis. Finally, this survey would enable researchers to access the appropriate GAN technique to maximize the potential of ophthalmology datasets for deep learning research.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Reviewmentioning

confidence: 99%

Section: Reviewmentioning

confidence: 99%

See 2 more Smart Citations

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

et al. 2022

View full text Add to dashboard Cite

show abstract

“…1(d) can be obtained at 200°, allowing imaging of approximately 80% of the retinal fundus. In addition, it can show both posterior pole and peripheral retinal images without dilating the patient's pupil, which can reveal occult peripheral retinal ischemia or neovascularisation, thus more effectively guiding relevant treatment plans and expanding clinicians' knowledge of the peripheral retina [7]. However, these differences between fundus datasets will lead to substantial performance degradation when transferring a deep learning model trained on one fundus dataset to another, since a highperformance deep learning model requires that the train data and test data are drawn from the same distribution [8].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Domain Adaptation for Retinal Vessel Segmentation with Adversarial Learning and Transfer Normalization

Feng,

Ju,

Wang

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Retinal vessel segmentation plays a key role in computer-aided screening, diagnosis, and treatment of various cardiovascular and ophthalmic diseases. Recently, deep learning-based retinal vessel segmentation algorithms have achieved remarkable performance. However, due to the domain shift problem, the performance of these algorithms often degrades when they are applied to new data that is different from the training data. Manually labeling new data for each test domain is often a timeconsuming and laborious task. In this work, we explore unsupervised domain adaptation in retinal vessel segmentation by using entropy-based adversarial learning and transfer normalization layer to train a segmentation network, which generalizes well across domains and requires no annotation of the target domain. Specifically, first, an entropy-based adversarial learning strategy is developed to reduce the distribution discrepancy between the source and target domains while also achieving the objective of entropy minimization on the target domain. In addition, a new transfer normalization layer is proposed to further boost the transferability of the deep network. It normalizes the features of each domain separately to compensate for the domain distribution gap. Besides, it also adaptively selects those feature channels that are more transferable between domains, thus further enhancing the generalization performance of the network. We conducted extensive experiments on three regular fundus image datasets and an ultra-widefield fundus image dataset, and the results show that our approach yields significant performance gains compared to other state-of-the-art methods.

show abstract

Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling

Chen

Liu

Jin

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Leveraging Regular Fundus Images for Training UWF Fundus Diagnosis Models via Adversarial Learning and Pseudo-Labeling

Cited by 41 publications

References 44 publications

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

Application of generative adversarial networks (GAN) for ophthalmology image domains: a survey

Unsupervised Domain Adaptation for Retinal Vessel Segmentation with Adversarial Learning and Transfer Normalization

Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling

Contact Info

Product

Resources

About