Optimal Transport driven CycleGAN for Unsupervised Learning in Inverse Problems

Sim, Byeongsu; Oh, Gyutaek; Kim, Jeongsol; Jung, Chan-Yong; Ye, Jong Chul

doi:10.48550/arxiv.1909.12116

Cited by 17 publications

(51 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also possible to train a GAN using model-based costs. In [75], an MBIR cost that embeds a generative network is used to formulate a training loss for the generator based on optimal transport [83]. Particularly, the generator takes measurements as inputs and outputs images, which models the inverse path of the imaging problem.…”

Section: Generative Adversarial Network (Gan) Based Methodsmentioning

confidence: 99%

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Huang,

Ye,

McCann

et al. 2021

Preprint

View full text Add to dashboard Cite

Many techniques have been proposed for image reconstruction in medical imaging that aim to recover high-quality images especially from limited or corrupted measurements. Model-based reconstruction methods have been particularly popular (e.g., in magnetic resonance imaging and tomographic modalities) and exploit models of the imaging system's physics together with statistical models of measurements, noise and often relatively simple object priors or regularizers. For example, sparsity or low-rankness based regularizers have been widely used for image reconstruction from limited data such as in compressed sensing. Learning-based approaches for image reconstruction have garnered much attention in recent years and have shown promise across biomedical imaging applications. These methods include synthesis dictionary learning, sparsifying transform learning, and different forms of deep learning involving complex neural networks. We briefly discuss classical model-based reconstruction methods and then review reconstruction methods at the intersection of model-based and learning-based paradigms in detail. This review includes many recent methods based on unsupervised learning, and supervised learning, as well as a framework to combine multiple types of learned models together.

show abstract

Section: Generative Adversarial Network (Gan) Based Methodsmentioning

confidence: 99%

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Huang,

Ye,

McCann

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Here we briefly review the OT-driven cycleGAN design in our companion paper [24], which is used for our unsupervised learning method.…”

Section: A Optimal Transport Driven Cycleganmentioning

confidence: 99%

“…where the minimum is taken over the joint distribution π(x, y) whose marginal distribution with respect to X and Y is µ and ν, respectively. Another important discovery in our companion paper [24] is that the resulting primal problem can be equivalently represented by the Kantorovich dual formulation [21], [22]:…”

Section: A Optimal Transport Driven Cycleganmentioning

confidence: 99%

“…Here, the term cycle is the cycle-consistency loss, W GAN is the Wasserstein GAN (WGAN) loss [26], and ϕ Υ is often called the discriminator. It is important to note that we just have one discriminator in (7), since H is a known deterministic generator and we only require one single CNN generator G Θ (y) [24]. Interestingly, the resulting dual formulation shows that the physics-driven data consistency term is used to stabilize the training of the neural network.…”

Section: A Optimal Transport Driven Cycleganmentioning

confidence: 99%

“…To address this issue, here we propose a novel cycleGAN architecture inspired by the optimal transport (OT) theory [21]- [24]. One of the important advantages of our cycleGAN is that the necessity of obtaining fast-temporal resolution data is no more required for training, as long as we can obtain unmatched reference data of high spatial resolution.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution

Cha,

Chung,

Kim

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k-space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-off. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k-space reference data for supervised training, which is not suitable for tMRA. This is because high spatio-temporal resolution ground-truth images are not available for tMRA. To address this problem, here we propose a novel unsupervised deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler and improves the performance. Reconstruction results using in vivo tMRA data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.

show abstract

Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data

Yaman

Hosseini

Moeller

et al. 2020

Magnetic Resonance in Med

199

227

View full text Add to dashboard Cite

Purpose To develop a strategy for training a physics‐guided MRI reconstruction neural network without a database of fully sampled data sets. Methods Self‐supervised learning via data undersampling (SSDU) for physics‐guided deep learning reconstruction partitions available measurements into two disjoint sets, one of which is used in the data consistency (DC) units in the unrolled network and the other is used to define the loss for training. The proposed training without fully sampled data is compared with fully supervised training with ground‐truth data, as well as conventional compressed‐sensing and parallel imaging methods using the publicly available fastMRI knee database. The same physics‐guided neural network is used for both proposed SSDU and supervised training. The SSDU training is also applied to prospectively two‐fold accelerated high‐resolution brain data sets at different acceleration rates, and compared with parallel imaging. Results Results on five different knee sequences at an acceleration rate of 4 shows that the proposed self‐supervised approach performs closely with supervised learning, while significantly outperforming conventional compressed‐sensing and parallel imaging, as characterized by quantitative metrics and a clinical reader study. The results on prospectively subsampled brain data sets, in which supervised learning cannot be used due to lack of ground‐truth reference, show that the proposed self‐supervised approach successfully performs reconstruction at high acceleration rates (4, 6, and 8). Image readings indicate improved visual reconstruction quality with the proposed approach compared with parallel imaging at acquisition acceleration. Conclusion The proposed SSDU approach allows training of physics‐guided deep learning MRI reconstruction without fully sampled data, while achieving comparable results with supervised deep learning MRI trained on fully sampled data.

show abstract

Optimal Transport driven CycleGAN for Unsupervised Learning in Inverse Problems

Cited by 17 publications

References 38 publications

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Model-based Reconstruction with Learning: From Unsupervised to Supervised and Beyond

Unsupervised Deep Learning for MR Angiography with Flexible Temporal Resolution

Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data

Contact Info

Product

Resources

About