Optimal Transport Structure of CycleGAN for Unsupervised Learning for Inverse Problems

Abstract

The penalized least squares (PLS) is a classic approach to inverse problems, where a regularization term is added to stabilize the solution. Optimal transport (OT) is another mathematical framework for computer vision tasks by providing means to transport one measure to another at minimal cost. Cycle-consistent generative adversarial network (cycleGAN) is a recent extension of GAN to learn target distributions with less mode collapsing behavior. Although similar in that no supervised training is required, the algorithms look different, so the mathematical relationship between these approaches is not clear. In this article, we provide an important advance to unveil the missing link. Specifically, we reveal that a cycle-GAN architecture can be derived as a dual formulation of the optimal transport problem, if the PLS with a deep learning penalty is used as a transport cost between the two probability measures from measurements and unknown images. This suggests that cycleGAN can be considered as stochastic generalization of classical PLS approaches. Our derivation is so general that various types of cy-cleGAN architecture can be easily derived by merely changing the transport cost. As proofs of concept, this paper provides novel cycleGAN architecture for unsupervised learning in accelerated MRI and deconvolution microscopy problems, which confirm the efficacy and the flexibility of the theory. * Jong Chul Ye is the corresponding author of this paper.

Self‐supervised learning of physics‐guided reconstruction neural networks without fully sampled reference data

“…An alternative approach, which assumes the same data are acquired with two separate acquisitions using different undersampling patterns, was also proposed, 63,64 extending on the Noise2Noise denoising framework 65 . In the same image‐domain reconstruction setting, a self‐supervised learning scheme using cycleGANs with optimal transport cost minimization was proposed, 66 although initial results exhibit blurring artifacts. Although purely data‐driven image domain methods have been used for DL‐MRI reconstruction, physics‐guided DL‐MRI techniques are more desirable, as they offer a degree of interpretability by incorporating domain knowledge on the MRI encoding mechanism 20,27,28,30,31,33 .…”

Fine-Grained Mri Reconstruction Using Attentive Selection Generative Adversarial Networks

“…We compute a consistency loss in k-space as it was also suggested in [9]. An alternative is to apply it in image domain as in [16]. We found that the two methods are not fundamentally different and in simulations they behave very similarly.…”

“…2. Comparison of 8× accelerated MRI reconstructions; first) zero-filled, second) CycleGAN-unsup [16], third) LCFI++ (ours) with half of the dataset, fourth) MICCAN [17], fifth) CycleGAN-sup [16], sixth) LCFI++ (ours), and last) ground truth. [17].…”

“…[17]. We implement the method proposed in [16], denoted by CycleGANunsup, which trains a CycleGAN [8] to recover the measurements in an "unsupervised" manner. We also present the outputs of the CycleGAN-based framework trained under supervision (CycleGAN-sup), i.e.…”

“…The method in [6] performs MRI reconstruction by jointly optimizing the latent space of a pre-trained GAN to enforce it being in agreement with the measurements and tuning the parameters of the model to leverage the deep image prior [7]. Motivated by the previous works in domain translation [8], concurrent researches using cyclic adversarial frameworks include: Refine-GAN in [9] which applies the cyclic consistency to learn the mapping of the under-sampled measurements, and [10] which utilizes the theory of optimal transport to provide a foundation for cycle-consistent GAN (CycleGAN).…”