DUAL-GLOW: Conditional Flow-Based Generative Model for Modality Transfer

Abstract

Positron emission tomography (PET) imaging is an imaging modality for diagnosing a number of neurological diseases. In contrast to Magnetic Resonance Imaging (MRI), PET is costly and involves injecting a radioactive substance into the patient. Motivated by developments in modality transfer in vision, we study the generation of certain types of PET images from MRI data. We derive new flow-based generative models which we show perform well in this small sample size regime (much smaller than dataset sizes available in standard vision tasks). Our formulation, DUAL-GLOW, is based on two invertible networks and a relation network that maps the latent spaces to each other. We discuss how given the prior distribution, learning the conditional distribution of PET given the MRI image reduces to obtaining the conditional distribution between the two latent codes w.r.t. the two image types. We also extend our framework to leverage "side" information (or attributes) when available. By controlling the PET generation through "conditioning" on age, our model is also able to capture brain FDG-PET (hypometabolism) changes, as a function of age. We present experiments on the Alzheimers Disease Neuroimaging Initiative (ADNI) dataset with 826 subjects, and obtain good performance in PET image synthesis, qualitatively and quantitatively better than recent works.

Conditional Invertible Neural Networks for Diverse Image-to-Image Translation

“…Despite the relatively small training set, we see little signs of overfitting, and the model generalizes well to the test set. Previously, [31] also found low overfitting and good generalization on small training sets using INNs. Several samples by the model are shown in Fig.…”

“…A special type of conditional latent space is demonstrated in [2], suitable for non-stochastic inverse problems of small dimensionality. Examples where the approach is extended to spatial conditioning include [31], where two separate INNs define a mapping between medical imaging domains. The model requires an additional loss term with hyperparameters, that has an unknown effect on the output distribution, and diversity is not considered.…”

Attribute Decomposition for Flow-Based Domain Mapping

“…In [20], the domain mapping based on a guided conditional image was performed for color transfer. In medical domains, there have been a number of applications where flow model was developed for medical image tasks, such as vessel segmentation [21] and image transfer from magnetic resonance imaging to positron emission tomography [14].…”

“…However, GAN was hard to converge due to minimax optimization. Recently, the flow model [12] has achieved a great success as a generative model for computer vision [13,14,15,16], natural language processing [17,18] and speech processing [19]. However, it was difficult to implement the conditional generation based on flow model since there was no reconstruction mechanism for generation phase during training process.…”

“…An alternative way was to indirectly use post-processing scheme to impose labels for semantic control [15,13]. In [14], dual model learning was exploited for flow-based transfer learning. The guided images were used as the conditions for flow-based image coloring [20] or edge detection [21].…”