Image Generation Using Continuous Conditional Generative Adversarial Networks

Abstract

Continuous Conditional Generative Modeling (CCGM) aims to estimate the distribution of high-dimensional data, typically images, conditioned on scalar continuous variables known as regression labels. While Continuous conditional Generative Adversarial Networks (CcGANs) were initially designed for this task, their adversarial training mechanism remains vulnerable to extremely sparse or imbalanced data, resulting in suboptimal outcomes. To enhance the quality of generated images, a promising alternative is to replace CcGANs with Conditional Diffusion Models (CDMs), renowned for their stable training process and ability to produce more realistic images. However, existing CDMs encounter challenges when applied to CCGM tasks due to several limitations such as inadequate U-Net architectures and deficient model fitting mechanisms for handling regression labels. In this paper, we introduce Continuous Conditional Diffusion Models (CCDMs), the first CDM designed specifically for the CCGM task. CCDMs address the limitations of existing CDMs by introducing specially designed conditional diffusion processes, a modified denoising U-Net with a custom-made conditioning mechanism, a novel hard vicinal loss for model fitting, and an efficient conditional sampling procedure. With comprehensive experiments on four datasets with varying resolutions ranging from 64 × 64 to 192 × 192, we demonstrate the superiority of the proposed CCDM over state-of-the-art CCGM models, establishing new benchmarks in CCGM. Extensive ablation studies validate the model design and implementation configuration of the proposed CCDM. Our code is publicly available at https://github.com/UBCDingXin/CCDM.

Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classification

“…-SCHIZCONNECT-VIP 8 It comprises N = 605 multi-site MRI scans including 275 patients with strict schizophrenia (SCZ) and 330 HC.…”

“…Inspired by vicinal risk minimization (VRM) [3], we propose to re-define p(v 1 , v 2 ) by integrating the proxy metadata y, modeled as a random variable, such that a small change in y results in a negligible change in p(v 1 , v 2 |y). Similarly to [8], we define the empirical joint distribution as:…”

Point Cloud Generation with Continuous Conditioning

“…Using continuous conditions is a mathematically different problem than solving categorical conditioning problems, such as classification (Ding et al, 2021). First, there may be few or no real samples for some regression labels and second, conventional label input methods, i.e.…”

“…one-hot encoding, is not possible for an infinite number of regression labels. CcGAN (Ding et al, 2021) is first to introduce a continuous conditional GAN for image generation. They solve the aforementioned problems by introducing a new GAN loss and a novel way to input the labels based on label projection (Miyato and Koyama, 2018).…”

“…These properties can for example describe the aspect-ratios of the object, such that the GAN generates a custom-fit shape while maintaining generation quality and diversity. Using continuous conditions introduces new challenges, since it is a mathematically different problem than solving categorical conditioning problems, such as classification (Ding et al, 2021). The challenges arise from an infinitely large set of parameters and conditioning regions where no training data samples exist.…”